Data Pipeline Architect

4 semanas atrás


Brasil beBeeDataEngineering Tempo inteiro
Data Engineering Specialist

Our organization is seeking a skilled and experienced Data Engineer to join our Threat Research team. The primary responsibility of this role will be to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows.

Responsibilities:
  • Design, develop, and maintain data pipelines for ingesting threat intelligence data from various sources into our data ecosystem.
  • Implement data validation processes to ensure data accuracy, completeness, and consistency.
  • Collaborate with threat analysts to understand data requirements and design appropriate solutions.
  • Develop automation scripts and workflows for data export processes to external systems or partners.
  • Optimize and enhance existing data pipelines for improved performance and scalability.
  • Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity.
  • Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
  • Stay updated on emerging technologies and best practices in data engineering and incorporate them into our data ecosystem.
  • Provide technical guidance and support to other team members on data engineering best practices and methodologies.
Requirements:
  • Proven experience as a Data Engineer or similar role, with a focus on data ingest, validation, and export automation.
  • Strong proficiency in Python.
  • Experience with data pipeline orchestration tools such as Apache Airflow, Apache NiFi, or similar.
  • Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform.
  • Experience with data validation techniques and tools for ensuring data quality.
  • Experience building and deploying images using containerization technologies such as Docker and Kubernetes.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.

  • Data Architect

    4 semanas atrás


    Brasil beBeeDataEngineering Tempo inteiro

    Job Title: Data ArchitectWe are seeking a skilled Data Architect to build and maintain our data pipelines. This role is critical in processing large-scale regulatory data worldwide, transforming usable datasets for downstream applications and APIs.

  • Data Pipeline Expert

    4 semanas atrás


    Brasil beBeeData Tempo inteiro

    Expert Data Pipeline Architect We are seeking a seasoned expert to craft and maintain scalable data pipelines on AWS, ensuring optimal performance, quality, and security. Key Responsibilities: Develop and optimize Extract-Transform-Load (ETL) pipelines with AWS Glue. Collaborate with data scientists and analysts to integrate data from multiple sources and...

  • Data Architect

    4 semanas atrás


    Brasil beBeeData Tempo inteiro

    Job Title: Lead Data Architect Key Responsibilities: The project focuses on developing and implementing advanced data architectures to solve complex business problems. Designing scalable data systems for the ingestion, processing, and storage of large datasets in a cloud environment. Building performant data pipelines to transform raw data into actionable...


  • Brasil beBeeDataEngineer Tempo inteiro

    Transforming global data to drive business forward. A strategic Data Engineer is required to build, maintain, and modernize complex data pipelines processing large-scale regulatory information from around the world. The successful candidate will work with Python, SQL, and related tools to untangle legacy code, migrate processes to maintainable platforms like...


  • Brasil CI&T Tempo inteiro R$90.000 - R$120.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality. At CI&T, we are seeking a highly skilled and motivated Systems Data Architect...

  • Chief Data Architect

    4 semanas atrás


    Brasil beBeeDataEngineer Tempo inteiro

    Project Overview We are seeking an experienced Data Engineer to join our team. The primary goal of this project is to design and implement a solution for processing big data volumes, integrating it with the current architecture. The successful candidate will be responsible for designing and implementing a robust and scalable ETL/ELT pipeline, extracting,...


  • Brasil VDart Digital Tempo inteiro

    Data Architect with Trade Promotion Experience Remote Long term Contract Key Accountabilities / Responsibilities Create a data-connected ecosystem that integrates various data sources to provide comprehensive insights for trade promotion optimization. Stablish and promote best practices for data management, analytics, and trade promotion strategies. Ensure...

  • Senior Data Architect

    1 dia atrás


    Índio do Brasil VDart Digital Tempo inteiro

    Data Architect with Trade Promotion Experience Remote Long term Contract Key Accountabilities / Responsibilities - Create a data-connected ecosystem that integrates various data sources to provide comprehensive insights for trade promotion optimization. - Stablish and promote best practices for data management, analytics, and trade promotion...


  • Brasil Tata Consultancy Services Tempo inteiro

    Role Overview : We are is seeking a hands-on Azure Platform Engineer/Architect with strong architectural knowledge of Azure Paa S services and Dev Ops skills. This person will work closely with backend engineers and automation teams to design, implement, and deploy cloud-native solutions on Azure. The role requires expertise in Azure services,...

  • Senior Data Architect

    4 semanas atrás


    Brasil beBeeDataEngineer Tempo inteiro

    Experience a career that truly lasts a lifetime with our technology consulting firm. We're dedicated to supporting our employees' professional growth and offering an exceptional work environment that fosters meaningful connections and shared success. With over 12 years of experience in the market, we've built a strong presence across 20+ countries, employing...