Senior Data Pipeline Developer

4 semanas atrás


Brasil beBeeDataEngineer Tempo inteiro

**Data Engineer Role Overview**

We are seeking a highly skilled and experienced data engineer to join our team. The primary responsibility of this role is to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows.

Our ideal candidate will have a strong background in designing, developing, and maintaining large-scale data pipelines, as well as experience with data validation techniques and tools for ensuring data quality.

The successful candidate will also be proficient in programming languages such as Python and have experience with data pipeline orchestration tools such as Apache Airflow or similar.

Key Responsibilities:
  • Design and develop scalable data pipelines for ingesting threat intelligence data from various sources into our data ecosystem.
  • Implement data validation processes to ensure data accuracy, completeness, and consistency.
  • Collaborate with threat analysts to understand data requirements and design appropriate solutions.
  • Develop automation scripts and workflows for data export processes to external systems or partners.
  • Optimize and enhance existing data pipelines for improved performance and scalability.
  • Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity.
  • Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
Requirements:
  • Proven experience as a Data Engineer or similar role, with a focus on data ingest, validation, and export automation.
  • Strong proficiency in Python programming language.
  • Experience with data pipeline orchestration tools such as Apache Airflow or similar.
  • Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform.
  • Experience with data validation techniques and tools for ensuring data quality.
  • Experience building and deploying images using containerization technologies such as Docker and Kubernetes.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.


  • Brasil CI&T Software S.A. Tempo inteiro

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality. At CI&T, we are seeking a highly skilled and motivated Senior Data Developer to join...


  • Brasil CI&T Software S.A. Tempo inteiro

    Position overview:As a Senior Data Developer, you will design and implement scalable data solutions that drive business value for our global clients. Working with modern data stacks in cloud environments, you'll develop robust data pipelines, ensure data quality, and implement effective data governance practices. Your expertise will be crucial in...


  • Brasil CI&T Tempo inteiro R$90.000 - R$120.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality. Hi there This is Jessica from CI&T I am a Talent Analyst looking for...


  • Brasil CI&T Tempo inteiro R$90.000 - R$120.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality. At CI&T, we are seeking a highly skilled and motivated Senior Data Developer to...

  • Data Pipeline Architect

    3 semanas atrás


    Brasil beBeeDataEngineering Tempo inteiro

    Data Engineering Specialist Our organization is seeking a skilled and experienced Data Engineer to join our Threat Research team. The primary responsibility of this role will be to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows. Responsibilities: Design, develop, and maintain data...


  • Brasil CI&T Tempo inteiro R$80.000 - R$120.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality. Hey everyone This is Thai Viapiana from CI&T's Talent Attraction Team :) I am...

  • Data Pipeline Expert

    3 semanas atrás


    Brasil beBeeData Tempo inteiro

    Expert Data Pipeline Architect We are seeking a seasoned expert to craft and maintain scalable data pipelines on AWS, ensuring optimal performance, quality, and security. Key Responsibilities: Develop and optimize Extract-Transform-Load (ETL) pipelines with AWS Glue. Collaborate with data scientists and analysts to integrate data from multiple sources and...


  • Brasil beBeeDataEngineer Tempo inteiro

    Transforming global data to drive business forward. A strategic Data Engineer is required to build, maintain, and modernize complex data pipelines processing large-scale regulatory information from around the world. The successful candidate will work with Python, SQL, and related tools to untangle legacy code, migrate processes to maintainable platforms like...

  • Senior Data Developer

    4 semanas atrás


    Brasil CI&T Software S.A. Tempo inteiro

    Somos especialistas em transformação tecnológica , unindo expertise humana à IA para criar soluções tech escaláveis. Com mais de 7 .400 CI&Ters ao redor do mundo, já formamos parcerias com mais de 1.000 clientes durante nossos 30 anos de história. Inteligência Artificial é nossa realidade. Importante : se você reside na Região Metropolitana de...

  • Senior Full Stack Developer

    3 semanas atrás


    Brasil beBeeSoftware Tempo inteiro

    Expert Software Developer Opportunity ">Our mission is to drive innovation and progress in the food and agricultural industry by providing technical expertise, testing, and certification services globally. We strive for excellence in ensuring food safety, quality, and sustainability across international supply chains. About the Role ">We are seeking an...