Data Engineer

Há 6 dias


Brasília, Distrito Federal, Brasil Fidelis Security Tempo inteiro

We are seeking a skilled and experienced Data Engineer to join our Threat Research team. The primary responsibility of this role will be to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows. The ideal candidate will have a strong background in data engineering, experience with data validation techniques, and proficiency in building scalable and efficient data pipelines. The candidate must have extensive experience with SQL, data orchestration tools, Python, and SEIM platforms, with a strong preference for experience in threat intelligence or other cybersecurity applications. Snowflake, AWS, Azure, or Google Bigtable experience is strongly preferred.

Responsibilities :

  • Design, develop, and maintain data pipelines for ingesting threat intelligence data from various sources into our data ecosystem.
  • Implement data validation processes to ensure data accuracy, completeness, and consistency.
  • Collaborate with threat analysts to understand data requirements and design appropriate solutions.
  • Develop automation scripts and workflows for data export processes to external systems or partners.
  • Optimize and enhance existing data pipelines for improved performance and scalability.
  • Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity.
  • Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
  • Stay updated on emerging technologies and best practices in data engineering and incorporate them into our data ecosystem.
  • Provide technical guidance and support to other team members on data engineering best practices and methodologies.

Requirements:

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • Proven experience as a Data Engineer or similar role, with a focus on data ingest, validation, and export automation.
  • Strong proficiency in Python.
  • Experience with data pipeline orchestration tools such as Apache Airflow, Apache NiFi, or similar.
  • Solid understanding of database systems (relational and NoSQL), data modeling, and SQL. Must demonstrate a strong understanding of SQL queries.
  • Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform. Experience with Snowflake is preferred.
  • Experience with data validation techniques and tools for ensuring data quality.
  • Experience building and deploying images using containerization technologies such as Docker and Kubernetes.
  • Strong understanding of Apache Avro or JSON schema validation.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.

  • Data Engineer

    Há 7 dias


    Brasília, Distrito Federal, Brasil Emotive Tempo inteiro

    We are looking for an entrepreneurial Data Engineer to join our team at Emotive. This person has to love the startup grind and hustle of getting things done quickly, but also without sacrificing the longevity of the technology. We are looking for true owners who are looking to build not only new and interesting technology, but also a world class technology...

  • Lead Data Engineer

    Há 7 dias


    Brasília, Distrito Federal, Brasil Mindera - Brazil Tempo inteiro

    Join our amazing team at Mindera! We are on the lookout for skilled engineers who enjoy tackling challenging problems and crafting top-notch user experiences.As a Data Engineer with us, you'll play a key role in constructing, enhancing, and connecting large-scale data platforms.This position is tailor-made for individuals enthusiastic about all things Data,...

  • Sr. Data Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil Amazing Devs Tempo inteiro

    Senior Data Engineer 5+ Years of Experience International job opportunity Only for BraziliansR$ 19.000,00 como PJ (dólar a R$5,00)Project Duration = - 6 mounth Required5+ Years of ExperienceEnglish - Advanced SQL ETL Description:As a Data Engineer, you will play a pivotal role in ensuring a seamless transition from our existing transactional system to the...

  • Sr. Data Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil Amazing Devs Tempo inteiro

    Senior Data Engineer 5+ Years of ExperienceInternational job opportunity Only for BraziliansR$ 19.000,00 como PJ (dólar a R$5,00)Project Duration = - 6 mounth FULL TIME Required5+ Years of ExperienceEnglish - Advanced Snowflake Python AWS SQL ETLDescription:As a Data Engineer, you will play a pivotal role in ensuring a seamless transition from our existing...

  • Sr. Data Engineer

    Há 7 dias


    Brasília, Distrito Federal, Brasil Amazing Devs Tempo inteiro

    Senior Data Engineer 5+ Years of Experience International job opportunity Only for BraziliansR$ 19.000,00 como PJ (dólar a R$5,00)Project Duration = - 6 mounth FULL TIME Required5+ Years of ExperienceEnglish - Advanced Snowflake Python AWS SQL ETL Description:As a Data Engineer, you will play a pivotal role in ensuring a seamless transition from our...


  • Brasília, Distrito Federal, Brasil Questrade Financial Group Tempo inteiro

    Questrade Financial Group (QFG) of Companies is committed to helping our customers become much more financially successful and secure. We are everything a traditional financial institution is not. At QFG, you will be constantly moving forward, bringing the future of fintech into existence. You will be a part of a collaborative team that cares deeply about...


  • Brasília, Distrito Federal, Brasil HCLTech Tempo inteiro

    Data Engineer Senior , HCLTech São Paulo, São Paulo (Remoto) / Tempo integral We are looking for a highly talented and self-motivated Data Engineer Senior to join us on our journey to advance the technology world through innovation and creativity. Essential Duties and Responsibilities Drive requirements, scope, and technical design of the...


  • Brasília, Distrito Federal, Brasil HCLTech Tempo inteiro

    Data Engineer Senior , HCLTechSão Paulo, São Paulo (Remoto) / Tempo integralWe are looking for a highly talented and self-motivated Data Engineer Senior to join us on our journey to advance the technology world through innovation and creativity.Essential Duties and ResponsibilitiesDrive requirements, scope, and technical design of the integration...

  • Data Engineer

    Há 7 dias


    Brasília, Distrito Federal, Brasil Wipro Tempo inteiro

    We are one of the largest and leading global technology services, consulting and corporate processes companies in the world. We are present in more than 69 countries and our team is composed of more than 220,000 employees working on six continents. We are technological partners of the largest companies in the world. We proudly serve more than 1500 customers...


  • Brasília, Distrito Federal, Brasil buscojobs Brasil Tempo inteiro

    Como é o ambiente de trabalho? Somos a Novatics, uma empresa que combina expertise em software, design e inovação.Desenvolvemos plataformas para impulsionar o crescimento e a evolução em todas as áreas relacionadas ao que é humano, com a missão de tornar realidade hoje os planos que muitos pensavam só serem possíveis no futuro. Nossa equipe,...


  • Brasília, Distrito Federal, Brasil Solvd, Inc. Tempo inteiro

    About the jobOur client is a widely-known American company that is working on tools and advice for you to make it easy to pay off debt, choose the best of the best financial products and services (insurance, credit cards, etc.), and tackle your major life goals (saving for retirement, buying a house, etc.). The Data Engineering Manager will play a critical...


  • Brasília, Distrito Federal, Brasil System Soft Technologies Tempo inteiro

    Job Description: Machine Learning EngineerLooking for a skilled and talented Machine Learning Engineer to join our team. You'll use your expertise in machine learning and data analysis to develop cutting-edge models and algorithms, extracting valuable insights from diverse data. Be a key player in our digital transformation, influencing the future of...

  • Backend Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil nOps Tempo inteiro

    About the CompanyAt nOps, we envision a world where DevOps and Engineering teams can take control of their cloud costs, so they only pay for what they use - not what's provisioned. nOps's ML-powered cloud optimization platform processes over $1.5 billion dollars of cloud spend — and we're just getting started. Our platform helps Engineering and DevOps...

  • Software Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil Etleap Tempo inteiro

    Software Engineer Integrations Etleap came to be out of the frustration with how much time data wrangling takes away from the actual analysis. We were just tired of spending time building and maintaining data pipelines. Then we noticed, so is everyone else That is why we've created an intuitive ETL tool that easily enables the data analysts themselves to...


  • Brasília, Distrito Federal, Brasil Wipro Tempo inteiro

    We are one of the largest and leading global technology services, consulting and corporate processes companies in the world. We are present in more than 69 countries and our team is composed of more than 220,000 employees working on six continents. We are technological partners of the largest companies in the world. We proudly serve more than 1500 customers...


  • Brasília, Distrito Federal, Brasil System Soft Technologies Tempo inteiro

    Job Description: Machine Learning EngineerClient is seeking a highly skilled and talented Machine Learning Engineer to join our innovative team. In this role, you will apply your expertise in machine learning and data analysis techniques to develop and deploy cutting-edge models and algorithms that enable us to extract valuable insights from complex and...


  • Brasília, Distrito Federal, Brasil System Soft Technologies Tempo inteiro

    Job Summary:The client is looking for a Machine Learning Engineer:ML Engineer- working with data science team. Helping automate things for the data science team to get things to the cloud. Testing is a skill they need. Batch scripting, automated testing will be a daily activity. Help teams deploy to the cloud.Responsibilities:Study and transform data science...

  • Data Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil Dexian Brasil Tempo inteiro

    Quem SomosA Dexian, lançada em 2023, tem presença global e traz consigo quase 30 anos de experiência através de suas companhias legadas, principalmente da combinação da DISYS e Signature Consultants.Iniciamos no Brasil em 2007 como DISYS e, desde então, conquistamos mais de 60 clientes de diferentes setores da economia.Como Dexian, nos tornamos uma...

  • Data Engineer

    Há 6 dias


    Brasília, Distrito Federal, Brasil Dexian Brasil Tempo inteiro

    Quem SomosA Dexian, lançada em 2023, tem presença global e traz consigo quase 30 anos de experiência através de suas companhias legadas, principalmente da combinação da DISYS e Signature Consultants.Iniciamos no Brasil 2007 como DISYS e, de lá para cá, conquistamos mais de 60 clientes de diferentes setores da economia. Como Dexian, passamos a ser uma...

  • Backend Engineer

    Há 7 dias


    Brasília, Distrito Federal, Brasil aoro Tempo inteiro

    Backend Engineer, Systems APIs, Databases, Event-Driven architecture. Systems, Architecture, Kafka, GraphQL, Apollo, PostgreSQL, Cryptography, Microservices, Messaging Systems, Payment Platforms, CI/CD, API Development, Synchronous systems. Overview: Join us as a Backend Engineer with a specialization in cutting-edge technologies including GraphQL,...