Data Engineer

1 semana atrás

São Paulo, São Paulo, Brasil Partner One Capital Tempo inteiro

This is a 100% fully remote position

We are seeking a skilled and experienced Data Engineer to join our Threat Research team. The primary responsibility of this role will be to design, develop, and maintain data pipelines for threat intelligence ingestion, validation, and export automation flows. The ideal candidate will have a strong background in data engineering, experience with data validation techniques, and proficiency in building scalable and efficient data pipelines. The candidate must have extensive experience with SQL, data orchestration tools, Python, and SEIM platforms, with a strong preference for experience in threat intelligence or other cybersecurity applications. Snowflake, AWS, Azure, or Google Bigtable experience is strongly preferred.

Responsibilities:

  • Design, develop, and maintain data pipelines for ingesting threat intelligence data from various sources into our data ecosystem.
  • Implement data validation processes to ensure data accuracy, completeness, and consistency.
  • Collaborate with threat analysts to understand data requirements and design appropriate solutions.
  • Develop automation scripts and workflows for data export processes to external systems or partners.
  • Optimize and enhance existing data pipelines for improved performance and scalability.
  • Monitor data pipelines and troubleshoot issues as they arise, ensuring continuous data availability and integrity.
  • Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
  • Stay updated on emerging technologies and best practices in data engineering and incorporate them into our data ecosystem.
  • Provide technical guidance and support to other team members on data engineering best practices and methodologies.

Requirements

  • Bachelor's degree in Computer Science, Engineering, or related field.
  • Proven experience as a Data Engineer or similar role, with a focus on data ingest, validation, and export automation.
  • Strong proficiency in Python.
  • Experience with data pipeline orchestration tools such as Apache Airflow, Apache NiFi, or similar.
  • Solid understanding of database systems (relational and NoSQL), data modeling, and SQL. Must demonstrate a strong understanding of SQL queries.
  • Familiarity with cloud platforms such as Snowflake, AWS, Azure, or Google Cloud Platform. Experience with Snowflake is preferred.
  • Experience with data validation techniques and tools for ensuring data quality.
  • Experience building and deploying images using containerization technologies such as Docker and Kubernetes.
  • Strong understanding of Apache Avro or JSON schema validation.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.

  • Senior Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Fusemachines Tempo inteiro

    We are on the lookout for an enthusiastic Senior Data Engineer to join our exceptional team at Fusemachines in São Paulo, SP.Growing your career as a Full Time Senior Data Engineer is an awesome opportunity to develop useful skills.If you are strong in persuasion, critical thinking and have the right experience for the job, then apply for the position of...

  • Data Engineer

    2 semanas atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil Wipro Tempo inteiro

    We are one of the largest and leading global technology services, consulting and corporate processes companies in the world. We are present in more than 69 countries and our team is composed of more than 220,000 employees working on six continents. We are technological partners of the largest companies in the world. We proudly serve more than 1500 customers...

  • Data Engineer

    4 semanas atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil Wipro Tempo inteiro

    We are one of the largest and leading global technology services, consulting and corporate processes companies in the world. We are present in more than 69 countries and our team is composed of more than 220,000 employees working on six continents. We are technological partners of the largest companies in the world. We proudly serve more than 1500 customers...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Wipro Tempo inteiro

    We are one of the biggest players in the global technology services, consulting, and corporate processes arena. We have a widespread presence across 69+ countries, boasting a team of over 220,000 professionals operating across six continents. Our clientele includes top-tier global corporations, and we cater to 1500+ customers with an extensive portfolio that...

  • Data Science Engineer

    1 semana atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil KPIT Tempo inteiro

    WeKPIT (are a global technologies company specializing inCASE (Connected, Autonomous, Shared, Electric)domains. Systems and Software in Electric & Conventional Powertrain, Autonomous Driving & ADAS, Digital Connected Solutions, Connected Vehicles, Vehicle Diagnostics, and Vehicle Networks. Join the leading software development and integration team helping...

  • Sr. Data Engineer

    Há 1 mês

    São Paulo, São Paulo, Estado de São Paulo, Brasil Amazing Devs Tempo inteiro

    Senior Data Engineer 5+ Years of Experience International job opportunity!! Only for Brazilians R$ 19.000,00 como PJ (dólar a R$5,00) Project Duration = 3 - 6 mounth FULL TIME Required 5+ Years of Experience English - Advanced Snowflake Python AWS SQL ETL Description: As a Data Engineer, you will play a pivotal role in ensuring a seamless transition from...

  • Sr. Data Engineer

    3 semanas atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil Amazing Devs Tempo inteiro

    Senior Data Engineer 5+ Years of Experience International job opportunity!! Only for Brazilians R$ 19.000,00 como PJ (dólar a R$5,00) Project Duration = 3 - 6 mounth FULL TIME Required 5+ Years of Experience English - Advanced Snowflake Python AWS SQL ETL Description: As a Data Engineer, you will play a pivotal role in ensuring a seamless transition from...

  • Data Engineer

    Há 7 dias

    São Paulo, São Paulo, Estado de São Paulo, Brasil Macarta Tempo inteiro

    Macarta was founded in 2016 by a small team of hungry professionals with an eye for the future of e-Commerce. Today, Macarta is one of the fastest growing companies in America, with 100+ employees around the globe and growing. Our business model consists of a full-service retail media agency, providing performance-driven management consulting to global...

  • Software Engineer

    Há 7 dias

    São Paulo, São Paulo, Brasil Data Meaning Tempo inteiro

    Software Engineer - LLM Powered Apps Location:BrazilPosition Type: Short Term - 3 monthsPosition Summary Data Meaning is a front-runner in Business Intelligence and Data Analytics consulting, renowned for our high-quality consulting services throughout the US and LATAM. Our expertise lies in delivering tailored solutions in Business Intelligence, Data...

  • Data Engineer

    2 meses atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil CriticalRiver Inc. Tempo inteiro

    Title: Data Engineer Location: Sao Paulo Brazil Remote: Need to Support in USA PST time zone Duration: Fulltime Interview process: Live Hacker rank test on Advanced SQL questions. Technical screening Client live Hacker rank interview. Summary : Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing...

  • Data Engineer

    3 semanas atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil CriticalRiver Inc. Tempo inteiro

    Title: Data Engineer Location: Sao Paulo Brazil Remote: Need to Support in USA PST time zone Duration: FulltimeInterview process: Live Hacker rank test on Advanced SQL questions. Technical screening Client live Hacker rank interview.Summary : Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing...

  • Data Engineer

    3 semanas atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil CriticalRiver Inc. Tempo inteiro

    Title: Data Engineer Location: Sao Paulo Brazil Remote: Need to Support in USA PST time zone Duration: Fulltime Interview process: Live Hacker rank test on Advanced SQL questions. Technical screening Client live Hacker rank interview. Summary : Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Estado de São Paulo, Brasil CriticalRiver Inc. Tempo inteiro

    Title: Data EngineerLocation: Sao Paulo Brazil Remote: Need to Support in USA PST time zone Duration: FulltimeInterview process: Live Hacker rank test on Advanced SQL questions. Technical screening Client live Hacker rank interview.Summary: Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil BRP Tempo inteiro

    JOB DESCRIPTION DATA ENGINEER WHY JOIN THE TEAM? Be a part of a fast-paced culture and community in a startup-like environment inside one of the most successful organizations in Québec. We are focused on building a highly talented and ambitious team from the ground up that will revolutionize the way we work around Data & Analytics at BRP. We are a...

  • Process Engineer, Product Management

    1 semana atrás

    São Paulo, São Paulo, Brasil The Chemical Engineer Tempo inteiro

    Ethicon, a member of the Johnson & Johnson Family of Companies, is recruiting for a Process Engineer, Product Management , to be based in São José dos Campos, Brazil.This role will support the execution of the strategy to drive the code-level product changes across overlapping projects across key strategic programs, such as Sapphire, Protect, Vantage,...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil CriticalRiver Inc. Tempo inteiro

    Title: Data EngineerLocation: Sao Paulo BrazilRemote: Need to Support in USA PST time zoneDuration: Fulltime Interview process:Live Hacker rank test on Advanced SQL questions.Technical screeningClient live Hacker rank interview. Summary:Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing database...

  • Data Lead Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil CAPCO Tempo inteiro

    Data Lead Engineer at Capco Brazil - Sao Paulo SOBRE A CAPCO A Capco é uma consultoria global de tecnologia e negócios, focada no setor de serviços financeiros e energia, somos apaixonados por ajudar nossos clientes, temos 33 escritórios nas Américas, Europa e Ásia-Pacífico Se criatividade e inovação são sua paixão, a Capco é ideal para...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Wipro Technologies Tempo inteiro

    We are one of the largest and leading global technology services, consulting and corporate processes companies in the world. We are present in more than 69 countries and our team is composed of more than 220,000 employees working on six continents. We are technological partners of the largest companies in the world. We proudly serve more than 1500 customers...

  • Senior Data Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil Cognizant Tempo inteiro

    Who We Are Cognizant (Nasdaq-100: CTSH) engineers modern businesses. We help our clients modernize technology, reimagine processes and transform experiences so they can stay ahead in our fast-changing world. Together, we're improving everyday life. See how at: Senior Data Engineer - Kafka What you'll do Your mission as a Senior Data Engineer – Kafka...

  • Senior Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Cognizant Tempo inteiro

    Who We Are Cognizant (Nasdaq-100: CTSH) engineers modern businesses. We help our clients modernize technology, reimagine processes and transform experiences so they can stay ahead in our fast-changing world. Together, we're improving everyday life. See how at: Senior Data Engineer - Kafka What you'll do Your mission as a Senior Data Engineer – Kafka...