AWS Data Engineer

2 semanas atrás

Remoto, Brasil Zummit Tempo inteiro R$120.000 - R$240.000 por ano

Project Description

We are looking for an experienced AWS data engineer, who will join the team responsible for the second phase of data warehouse cloud migration project. The initial "lift-and-shift" of an on-premise MS SQL data warehouse to AWS RDS has been completed. This next phase focuses on refactoring the legacy system into a modern, serverless data architecture. The primary goal is to migrate all data transformation logic from MS SQL to AWS Glue and prepare the data for consumption by business intelligence tools like AWS QuickSight.

Key Responsibilities

  • Analyze existing MS SQL stored procedures and data structures within the PMM data warehouse to understand the current business and transformation logic.
  • Design, develop, and deploy scalable and efficient ETL/ELT pipelines using AWS Glue and PySpark to replace the legacy SQL processes.
  • Ingest and process data from various sources currently landing in Amazon S3, including SAP extracts, operational system feeds, and flat files.
  • Implement robust data validation, error handling, and logging mechanisms within Glue jobs.
  • Collaborate with architects to define data models and schemas suitable for the new data lakehouse architecture.
  • Optimize Glue jobs for performance and cost-efficiency.
  • Document the new data pipelines and processes.

Skills

  • Must-Haves:
  • High proficiency in AWS Glue, including hands-on experience writing and deploying PySpark scripts.
  • Strong experience with Python and the Spark framework (PySpark), particularly for data manipulation and transformation.
  • Knowledge of MS SQL Server, including the ability to read, understand, and reverse-engineer T-SQL, stored procedures, and functions.
  • Hands-on experience with Amazon S3 as a data source and target.
  • Good understanding of IAM roles and policies for managing secure access to AWS resources.

  • Strong proficiency in English

  • Highly Desirable:

  • Experience with AWS data orchestration services like AWS Step Functions and Lambda.
  • Familiarity with data warehousing concepts and dimensional modeling.
  • Knowledge of monitoring and logging using Amazon CloudWatch.
  • Experience working with data from SAP systems is a significant plus.

  • Analytics Engineer AWS

    1 semana atrás

    Remoto, Brasil Leega Consultoria Tempo inteiro R$60.000 - R$120.000 por ano

    A Leega é uma empresa focada no atendimento eficiente e inovador em seus clientes.Isso não poderia ser diferente com o nosso principal combustível: as pessoasNossa cultura é inspiradora e nossos valores estão presentes no dia a dia: ética e transparência, excelência de qualidade, trabalho em equipe, responsabilidade econômica, social e ambiental,...

  • Data Engineer AWS

    1 semana atrás

    Remoto, Brasil Compass Uol Tempo inteiro R$80.000 - R$120.000 por ano

    .Responsabilidades e atribuições Projetar e manter pipelines de dados para ingestão e transformação de múltiplas fontes, com foco em dados de sistemas de RH;Conectar, integrar e sincronizar dados entre canais e aplicações que compõem o ecossistema de multiagentes de IA;Trabalhar em parceria com engenheiros de IA e conversational designers na...

  • Lead - Data Engineer |

    1 semana atrás

    Remoto, Brasil Grupo Autoglass Tempo inteiro R$90.000 - R$120.000 por ano

    Descrição da vagaOlá :DBuscamos um profissional para liderar o time de Data Engineer (DataOps) da Autoglass Tech Esse profissional possui o compromisso de garantir a entrega contínua, confiável e automatizada de dados para consumo analítico e operacional, liderando a equipe técnica e promovendo integração entre engenharia de dados, governança de...

  • Data Engineer

    2 semanas atrás

    Remoto, Brasil Dadosfera Brasil Tempo inteiro R$90.000 - R$120.000 por ano

    Company OverviewDadosfera is transforming the data landscape with a platform that delivers advanced Data, AI, and Analytics capabilities—previously available only to tech giants like Meta, Amazon, Alphabet, and Microsoft—into the hands of small and medium-sized businesses. By democratizing access to these technologies, we empower our clients to...

  • AWS Developer

    1 dia atrás

    Remoto, Brasil Meta IT Tempo inteiro R$8.000 - R$15.000 por ano

    What are we looking for?We are seeking an AWS Developer with strong cloud and on-premise experience to help modernize our data and analytics platforms. This role focuses on migrating Oracle-based systems to AWS, building and automating data pipelines, and ensuring that analytical capabilities are scalable and reliable in the cloud.The ideal candidate is...

  • Senior DevOps Engineer

    Há 3 dias

    Remoto, Brasil BlueGrid Tempo inteiro R$72.000 - R$180.000 por ano

    We're hiring a full-time remote DevOps Engineer (Contractor).We're looking for a DevOps Engineer with 5+ years of experience in designing, building, and optimizing scalable data infrastructure—particularly within AWS environments.This role is perfect for someone who thrives on solving complex technical challenges, enjoys cross-functional collaboration, and...

  • Senior Data Engineer, Brazil

    1 semana atrás

    Remoto, Brasil Ci&T Tempo inteiro R$90.000 - R$120.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions.With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality.Hey everyone This is Thai Viapiana from CI&T's Talent Attraction Team :) I am looking...

  • Solution Architect – Data

    1 semana atrás

    Remoto, Brasil e-Core Tempo inteiro R$100.000 - R$150.000 por ano

    Be yourself, be your best Combinamos nossa experiência global em indústrias-chave e as últimas tecnologias para ajudar empresas em rápido crescimento e estabelecidas a transformar e acelerar seus modelos de negócios, desenvolver soluções digitais inovadoras, escalar capacidades tecnológicas e sustentar seu crescimento.Na e-Core, combinamos...

  • Senior Data Engineer

    Há 3 dias

    Remoto, Brasil Ci&T Tempo inteiro R$120.000 - R$180.000 por ano

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions.With over 7.400 CI&Ters around the world, we've built partnerships with more than 1,000 clients during our 30 years of history. Artificial Intelligence is our reality.About the RoleWe are seeking an experienced Senior Data Engineer to lead the migration...

  • Data Engineer Assistant

    1 semana atrás

    Remoto, Brasil Teltec Solutions Tempo inteiro R$80.000 - R$120.000 por ano

    Você é movido(a) por dados e tem interesse em engenharia de dados? Estamos procurando um(a) Data Engineer Assistant para apoiar na construção e manutenção de pipelines e estruturas de dados eficientes e escaláveis.Responsabilidades e atribuiçõesSUAS RESPONSABILIDADES SERÃO:Desenvolver e manter dashboards interativos e relatórios no Power BI,...