AWS Data Engineer

Há 8 horas

São Paulo São Paulo, Estado de São Paulo, Brasil Decision group Tempo inteiro

Nível Acadêmico: Ensino Superior Completo Turno/Horas -- Áreas de Atuação Profissional: TI - Projetos Descrição: Project Description We are looking for an experienced AWS data engineer, who will join the team responsible for the second phase of data warehouse cloud migration project. The initial 'lift-and-shift' of an on-premise MS SQL data warehouse to AWS RDS has been completed. This next phase focuses on refactoring the legacy system into a modern, serverless data architecture. The primary goal is to migrate all data transformation logic from MS SQL to AWS Glue and prepare the data for consumption by business intelligence tools like AWS QuickSight. Key Responsibilities Analyze existing MS SQL stored procedures and data structures within the PMM data warehouse to understand the current business and transformation logic. Design, develop, and deploy scalable and efficient ETL/ELT pipelines using AWS Glue and PySpark to replace the legacy SQL processes. Ingest and process data from various sources currently landing in Amazon S3, including SAP extracts, operational system feeds, and flat files. Implement robust data validation, error handling, and logging mechanisms within Glue jobs. Collaborate with architects to define data models and schemas suitable for the new data lakehouse architecture. Optimize Glue jobs for performance and cost-efficiency. Document the new data pipelines and processes. Skills Must-Haves: oHigh proficiency in AWS Glue, including hands-on experience writing and deploying PySpark scripts. oStrong experience with Python and the Spark framework (PySpark), particularly for data manipulation and transformation. oKnowledge of MS SQL Server, including the ability to read, understand, and reverse-engineer T-SQL, stored procedures, and functions. oHands-on experience with Amazon S3 as a data source and target. oGood understanding of IAM roles and policies for managing secure access to AWS resources. oStrong proficiency in English Habilidades: Highly Desirable: oExperience with AWS data orchestration services like AWS Step Functions and Lambda. oFamiliarity with data warehousing concepts and dimensional modeling. oKnowledge of monitoring and logging using Amazon CloudWatch. oExperience working with data from SAP systems is a significant plus. Atuação remotaFormação Acadêmica:Não informadoSalário:A combinarCargo:Engenheiro de TestesEmpresa:Decision groupAtua com consultoria de mão de obra de serviços de tecnologia.Ramo:Recursos Humanos/ Recrutamento e seleção (CK)

  • AWS Data Engineer

    Há 3 dias

    São Paulo, São Paulo, Brasil Invictus Data & AI Tempo inteiro R$80.000 - R$120.000 por ano

    Sobre o INVICTUS DATA & AIA Invictus Data & AI busca um(a) Engenheiro(a) de Dados AWS Pleno para integrar nosso time de dados e construir soluções escaláveis e robustas. Você será responsável por desenvolver e manter pipelines de dados, modelar dados e otimizar a performance da nossa infraestrutura de dados na AWS.A Invictus Data & AI foi fundada para...

  • Senior Data Engineer AWS

    Há 5 dias

    São Paulo, São Paulo, Brasil OpsBrasil Serviços Cloud LTDA Tempo inteiro

    We're looking for a Data Engineer with strong experience in AWS (primary) and good knowledge of Azure or GCP. The ideal candidate will design and implement scalable data pipelines, handle database migrations, and develop efficient ETL processes.Responsibilities:Build and maintain scalable data pipelines in AWS.Design and execute database migrations and ETL...

  • Technical Lead

    Há 8 horas

    São Paulo, São Paulo, Estado de São Paulo, Brasil Decision group Tempo inteiro

    Nível Acadêmico: Ensino Superior Completo Turno/Horas -- Áreas de Atuação Profissional: TI - Projetos Descrição: HibridoBrazil (preferably Londrina and São Paulo (escritorio de Barueri)> 5 Years: AWS (S3, Redshift), Snowflake. Strong in SQL, Python/PySpark. Familiarity with GitHub, CI/CD. Knowledge of data modeling, architecture, and system...

  • Senior Data Engineer

    Há 8 horas

    São Paulo, Estado de São Paulo, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career!Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing to...

  • Data Engineer

    Há 8 horas

    São Paulo, Estado de São Paulo, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career! Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing...

  • Data Engineer

    Há 8 horas

    São Paulo, Estado de São Paulo, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career!Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing to...

  • Data Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil Perform Tempo inteiro R$60.000 - R$180.000 por ano

    We are seeking a skilled and motivated Data Engineer to join our Business Technology team. You'll help drive our business data analytics strategy by building reliable, scalable data pipelines and ensuring strong data engineering practices are in place.What You'll DoDesign, build, and maintain scalable data pipelines and ETL processes to support business...

  • Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Blue Orange Digital Tempo inteiro US$60.000 - US$90.000 por ano

    Company overview:Blue Orange Digital is a boutique data & AI consultancy that delivers enterprise-grade results. We design and build modern data platforms, analytics, and ML/AI Agent solutions for mid‑market and enterprise clients across Private Equity, Financial Services, Healthcare, and Retail.Our teams work with technologies like Databricks, Snowflake,...

  • Senior Data Engineer

    Há 22 horas

    São Paulo, São Paulo, Brasil Celebal Technologies Tempo inteiro R$90.000 - R$120.000 por ano

    Job Title:Data EngineerLocation:São Paulo, Brazil (Hybrid or Remote within Brazil)Employment Type:Full-timeAbout the RoleWe are looking for an experienced Data Engineer to join our data team in Brazil. The ideal candidate will design and maintain modern data pipelines and architectures, with a strong focus on Databricks development and migration projects...

  • Senior Data Engineer

    Há 8 horas

    São Paulo, Estado de São Paulo, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career! Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing...