Data Engineer

Há 7 horas

Belo Horizonte, Brasil Zunzun Solutions Tempo inteiro

Summary: We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on the Azure Cloud, ensuring reliability, efficiency, and compliance across the full data lifecycle. Key Responsibilities: · Data Pipeline Development: Design, build, and optimize ETL/ELT pipelines using Azure Databricks (PySpark, Delta Lake) and Azure Data Factory (ADF). · Data Flows & Transformations: Develop pipelines, data flows, and complex transformations with ADF, PySpark, and T-SQL for seamless data extraction, transformation, and loading. · Data Processing: Develop Databricks Python notebooks for tasks such as joining, filtering, and pre-aggregation. · Database & Query Optimization: Optimize database performance through SQL query tuning, index optimization, and code improvements to ensure efficient data retrieval and manipulation. · SSIS & Migration Support: Maintain and enhance SSIS package design and deployment for legacy workloads; contribute to migrationand modernization into cloud-native pipelines. · Collaboration & DevOps: Work with cross-functional teams using Git (Azure Repos) for version control and Azure DevOps pipelines (CI/CD) for deployment. · Data Governance & Security : Partner with governance teams to integrate Microsoft Purview and Unity Catalog for cataloging, lineage tracking, and role-based security. · API & External Integration: Implement REST APIs to retrieve analytics data from diverse external data feeds, enhancing accessibility and interoperability. · Automation: Automate ETL processes and database maintenance tasks using SQL Agent Jobs, ensuring data integrity and operational reliability. · Advanced SQL Expertise: Craft and optimize complex T-SQL queries to support efficient data processing and analytical workloads. Required Califications: · 5+ years of hands-on expertise with Azure Databricks, Python, PySpark, and Delta Lake. · 5+ years of proven experience with Azure Data Factory for orchestrating and monitoring pipelines. · Strong SQL Server / T-SQL experience with a focus on query optimization, indexing strategies, and coding best practices. · Demonstrated experience in SSIS package design, deployment, and performance tuning. · Hands-on knowledge of Unity Catalog for governance. · Experience with Git (Azure DevOps Repos) and CI/CD practices in data engineering projects. Nice to Have: · Exposure to Change Data Capture (CDC), Change Data Feed (CDF), and Temporal Tables. · Experience with Microsoft Purview, Power BI, and Azure-native integrations. · Familiarity with Profisee Master Data Management (MDM). · Working in Agile/Scrum environments. Preferred Qualifications: Microsoft Certified: Azure Data Engineer Associate (DP-203) Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced Azure certification Databricks Certified Data Engineer Associate or Professional Additional Microsoft SQL Server or Azure certifications demonstrating advanced database and cloud expertise

  • Data Engineer – Databricks | Azure

    3 semanas atrás

    Belo Horizonte, Brasil UPBI Data & AI Tempo inteiro

    A UPBI Data & AI, consultoria especializada em soluções digitais, parceira Microsoft e Databricks, está buscando um(a) Data Engineer com sólida experiência em Databricks e Azure para atuação em projeto internacional estratégico.Modelo: RemotoRegime: PJResponsabilidades:Desenvolver e otimizar pipelines de dados utilizando

  • Data Engineer

    Há 2 dias

    Belo Horizonte, Brasil Pixie Tempo inteiro

    Position: Data Engineer Location: ********* (Hybrid - 3 days onsite) Duration: 12-month contract Data Engineer We are seeking a Data Engineer to design, develop, and optimize data pipelines that support Retail Power's forecasting, pricing, and reporting operations.This role is responsible for integrating Salesforce data, usage history, pricing curves, and...

  • Data Engineer Ii

    2 semanas atrás

    Belo Horizonte, Brasil Vaga Para Data Engineer Ii Na Inter Carreiras Tempo inteiro

    Vem ser Inter com a gente Conta digital gratuita, plataforma de investimentos, shopping e o que mais a vida precisar.O Inter é isso, e é muito mais.A gente acredita que tudo pode ser mais simples, e que cada dia é mais uma oportunidade de fazer diferente.Se você também é assim, vem ser #sanguelaranja.Join us in Inter&Co Free digital account, investment...

  • Data Engineer

    Há 3 dias

    Belo Horizonte, Brasil Zunzun Solutions Tempo inteiro

    Summary:We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on the...

  • Data Engineer

    Há 20 horas

    Belo Horizonte, Brasil GraceMark Solutions Tempo inteiro

    Hello!We currently have a great job opportunity for you!About the role:Role: Data EngineerLocation: São Paulo, Brazil Duration: 6 monthsSalary: $ 2200 USD monthly (FTE)We’re looking for a Data Engineer with backend experience to join a high-impact engineering team building large-scale user-facing products.What You’ll DoBuild

  • Semantic Health Data Engineer

    Há 7 dias

    Belo Horizonte, Brasil Precision Data Engineering & Data Science Tempo inteiro

    Sobre a posição Estamos buscando uma pessoa para atuar como Engenheiro(a) de Dados Semânticos em Saúde (Semantic Health Data Engineer), ajudando a construir a próxima geração de soluções em Real World Evidence (RWE) na América Latina.Você vai trabalhar com dados de saúde de mundo real (Real World Data), conectando conhecimento clínico com...

  • Data Engineer

    2 semanas atrás

    Belo Horizonte, Brasil HeartCentrix Solutions Tempo inteiro

    We are seeking a highly skilled Python Data Engineer with an AI/ML focus to join our client's growing data & analytics team in Brazil. This role is ideal for someone who loves building scalable data pipelines, operationalizing machine learning workflows, and partnering closely with data scientists to bring models into production.You will design, develop, and...

  • Data Engineer

    1 semana atrás

    Belo Horizonte, Brasil Ascendion Tempo inteiro

    Overview:We are seeking a highly skilled Data Engineer to support the development of personalized search capabilities and data assimilation initiatives within the organization. This is a remote role aligned to the EST time zone. The ideal candidate will have strong experience working with Python, databricks, and Kafka, and will contribute directly to ongoing...

  • Semantic Health Data Engineer

    2 semanas atrás

    Belo Horizonte, Brasil Precision Data Engineering & Data Science Tempo inteiro

    Sobre a posição Estamos buscando uma pessoa para atuar como Engenheiro(a) de Dados Semânticos em Saúde (Semantic Health Data Engineer) , ajudando a construir a próxima geração de soluções em Real World Evidence (RWE) na América Latina.Você vai trabalhar com dados de saúde de mundo real (Real World Data) , conectando conhecimento clínico com...

  • Semantic Health Data Engineer

    1 semana atrás

    Belo Horizonte, Brasil Precision Data Engineering & Data Science Tempo inteiro

    Sobre a posição Estamos buscando uma pessoa para atuar como Engenheiro(a) de Dados Semânticos em Saúde (Semantic Health Data Engineer), ajudando a construir a próxima geração de soluções em Real World Evidence (RWE) na América Latina. Você vai trabalhar com dados de saúde de mundo real (Real World Data), conectando conhecimento clínico com...