Data Engineer

Há 6 horas

Rio de Janeiro, Brasil Zunzun Solutions Tempo inteiro

Summary:

We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on the Azure Cloud, ensuring reliability, efficiency, and compliance across the full data lifecycle.

Key Responsibilities:

· Data Pipeline Development: Design, build, and optimize ETL/ELT pipelines using Azure Databricks (PySpark, Delta Lake) and Azure Data Factory (ADF).

· Data Flows & Transformations: Develop pipelines, data flows, and complex transformations with ADF, PySpark, and T-SQL for seamless data extraction, transformation, and loading.

· Data Processing: Develop Databricks Python notebooks for tasks such as joining, filtering, and pre-aggregation.

· Database & Query Optimization: Optimize database performance through SQL query tuning, index optimization, and code improvements to ensure efficient data retrieval and manipulation.

· SSIS & Migration Support: Maintain and enhance SSIS package design and deployment for legacy workloads; contribute to migration and modernization into cloud-native pipelines.

· Collaboration & DevOps: Work with cross-functional teams using Git (Azure Repos) for version control and Azure DevOps pipelines (CI/CD) for deployment.

· Data Governance & Security: Partner with governance teams to integrate Microsoft Purview and Unity Catalog for cataloging, lineage tracking, and role-based security.

· API & External Integration: Implement REST APIs to retrieve analytics data from diverse external data feeds, enhancing accessibility and interoperability.

· Automation: Automate ETL processes and database maintenance tasks using SQL Agent Jobs, ensuring data integrity and operational reliability.

· Advanced SQL Expertise: Craft and optimize complex T-SQL queries to support efficient data processing and analytical workloads.

Required Califications:

· 5+ years of hands-on expertise with Azure Databricks, Python, PySpark, and Delta Lake.

· 5+ years of proven experience with Azure Data Factory for orchestrating and monitoring pipelines.

· Strong SQL Server / T-SQL experience with a focus on query optimization, indexing strategies, and coding best practices.

· Demonstrated experience in SSIS package design, deployment, and performance tuning.

· Hands-on knowledge of Unity Catalog for governance.

· Experience with Git (Azure DevOps Repos) and CI/CD practices in data engineering projects.

Nice to Have:

· Exposure to Change Data Capture (CDC), Change Data Feed (CDF), and Temporal Tables.

· Experience with Microsoft Purview, Power BI, and Azure-native integrations.

· Familiarity with Profisee Master Data Management (MDM).

· Working in Agile/Scrum environments.

Preferred Qualifications:

Microsoft Certified: Azure Data Engineer Associate (DP-203)

Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced Azure certification

Databricks Certified Data Engineer Associate or Professional

- Additional Microsoft SQL Server or Azure certifications demonstrating advanced database and cloud expertise

  • Data Engineer – Databricks | Azure

    3 semanas atrás

    Rio de Janeiro, Brasil UPBI Data & AI Tempo inteiro

    A UPBI Data & AI, consultoria especializada em soluções digitais, parceira Microsoft e Databricks, está buscando um(a) Data Engineer com sólida experiência em Databricks e Azure para atuação em projeto internacional estratégico.Modelo: RemotoRegime: PJResponsabilidades:Desenvolver e otimizar pipelines de dados utilizando Databricks e serviços...

  • Data Engineer

    Há 6 horas

    Rio de Janeiro, Brasil GraceMark Solutions Tempo inteiro

    Hello! We currently have a great job opportunity for you! About the role: Role: Data Engineer Location: São Paulo, Brazil Duration: 6 months Salary: $ 2200 USD monthly (FTE) We’re looking for a Data Engineer with backend experience to join a high-impact engineering team building large-scale user-facing products. What You’ll Do - Build and...

  • Data Engineer

    Há 5 dias

    Rio de Janeiro, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career! Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing...

  • Data Engineer

    Há 4 dias

    Rio de Janeiro, Brasil Insight Global Tempo inteiro

    Position: Data Engineer Location: Remote in BrazilDuration: 3 year+ PJ ContractMonthly Salary Range: 3-5k USD/MonthRequirements:3+ years of experience in Data EngineeringHands on data engineering experience with Microsoft Fabric (highly preferred) or Azure Synapse/Databricks are also acceptableAdvanced Data Manipulation experience using SQL, Pandas, and...

  • Data Engineer

    Há 4 dias

    Rio De Janeiro, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world Here you can transform your career Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing to...

  • Data Engineer

    2 semanas atrás

    Rio de Janeiro, Brasil Darkroom Tempo inteiro

    About The Role We are seeking an experienced Senior Data Engineer . You’ll work closely with our CTO, CPO, and product teams to architect, build, and deliver robust data pipelines and transformations for our always‑on AI platform. Success in this role will have a direct, measurable impact on business outcomes and future product velocity. Key...

  • Pursuing Excellence as a Data Engineer

    Há 4 dias

    Rio de Janeiro, Brasil beBeeDataEngineer Tempo inteiro

    Senior Data Engineer We're seeking an experienced Senior Data Engineer with a strong background in Pyspark to drive data-driven decision making across our organization. A minimum of 4 years of professional experience as a Data Engineer; A minimum of 4 years of professional experience with Pyspark; Demonstrated expertise in AWS and services such as Lambda,...

  • Data Engineer Opportunity

    Há 4 dias

    Rio de Janeiro, Brasil beBeeData Tempo inteiro

    Senior Data Engineer Position As a Senior Data Engineer, you will play a key role in designing and building scalable data pipelines and ingestion processes that support analytics and AI use cases. You will collaborate with analytics, AI/Data Science, and product teams to understand data requirements and SLAs. Key Responsibilities: Design, build, and maintain...

  • Databricks Data Engineer

    Há 13 horas

    Rio De Janeiro, Brasil GlobalSource IT Tempo inteiro

    Databricks Data EngineerFully Remote Contract We’re looking for a hands-on Databricks Data Engineer with strong experience building scalable data pipelines using Spark, PySpark, SQL, and Delta Lake. This role focuses on ingesting data from multiple sources, transforming it for analytics, and publishing high-quality datasets and...

  • Data Engineer

    2 semanas atrás

    Rio de Janeiro, Rio de Janeiro, Brasil Blue Orange Digital Tempo inteiro US$6.825 - US$8.350

    Company overview:Blue Orange Digital is a boutique data & AI consultancy that delivers enterprise-grade results. We design and build modern data platforms, analytics, and ML/AI Agent solutions for mid‑market and enterprise clients across Private Equity, Financial Services, Healthcare, and Retail. Our teams work with technologies like Databricks, Snowflake,...