Data Engineer

Há 6 dias

Nova Lima, Brasil Zunzun Solutions Tempo inteiro

Summary:We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on the Azure Cloud, ensuring reliability, efficiency, and compliance across the full data lifecycle.Key Responsibilities:· Data Pipeline Development: Design, build, and optimize ETL/ELT pipelines using Azure Databricks (PySpark, Delta Lake) and Azure Data Factory (ADF).· Data Flows & Transformations: Develop pipelines, data flows, and complex transformations with ADF, PySpark, and T-SQL for seamless data extraction, transformation, and loading.· Data Processing: Develop Databricks Python notebooks for tasks such as joining, filtering, and pre-aggregation.· Database & Query Optimization: Optimize database performance through SQL query tuning, index optimization, and code improvements to ensure efficient data retrieval and manipulation.· SSIS & Migration Support: Maintain and enhance SSIS package design and deployment for legacy workloads; contribute to migration and modernization into cloud-native pipelines.· Collaboration & DevOps: Work with cross-functional teams using Git (Azure Repos) for version control and Azure DevOps pipelines (CI/CD) for deployment.· Data Governance & Security: Partner with governance teams to integrate Microsoft Purview and Unity Catalog for cataloging, lineage tracking, and role-based security.· API & External Integration: Implement REST APIs to retrieve analytics data from diverse external data feeds, enhancing accessibility and interoperability.· Automation: Automate ETL processes and database maintenance tasks using SQL Agent Jobs, ensuring data integrity and operational reliability.· Advanced SQL Expertise: Craft and optimize complex T-SQL queries to support efficient data processing and analytical workloads.Required Califications:· 5+ years of hands-on expertise with Azure Databricks, Python, PySpark, and Delta Lake.· 5+ years of proven experience with Azure Data Factory for orchestrating and monitoring pipelines.· Strong SQL Server / T-SQL experience with a focus on query optimization, indexing strategies, and coding best practices.· Demonstrated experience in SSIS package design, deployment, and performance tuning.· Hands-on knowledge of Unity Catalog for governance.· Experience with Git (Azure DevOps Repos) and CI/CD practices in data engineering projects.Nice to Have:· Exposure to Change Data Capture (CDC), Change Data Feed (CDF), and Temporal Tables.· Experience with Microsoft Purview, Power BI, and Azure-native integrations.· Familiarity with Profisee Master Data Management (MDM).· Working in Agile/Scrum environments.Preferred Qualifications:Microsoft Certified: Azure Data Engineer Associate (DP-203)Microsoft Certified: Azure Solutions Architect Expert or equivalent advanced Azure certificationDatabricks Certified Data Engineer Associate or ProfessionalAdditional Microsoft SQL Server or Azure certifications demonstrating advanced database and cloud expertise

  • Data Engineer

    3 semanas atrás

    Nova Lima, Brasil Brick Abode Tempo inteiro

    Software Engineer, Early Career (For Women in Tech Candidates) Location: Remote – Full-time About Brick Abode We are a software development company serving clients worldwide with varying technological needs. We value a strong computer science foundation and autonomy for our contractors. The team is based in Brazil and the USA, and we communicate in...

  • Senior Data Engineer

    Há 3 dias

    Nova Lima, Minas Gerais, Brasil Energy Job Search Tempo inteiro

    Job Description:Job SummaryData Engineers perform a vital role helping produce analytics that provide value for our customers and accelerate our business. This job requires great communication skills, a high degree data literacy, and tool fluency in the modern data stack. Data Engineers typically work in a multi-disciplinary teams with other data, software,...

  • Aws Data Engineer

    Há 3 dias

    Nova Lima, Brasil Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career!Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools, contributing to...

  • Data Engineer

    Há 21 horas

    Nova Lima, Brasil Artefact Tempo inteiro

    The current vacancy is for the Brazilian office and we work in a Free Office model.Who we areAt Artefact LatAm, we believe in and live a culture based on empathy!A healthy work environment is a place where all voices are heard, respected, and valued.Our commitment is to build a more diverse and inclusive environment, because empathy is for everyone,...

  • Data Engineer

    1 dia atrás

    Nova Iguaçu, Brasil Forkast Tempo inteiro

    Data Engineer Forkast•Nova Iguaçu, Rio de Janeiro, BrazilDescrição da vagaAbout the RoleWe are looking for a Data & Security Engineer to build, maintain, and secure analytics infrastructure for a prediction market platform (Forkast).This role sits at the intersection of data engineering, on-chain analytics, and security, with a strong focus on trade...

  • Data Engineer

    Há 2 dias

    Nova Iguaçu, Brasil Forkast Tempo inteiro

    Data Engineer Forkast•Nova Iguaçu, Rio de Janeiro, Brazil Descrição da vaga About the Role We are looking for a Data & Security Engineer to build, maintain, and secure analytics infrastructure for a prediction market platform (Forkast). This role sits at the intersection of data engineering, on-chain analytics, and security, with a strong focus on trade...

  • Data Engineer

    Há 7 dias

    Morada Nova, Brasil Brick Abode Tempo inteiro

    Data Engineer - Brick AbodeJoin us as a Data Engineer at Brick Abode, a software development company serving clients worldwide.We value a strong computer science base and offer autonomy, continuous learning, and direct communication with business stakeholders.About Brick AbodeWe are a software development company that serves clients from multiple countries...

  • Data Engineer

    1 dia atrás

    Nova Santa Rita, Brasil Forkast Tempo inteiro

    About the RoleWe are looking for a Data & Security Engineer to build, maintain, and secure analytics infrastructure for a prediction market platform (Forkast).This role sits at the intersection of data engineering, on-chain analytics, and security, with a strong focus on trade activity analysis, dashboarding, and anomaly detection.You will be responsible for...

  • Data Engineer

    Há 2 dias

    Nova Santa Rita, Brasil Forkast Tempo inteiro

    About the Role We are looking for a Data & Security Engineer to build, maintain, and secure analytics infrastructure for a prediction market platform (Forkast). This role sits at the intersection of data engineering, on-chain analytics, and security, with a strong focus on trade activity analysis, dashboarding, and anomaly detection. You will be responsible...

  • Senior Data Engineer

    4 semanas atrás

    Nova Santa Rita, Brasil Applaudo Tempo inteiro

    About You You are a Senior Data Engineer who enjoys building reliable, scalable data platforms and enabling data-driven and AI-powered products. You feel comfortable owning data pipelines end to end, working closely with analytics and AI / Data Science teams, and translating complex data problems into robust technical solutions. You value clean architecture,...