Data Engineer
Há 17 horas
curitiba, Brasil
Cognizant
Tempo integral
Grátis com e-mail ou Google
Salve esta vaga e mantenha sua pesquisa organizada
Crie uma conta gratuita para salvar vagas, criar alertas e retornar a esta listagem a partir do seu painel.
Grátis com e-mail ou Google
Ao continuar, você concorda com nossos Termos & Política de Privacidade.
Job Summary
The Architect role involves designing and implementing scalable data solutions using advanced technologies like MS Data Fabric Power BISpark OneLake SQL PySpark and Have hands
Key Responsibilities: Data Engineer
Pipeline Development: Design, develop, and maintain robust, scalable ETL/ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks.
Data Transformation: Utilize PySpark and SparkSQL to perform complex data transformations, cleaning, and aggregation.
Azure Data Lake Management: Work with Azure Data Lake Storage (ADLS) Gen2 and Azure Blob Storage for efficient data handling.
Data Modeling: Create and optimize data models (conceptual/logical/physical) for Lakehouse architecture (Medallion: Bronze, Silver, Gold).
Optimization: Perform advanced performance tuning for PySpark jobs and ADF pipelines to ensure reliability and cost efficiency.
CI/CD & DevOps: Implement CI/CD pipelines for data engineering code using Git, Azure DevOps, or GitHub Actions.
Collaboration: Work with stakeholders, data scientists, and analysts to translate business requirements into technical solutions
Required Skills: Microsoft Fabric, ADF, Azure Databricks, PySpark, Python, ADLS, SQL, ADO
Certifications Required
Certified Spark Developer Python Data Engineering Certification SQL Database Certification MS Power BI
The Architect role involves designing and implementing scalable data solutions using advanced technologies like MS Data Fabric Power BISpark OneLake SQL PySpark and Have hands
Key Responsibilities: Data Engineer
Pipeline Development: Design, develop, and maintain robust, scalable ETL/ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks.
Data Transformation: Utilize PySpark and SparkSQL to perform complex data transformations, cleaning, and aggregation.
Azure Data Lake Management: Work with Azure Data Lake Storage (ADLS) Gen2 and Azure Blob Storage for efficient data handling.
Data Modeling: Create and optimize data models (conceptual/logical/physical) for Lakehouse architecture (Medallion: Bronze, Silver, Gold).
Optimization: Perform advanced performance tuning for PySpark jobs and ADF pipelines to ensure reliability and cost efficiency.
CI/CD & DevOps: Implement CI/CD pipelines for data engineering code using Git, Azure DevOps, or GitHub Actions.
Collaboration: Work with stakeholders, data scientists, and analysts to translate business requirements into technical solutions
Required Skills: Microsoft Fabric, ADF, Azure Databricks, PySpark, Python, ADLS, SQL, ADO
Certifications Required
Certified Spark Developer Python Data Engineering Certification SQL Database Certification MS Power BI