Senior Data Engineer – Databricks
Há 1 dia
São Paulo, Brasil
EVOX Global
Tempo integral
Grátis com e-mail ou Google
Salve esta vaga e mantenha sua pesquisa organizada
Crie uma conta gratuita para salvar vagas, criar alertas e retornar a esta listagem a partir do seu painel.
Grátis com e-mail ou Google
Ao continuar, você concorda com nossos Termos & Política de Privacidade.
Evox is looking for a Senior Data Engineer – Databricks & AI to work with one of our clients.
We are seeking a senior-level Data Engineer with deep expertise in Databricks, cloud data platforms, and modern engineering practices. This role combines hands-on development with technical leadership, supporting scalable data solutions for analytics, machine learning, and Generative AI.
Key Responsibilities
• Design, build, and optimize scalable data pipelines using Databricks, Apache Spark, Python, SQL, and Scala.
• Define data architecture standards, reusable frameworks, and engineering best practices.
• Optimize data processing, platform performance, reliability, and costs.
• Implement Delta Lake, Medallion Architecture, Unity Catalog, and data governance practices.
• Lead CI/CD, DevOps, and Infrastructure as Code initiatives using Terraform.
• Build and support data solutions for AI and machine learning, including feature engineering, MLOps, RAG, and Generative AI applications.
• Collaborate with Data Science and AI Engineering teams to support AI solutions in production.
• Conduct architecture reviews, provide technical guidance, and mentor engineering teams.
Requirements:
• Strong experience in Data Engineering and enterprise data platforms.
• Advanced hands-on experience with Databricks and Apache Spark.
• Strong programming skills in Python, SQL, and Scala.
• Experience with Delta Lake, Unity Catalog, and data architecture.
• Experience with Azure and/or AWS.
• Hands-on experience with Terraform, CI/CD, and DevOps.
• Proven technical leadership, architecture design, and mentoring experience.
• Knowledge of AI/ML workloads, MLOps, and Generative AI.
• Fluent English. Nice to Have:
• Databricks Certified Data Engineer Professional.
• Experience with RAG, vector databases, and production AI applications.
• Experience with Kafka, Snowflake, and real-time data processing.
• Experience in regulated industries, such as Financial Services, Insurance, or Healthcare. Additional Information:
• Work Model: Hybrid – 3 days per week on-site in Pinheiros, São Paulo.
• Contract: PJ.
• Compensation: Negotiable.
Key Responsibilities
• Design, build, and optimize scalable data pipelines using Databricks, Apache Spark, Python, SQL, and Scala.
• Define data architecture standards, reusable frameworks, and engineering best practices.
• Optimize data processing, platform performance, reliability, and costs.
• Implement Delta Lake, Medallion Architecture, Unity Catalog, and data governance practices.
• Lead CI/CD, DevOps, and Infrastructure as Code initiatives using Terraform.
• Build and support data solutions for AI and machine learning, including feature engineering, MLOps, RAG, and Generative AI applications.
• Collaborate with Data Science and AI Engineering teams to support AI solutions in production.
• Conduct architecture reviews, provide technical guidance, and mentor engineering teams.
Requirements:
• Strong experience in Data Engineering and enterprise data platforms.
• Advanced hands-on experience with Databricks and Apache Spark.
• Strong programming skills in Python, SQL, and Scala.
• Experience with Delta Lake, Unity Catalog, and data architecture.
• Experience with Azure and/or AWS.
• Hands-on experience with Terraform, CI/CD, and DevOps.
• Proven technical leadership, architecture design, and mentoring experience.
• Knowledge of AI/ML workloads, MLOps, and Generative AI.
• Fluent English. Nice to Have:
• Databricks Certified Data Engineer Professional.
• Experience with RAG, vector databases, and production AI applications.
• Experience with Kafka, Snowflake, and real-time data processing.
• Experience in regulated industries, such as Financial Services, Insurance, or Healthcare. Additional Information:
• Work Model: Hybrid – 3 days per week on-site in Pinheiros, São Paulo.
• Contract: PJ.
• Compensation: Negotiable.