Data Warehouse Architect
2 semanas atrás
Databricks Data Warehouse Architect About the role: One of the largest companies in the world in the Gaming industry is seeking a hands-on Data Architect with DataWarehouse Engineer expertise in Databricks (DBX) and AWS-native data services to spearhead the design and implementation of a new data warehouse instance for a major product line. This role will involve building from the ground up—architecting scalable pipelines, optimizing lakehouse performance, and integrating seamlessly with diverse real-time and batch data sources across AWS.
The ideal candidate is passionate about data architecture , thrives in fast-moving environments, and has a proven track record of setting up high-performance lakehouse platforms on Databricks with a strong foundation in data warehousing principles .
Key Responsibilities: Design and deploy a new Databricks Lakehouse instance tailored to the client’s product-level data needs.
Architect and implement robust data ingestion pipelines using Spark (PySpark/Scala) and Delta Lake.
Integrate AWS-native services (S3, Glue, Athena, Redshift, Lambda) with Databricks for optimized performance and scalability.
Define data models, optimize query performance, and establish warehouse governance best practices.
Collaborate cross-functionally with product teams, data scientists, and DevOps to streamline data workflows.
Maintain CI/CD, preferably DBX for data pipelines using GitOps and Infrastructure-as-Code.
Monitor data jobs and resolve performance bottlenecks or failures across environments.
Required Skills & Experience: Databricks / Lakehouse Architecture End-to-end setup of Databricks workspaces and Unity Catalog
Expertise in Delta Lake internals , file compaction, and schema enforcement
Advanced PySpark/SQL skills for ETL and transformations
AWS Native Integration Deep experience with AWS Glue , S3 , Redshift Spectrum , Lambda , and Athena
IAM and VPC configuration knowledge for secure cloud integrations
Data Warehousing & Modeling Strong grasp of modern dimensional modeling (star/snowflake schemas)
Experience setting up lakehouse design patterns for mixed workloads
Automation & DevOps Familiarity with CI/CD for data engineering using tools like DBX, Terraform, GitHub Actions, or Azure DevOps
Proficient in monitoring tools like CloudWatch, Datadog, or New Relic for data pipelines
Bonus/Nice to Have: Experience supporting gaming or real-time analytics workloads
Familiarity with Airflow , Kafka , or EventBridge
Exposure to data privacy and compliance practices (GDPR, CCPA)
Other Details: Location : Latin America (LATAM) region - Remote, USA - Remote
Length : 1+ Year
Client : Gaming giant
-
Arquiteto de dados
1 semana atrás
Guarapuava, Brasil Ahoy by Belago Tempo inteiroArquiteto de dados Sênior ----Impresindível possui umas dessas certificaçãoes: Azure Platform Architect Badge GCP Platform Architect Badge Databricks Platform Administrator Apache developer Responsabilidades • Projetar e manter a arquitetura de cloud focada em dados (Data Lake, Lakehouse, DWH); • Gerenciar e provisionar recursos de computação...
-
Analista Engenheiro de Dados PL
2 semanas atrás
Guarapuava, Brasil Grupo Fleury Tempo inteiroQuais são os desafios da área e do cargo: Esta pessoa irá contribuir com os processos de qualidade em tecnologia do Grupo Fleury por meio de técnicas, ferramentas e parceria entre os times de Digital e Tech da empresa. Fazemos uso da tecnologia e inovação para gerar a movimentação que o mercado precisa. Somos reconhecidos pelo nosso propósito:...