Lead Data Engineer

Lead Data Engineer Description
We are looking for a highly experienced Lead Data Engineer to lead and guide the technical evolution of our enterprise data lake. This role will combine hands-on development, tactical guidance, team leadership, and partner engagement across our technical team.

English fluency is a MUST for this role Only candidates with level C1 or C2 will be considered:

A1 Beginner
A2 Elementary
B1 Intermediate
B2 Upper-Intermediate
C1 Advanced
C2 Proficient

Responsibilities
Lead the implementation of scalable Lakehouse architecture using Delta and medallion architecture
Elevate data engineering skills across our team through mentorship, collaboration, and exemplification.
Lead development of scalable data pipelines using incremental ingestion and streaming
Build and maintain data infrastructure with tools like Databricks, Spark, Airflow 2 in an AWS environment
Collaborate closely with technical teams across the enterprise to help us transform complex data into actionable insights
Ensure high data quality standards by leveraging best practice approaches
Help architect storage layers, metastore/catalog solutions, and performance optimization
Oversee cloud infrastructure in AWS using Terraform, with a deep understanding of services like S3, IAM, KMS, Glue, Athena, Redshift, SNS/SQS, MSK, and Kinesis.
Help guide the integration of master data management and governance solutions like Datahub to track the health, lineage, and meaning of data across the enterprise
Help establish team coding standards and best practices.
Help refine development and operational processes to drive consistent, predictable quality
Participate in our on-call schedule to help us ensure availability of critical data services
Provide technical leadership on DevOps, security, and deployment strategies for AI/ML models

Requirements
8+ years of experience in data engineering, including senior/lead experience
Expertise in Lakehouse architectures, Delta/Parquet file formats, hive metastore solutions (Glue, Unity Catalog), and ETL processes (Airflow and Databricks)
Strong Experience with Python and SQL
Strong proficiency in data modeling (OLTP/OLAP)
Strong experience working with AWS cloud infrastructure services
Familiarity with Infrastructure as Code (IAC) using Terraform or CloudFormation
Databricks experience
Strong AWS experience. Certified AWS Solutions Architect or Data Engineer preferred.
Bachelor’s degree in computer science, Information Systems, Data Science, Mathematics, or similar fields, or equivalent experience