Lead Data Engineer

Overview Lead Data Engineer role located remotely (Contract) within the Media domain. Responsible for leading, designing, building, and maintaining the infrastructure for data integration, storage, processing, and analytics (BI, visualization, and Advanced Analytics) using Microsoft Azure. About Fusemachines Fusemachines is a leading AI strategy, talent, and education services provider. Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines aims to democratize AI. With a presence in four countries and more than 400 full-time employees, Fusemachines brings its global AI expertise to transform companies around the world. Role Details Location: Remote (Contract) Responsibilities : Lead engineers on the team to meet product deliverables Architect, design, develop, test, optimize and maintain high-performance, large-scale data architectures for data intake, validation, mining, and engineering to deliver data products Support data integration (batch and real-time), storage, processing, and infrastructure; ensure scalability, reliability, and performance of data systems Provide mentorship and guidance to junior data engineers and foster professional growth Collaborate with cross-functional teams to drive discovery and requirements for data management and business analytics Translate roadmap commitments into design and implementation milestones Identify and solve code/design optimization opportunities Learn and integrate with a variety of systems, APIs, and platforms Establish a robust QA process for data validations and overall quality control Continuously evaluate and implement new technologies; promote reusable components Design, implement, and maintain data governance solutions, including cataloging, lineage, data quality, and governance frameworks; lead data security and privacy efforts Work in an Agile environment; participate in ceremonies; collaborate on multi-disciplinary teams Contribute to continuous improvement and stay updated on market trends Qualifications & Experience Must have a full-time Bachelor's degree in Computer Science or similar from a top-tier school 4+ years of experience with Azure DevOps, Azure Cloud Platform, or other hyperscalers At least 4 years of experience as a data engineer with Azure, handling large datasets from diverse sources in the Media industry Proven experience delivering Data & Analytics projects/products as a data engineer Certifications: Microsoft Certified Azure Fundamentals, Microsoft Certified Azure Data Engineer Associate, Microsoft Certified Azure Solutions Architect Expert (nice to have), Databricks Certified Associate Developer for Apache Spark, Databricks Certified Data Engineer Associate (nice to have) Required Skills/Competencies Strong programming in Python (must have), plus Scala; write efficient, optimized code for data integration, storage, processing Experience documenting code or automated docs (e.g., PyDoc) Experience with scalable distributed processing (Spark/PySpark; Azure Databricks a plus), DBT, Kafka Designing ETL/ELT processes in Azure (Azure Data Factory a plus); develop custom integration solutions Data integration from APIs, databases, flat files, event streaming (Azure Data Factory) Data cleansing, transformation, and validation Jupyter Notebooks and Python packaging/dependency management (Poetry, Pipenv) Proficiency with relational databases (Oracle, SQL Server, MySQL, PostgreSQL) and NoSQL databases (MongoDB) Data modeling and database design; scalable schemas for data solutions Strong SQL skills Experience with Data Warehousing in Azure (Azure Synapse Analytics) and/or Snowflake Migration experience from SAS, R, Julia, SPSS to Python Technical leadership on Big Data projects; strong SDLC and Agile knowledge Familiarity with Azure DevOps tools, Jira/Azure Boards, GitHub/Azure Repos, CI/CD (GitHub Actions, Azure Pipelines, Jenkins), and artifact management Cloud security practices and data security/compliance awareness Stays updated on Azure services, data engineering trends, and best practices Organized, able to work in fast-paced environments, and capable of prioritizing effectively Team player, self-motivated, able to work independently Equal Opportunity Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status. Additional Referrals increase your chances of interviewing at Fusemachines by 2x Seniority level: Mid-Senior level Employment type: Contract Job function: Information Technology Industries: Internet Publishing #J-18808-Ljbffr