Lead Data Engineer

OverviewLead Data Engineer role located remotely (Contract) within the Media domain.Responsible for leading, designing, building, and maintaining the infrastructure for data integration, storage, processing, and analytics (BI, visualization, and Advanced Analytics) using Microsoft Azure.About FusemachinesFusemachines is a leading AI strategy, talent, and education services provider.Founded by Sameer Maskey Ph.D., Adjunct Associate Professor at Columbia University, Fusemachines aims to democratize AI.With a presence in four countries and more than 400 full-time employees, Fusemachines brings its global AI expertise to transform companies around the world.Role DetailsLocation: Remote (Contract)Responsibilities:Lead engineers on the team to meet product deliverablesArchitect, design, develop, test, optimize and maintain high-performance, large-scale data architectures for data intake, validation, mining, and engineering to deliver data productsSupport data integration (batch and real-time), storage, processing, and infrastructure; ensure scalability, reliability, and performance of data systemsProvide mentorship and guidance to junior data engineers and foster professional growthCollaborate with cross-functional teams to drive discovery and requirements for data management and business analyticsTranslate roadmap commitments into design and implementation milestonesIdentify and solve code/design optimization opportunitiesLearn and integrate with a variety of systems, APIs, and platformsEstablish a robust QA process for data validations and overall quality controlContinuously evaluate and implement new technologies; promote reusable componentsDesign, implement, and maintain data governance solutions, including cataloging, lineage, data quality, and governance frameworks; lead data security and privacy effortsWork in an Agile environment; participate in ceremonies; collaborate on multi-disciplinary teamsContribute to continuous improvement and stay updated on market trendsQualifications & ExperienceMust have a full-time Bachelor's degree in Computer Science or similar from a top-tier school4+ years of experience with Azure DevOps, Azure Cloud Platform, or other hyperscalersAt least 4 years of experience as a data engineer with Azure, handling large datasets from diverse sources in the Media industryProven experience delivering Data & Analytics projects/products as a data engineerCertifications: Microsoft Certified Azure Fundamentals, Microsoft Certified Azure Data Engineer Associate, Microsoft Certified Azure Solutions Architect Expert (nice to have), Databricks Certified Associate Developer for Apache Spark, Databricks Certified Data Engineer Associate (nice to have)Required Skills/CompetenciesStrong programming in Python (must have), plus Scala; write efficient, optimized code for data integration, storage, processingExperience documenting code or automated docs (e.g., PyDoc)Experience with scalable distributed processing (Spark/PySpark; Azure Databricks a plus), DBT, KafkaDesigning ETL/ELT processes in Azure (Azure Data Factory a plus); develop custom integration solutionsData integration from APIs, databases, flat files, event streaming (Azure Data Factory)Data cleansing, transformation, and validationJupyter Notebooks and Python packaging/dependency management (Poetry, Pipenv)Proficiency with relational databases (Oracle, SQL Server, MySQL, PostgreSQL) and NoSQL databases (MongoDB)Data modeling and database design; scalable schemas for data solutionsStrong SQL skillsExperience with Data Warehousing in Azure (Azure Synapse Analytics) and/or SnowflakeMigration experience from SAS, R, Julia, SPSS to PythonTechnical leadership on Big Data projects; strong SDLC and Agile knowledgeFamiliarity with Azure DevOps tools, Jira/Azure Boards, GitHub/Azure Repos, CI/CD (GitHub Actions, Azure Pipelines, Jenkins), and artifact managementCloud security practices and data security/compliance awarenessStays updated on Azure services, data engineering trends, and best practicesOrganized, able to work in fast-paced environments, and capable of prioritizing effectivelyTeam player, self-motivated, able to work independentlyEqual OpportunityEqual Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.AdditionalReferrals increase your chances of interviewing at Fusemachines by 2xSeniority level: Mid-Senior levelEmployment type: ContractJob function: Information TechnologyIndustries: Internet Publishing#J-*****-Ljbffr