Data Engineer

Há 2 dias

Belo Horizonte, Minas Gerais, Brasil Luxoft Tempo inteiro

Project Description:

  • The primary goal of the project is the modernization, maintenance and development of an eCommerce platform for a big US-based retail company, serving millions of omnichannel customers each week.
    • Solutions are delivered by several Product Teams focused on different domains - Customer, Loyalty, Search and Browse, Data Integration, Cart.
      • Current overriding priorities are new brands onboarding, re-architecture, database migrations, migration of microservices to a unified cloud-native solution without any disruption to business.

Responsibilities:

  • We are looking for Data Engineer who will be responsible for designing a solution for a big retail company. The main focus is to support processing of big data volumes and integrate solution to current architecture.

Mandatory Skills Description:


• Strong, recent hands-on expertise with Azure Data Factory and Synapse is a must (3+ years).


• Experience in leading a distributed team.


• Strong expertise in designing and implementing data models, including conceptual, logical, and physical data models, to support efficient data storage and retrieval.


• Strong knowledge of Microsoft Azure, including Azure Data Lake Storage, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks, pySpark for building scalable and reliable data solutions.


• Extensive experience with building robust and scalable ETL/ELT pipelines to extract, transform, and load data from various sources into data lakes or data warehouses.


• Ability to integrate data from disparate sources, including databases, APIs, and external data providers, using appropriate techniques such as API integration or message queuing.


• Proficiency in designing and implementing data warehousing solutions (dimensional modeling, star schemas, Data Mesh, Data/Delta Lakehouse, Data Vault)


• Proficiency in SQL to perform complex queries, data transformations, and performance tuning on cloud-based data storages.


• Experience integrating metadata and governance processes into cloud-based data platforms


• Certification in Azure, Databricks, or other relevant technologies is an added advantage


• Experience with cloud-based analytical databases.


• Experience with Azure MI, Azure Database for Postgres, Azure Cosmos DB, Azure Analysis Services, and Informix.


• Experience with Python and Python-based ETL tools.


• Experience with shell scripting in Bash, Unix or windows shell is preferable.

Nice-to-Have Skills Description:


• Experience with Elasticsearch


• Familiarity with containerization and orchestration technologies (Docker, Kubernetes).


• Troubleshooting and Performance Tuning: Ability to identify and resolve performance bottlenecks in data processing workflows and optimize data pipelines for efficient data ingestion and analysis.


• Collaboration and Communication: Strong interpersonal skills to collaborate effectively with stakeholders, data engineers, data scientists, and other cross-functional teams.

Languages:

English: B2 Upper Intermediate

  • Cloud Data Engineer

    1 semana atrás

    Belo Horizonte, Minas Gerais, Brasil beBeeDataEngineering Tempo inteiro US$98.320 - US$124.960

    Unlock Data Engineering ExcellenceDrive business outcomes through strategic data management as a skilled data engineer.Data Pipeline Design: Develop and maintain scalable data pipelines using Snowflake to ingest and transform data from diverse sources.ETL/ELT Implementation: Implement robust ETL/ELT processes and validation routines using SQL, Python, and...

  • Lead Data Engineer

    1 semana atrás

    Belo Horizonte, Minas Gerais, Brasil beBeeData Tempo inteiro €70.000 - €85.000

    Unlock Your Career Potential in Data EngineeringYour journey to becoming a skilled data engineer starts here. At our company, you'll find a collaborative and innovative work environment that fosters growth and learning.Job Description:We are seeking an experienced data engineer with a strong background in Pyspark to join our team. As a data engineer, you...

  • Senior Data Engineer

    Há 6 dias

    Belo Horizonte, Minas Gerais, Brasil SecurityScorecard Tempo inteiro

    Join to apply for the Senior Data Engineer role at SecurityScorecard 2 days ago Be among the first 25 applicants Join to apply for the Senior Data Engineer role at SecurityScorecard Get AI-powered advice on this job and more exclusive features. Funded by world-class investors including Silver Lake Waterman, Moody's, Sequoia Capital, GV, Riverwood...

  • Senior Data Engineer

    Há 6 dias

    Belo Horizonte, Minas Gerais, Brasil SecurityScorecard Tempo inteiro

    Join to apply for the Senior Data Engineer role at SecurityScorecard2 days ago Be among the first 25 applicantsJoin to apply for the Senior Data Engineer role at SecurityScorecardGet AI-powered advice on this job and more exclusive features.Funded by world-class investors including Silver Lake Waterman, Moody's, Sequoia Capital, GV, Riverwood Capital, and...

  • Senior Data Engineer

    Há 2 dias

    Belo Horizonte, Minas Gerais, Brasil SecurityScorecard Tempo inteiro

    Join to apply for the Senior Data Engineer role at SecurityScorecard2 days ago Be among the first 25 applicantsJoin to apply for the Senior Data Engineer role at SecurityScorecardGet AI-powered advice on this job and more exclusive features.Funded by world-class investors including Silver Lake Waterman, Moody's, Sequoia Capital, GV, Riverwood Capital, and...

  • Data Engineer

    1 semana atrás

    Belo Horizonte, Minas Gerais, Brasil FullStack Labs Tempo inteiro

    Join to apply for the Data Engineer - Remote - Latin America role at FullStack Labs About FullStack FullStack is the most transparent IT talent network, connecting highly skilled individuals with top global companies and Silicon Valley startups for remote, on-demand projects. We focus on building a trusted, high-performance network where talent can...

  • Data Engineer

    1 semana atrás

    Belo Horizonte, Minas Gerais, Brasil FullStack Labs Tempo inteiro

    Join to apply for the Data Engineer - Remote - Latin America role at FullStack LabsAbout FullStackFullStack is the most transparent IT talent network, connecting highly skilled individuals with top global companies and Silicon Valley startups for remote, on-demand projects. We focus on building a trusted, high-performance network where talent can thrive in a...

  • Scalable Data Engineer

    Há 5 dias

    Belo Horizonte, Minas Gerais, Brasil beBeeData Tempo inteiro €55.000 - €77.000

    Job Title: Scalable Data EngineerWe are looking for a skilled data engineer to design, develop and maintain large-scale data processing systems.Key Responsibilities:Design and develop scalable data systems using Scala and Cloud AWS.Collaborate with cross-functional teams to ensure seamless integration of data solutions.Develop and maintain high-quality code...

  • Senior Data Engineer

    Há 10 horas

    Belo Horizonte, Minas Gerais, Brasil FullStack Labs Tempo inteiro

    OverviewSenior Data Engineer - Remote - Latin America at FullStack LabsJoin to apply for the Senior Data Engineer - Remote - Latin America role at FullStack Labs.2 days ago Be among the first 25 applicantsAbout FullStackFullStack is the most transparent IT talent network, connecting highly skilled individuals with top global companies and Silicon Valley...

  • Senior Data Engineer

    1 semana atrás

    Belo Horizonte, Minas Gerais, Brasil beBeeDataEngineer Tempo inteiro US$80.000 - US$150.000

    Build Data Pipelines for Global ImpactOur mission is to ensure food safety, quality, and sustainability across international supply chains. We're seeking a talented engineer to build, maintain, and modernize data pipelines that process large-scale regulatory data from around the world.This role involves working hands-on with Python, SQL, and related tools to...