Senior Data Engineer

2 meses atrás


brasil Luxoft Tempo inteiro

Responsibilities

Design, build, and maintain scalable data pipelines on Azure Databricks and Google Cloud Platform (GCP) using PySpark and Python.

Work with Azure and GCP platforms, specifically managing services like Azure Databricks, Azure SQL Server, BigQuery, DataProc, and Azure Machine Learning.

Develop and maintain CI/CD pipelines using GitHub Actions to automate the deployment of data engineering solutions.

Utilize GitHub for version control, ensuring that all code changes are tracked, reviewed, and managed effectively.

Work closely with cross-functional teams to define requirements, design solutions, and deliver high-quality data engineering outputs.


Skills Must have

  • It is expected that the engineer:
  • Is proficient in Data Engineering concepts (Object Storage, Data Pipelines, ELTs, Data Modeling, etc.)
  • Is proficient in Software Engineering concepts (Continuous Integration, Continuous Deployment, Unit Testing, Coverage, Linting, Version Control System, etc.)
  • Have knowledge or understanding the basic concepts of Software Development lifecycle (Release Engineering, Deployments, Code review, Requirements definition)
  • Platforms - Azure Databricks, SQL Server, Azure Machine Learning, Azure Blob Storage
  • Programming Language - Python, PySpark
  • CI/CD - GitHub Actions, GitHub
  • VCS - GitHub


LANGUAGES

  • English: B2 Upper Intermediate



  • Brasil Data Privacy Tempo inteiro

    About the RoleWe are seeking a highly skilled Data Platform Engineer to join our team at TRM Labs. As a key member of our Data Platform team, you will be responsible for designing, building, and maintaining scalable and highly available data infrastructure for our products and services.Key ResponsibilitiesDesign and implement complex data pipelines to...

  • Senior Data Engineer

    3 semanas atrás


    Brasil Luxoft Tempo inteiro

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Luxoft. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines on Azure Databricks and Google Cloud Platform (GCP) using PySpark and Python.Key Responsibilities:Design and implement...

  • Senior Data Engineer

    2 semanas atrás


    Brasil HCLTech Tempo inteiro

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at HCLTech. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining data pipelines using cloud-native tools in a modern data stack.Responsibilities:• Design and implement end-to-end data pipelines using...


  • Brasil Data Privacy Tempo inteiro

    About the RoleWe're seeking a highly skilled Senior/Staff Software Engineer to join our Data Platform team. As a key member of our team, you will be responsible for designing, developing, and deploying scalable and reliable data infrastructure for our products and services.Key ResponsibilitiesDesign and develop highly available and scalable data services to...


  • Brasil Data Privacy Tempo inteiro

    About TRM LabsTRM Labs is a mission-driven company that aims to build a safer financial system for billions of people. We deliver a blockchain intelligence data platform to financial institutions, crypto companies, and governments to fight cryptocurrency fraud and financial crime.Job DescriptionWe are seeking a highly skilled Software Engineer to join our...

  • Data Engineer Specialist

    2 semanas atrás


    Brasil MG Info | Data Driven Culture Tempo inteiro

    Job Title: Data Engineer SpecialistWe are seeking a highly qualified Senior Data Engineer to join our team at MG Info | Data Driven Culture. This professional will be involved in the development and management of critical components of a data platform.The ideal candidate will have strong proficiency in Python, Apache Airflow, REST APIs, KEDA, Azure DevOps,...

  • Senior Data Engineer

    4 semanas atrás


    Brasil AgileEngine, LLC Tempo inteiro

    Job DescriptionWe are seeking a highly skilled Senior/Lead BI Engineer to join our team at AgileEngine, LLC.Key Responsibilities:Assess and analyze large datasets, including census, eligibility, medical, and Rx claims data.Map incoming data dictionaries to our data dictionary and merge information into the dataset for analysis.Collaborate with the QA team to...

  • Senior Data Engineer

    4 semanas atrás


    Brasil AgileEngine, LLC Tempo inteiro

    {"h1": "Data Analysis and Engineering Expert","p": "We are seeking a highly skilled Data Analysis and Engineering Expert to join our team at AgileEngine, LLC. As a Senior Data Engineer, you will be responsible for assessing census/eligibility, medical, and Rx claims data, mapping incoming data dictionaries to our data dictionary, merging information into the...

  • Senior Data Engineer

    2 semanas atrás


    Brasil Acendeo Tempo inteiro

    About AcendeoWe are a growing team driven by innovation and experimentation. As a team member, you'll work closely with your cross-functional partners across Product Management, Data Engineering, and Platform Engineering to build next-generation products.English Skills RequiredC1 or C2 level of English is a must for this role.About the RoleAs a Senior Data...

  • Senior Data Engineer

    2 semanas atrás


    Brasil Sigma Software Tempo inteiro

    Job Title: Senior Data Engineer - Scala ExpertJob Summary:We are seeking a highly skilled Senior Data Engineer to join our dynamic team at Sigma Software. As a key player in our data engineering team, you will be responsible for designing, implementing, and maintaining robust data pipelines and infrastructure.Key Responsibilities:Design and implement...

  • Data Engineer

    4 semanas atrás


    Brasil Planahead Tempo inteiro

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Planahead. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement data pipelines using cutting-edge tools and technologiesDevelop and maintain large-scale data systems,...

  • Data Engineer

    4 semanas atrás


    Brasil Planahead Tempo inteiro

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Planahead. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement data pipelines using cutting-edge tools and technologies.Collaborate with cross-functional teams to ensure...

  • Senior Data Engineer

    Há 23 horas


    brasil DeepRec.ai Tempo inteiro

    Senior Data Engineer – ML Services and Data Pipelines Location: Remote Employment Type: Contract Experience: 5+ years 3 month contract + multiple extensions possible Our client is based in the USA but employs a remote workforce. You should be available for team meetings at 8am PT and have some cross over for team collaboration in PST/ET timezone. Happy...

  • Senior Data Engineer

    1 dia atrás


    Brasil DeepRec.ai Tempo inteiro

    Senior Data Engineer – ML Services and Data Pipelines Location: Remote Employment Type: Contract Experience: 5+ years 3 month contract + multiple extensions possible Our client is based in the USA but employs a remote workforce. You should be available for team meetings at 8am PT and have some cross over for team collaboration in PST/ET timezone....


  • brasil DeepRec.ai Tempo inteiro

    Senior Data Engineer – ML Services and Data PipelinesLocation: RemoteEmployment Type: ContractExperience: 5+ years3 month contract + multiple extensions possibleOur client is based in the USA but employs a remote workforce. You should be available for team meetings at 8am PT and have some cross over for team collaboration in PST/ET timezone. Happy to hire...

  • Data Engineer

    4 semanas atrás


    brasil Premier Group Recruitment Tempo inteiro

    Data EngineerContract$32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...

  • Data Engineer

    4 semanas atrás


    brasil Premier Group Recruitment Tempo inteiro

    Data Engineer Contract $32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...

  • Data Engineer

    4 semanas atrás


    Brasil Premier Group Recruitment Tempo inteiro

    Data Engineer Contract $32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...

  • Senior Software Engineer

    4 semanas atrás


    Brasil Ignitho Technologies Inc. Tempo inteiro

    Senior Software EngineerIgnitho Technologies Inc. is seeking a highly skilled Senior Software Engineer to join our team.Key Responsibilities:Design and develop scalable.NET applications hosted on Azure cloud services, with a focus on Azure Kubernetes Services.Optimize SQL databases for efficient data retrieval and manipulation, and execute complex queries...

  • Senior Data Architect

    4 semanas atrás


    Brasil Thoughtworks Inc. Tempo inteiro

    Senior Data EngineerAt Thoughtworks Inc., we're seeking a highly skilled Senior Data Engineer to join our team. As a Senior Data Engineer, you will play a key role in building, maintaining, and testing the software architecture and infrastructure for managing data applications.Key Responsibilities:Develop and operate modern data architecture approaches to...