Senior Data Engineer

Há 3 horas


Brazil, BR Luxoft Tempo inteiro

Responsibilities

Design, build, and maintain scalable data pipelines on Azure Databricks and Google Cloud Platform (GCP) using PySpark and Python.

Work with Azure and GCP platforms, specifically managing services like Azure Databricks, Azure SQL Server, BigQuery, DataProc, and Azure Machine Learning.

Develop and maintain CI/CD pipelines using GitHub Actions to automate the deployment of data engineering solutions.

Utilize GitHub for version control, ensuring that all code changes are tracked, reviewed, and managed effectively.

Work closely with cross-functional teams to define requirements, design solutions, and deliver high-quality data engineering outputs.


Skills Must have

  • It is expected that the engineer:
  • Is proficient in Data Engineering concepts (Object Storage, Data Pipelines, ELTs, Data Modeling, etc.)
  • Is proficient in Software Engineering concepts (Continuous Integration, Continuous Deployment, Unit Testing, Coverage, Linting, Version Control System, etc.)
  • Have knowledge or understanding the basic concepts of Software Development lifecycle (Release Engineering, Deployments, Code review, Requirements definition)
  • Platforms - Azure Databricks, SQL Server, Azure Machine Learning, Azure Blob Storage
  • Programming Language - Python, PySpark
  • CI/CD - GitHub Actions, GitHub
  • VCS - GitHub


LANGUAGES

  • English: B2 Upper Intermediate


  • Senior Data Engineer

    2 semanas atrás


    Brazil, BR Aptonet Inc Tempo inteiro

    As a Senior Data Engineer (Python/Spark) , you will play a crucial role in building and maintaining robust data ingestion pipelines, applications, and data processing systems. You will work on cutting-edge technologies and be responsible for designing, implementing, and optimizing solutions that support our Data Product Hub.Key Responsibilities:Develop and...

  • Senior Data Engineer

    2 semanas atrás


    Brazil, BR Sandbox Group Tempo inteiro

    About the roleThis is a 100% remote positionAs a Senior Data Engineer, is responsible for the development, construction, maintenance, and testing of architectures, such as databases and large-scale processing systems. Is responsible play a pivotal role in optimizing data systems and building them from the ground up, working closely with data analysts to help...

  • Data Engineer

    2 semanas atrás


    Brazil, BR Blueclip Tempo inteiro

    As Blueclip, we started our journey to bring the most creative minds and the know-how together to deliver great works.About Our SearchWe are looking for a multi-talented Big Data Engineer to facilitate the operations of the Data scientists and Engineering team of our partner which is a cryptocurrency investment research platform that’s driven by machine...

  • Data Engineer

    2 semanas atrás


    Brazil, BR Remobi Tempo inteiro

    Data Engineer - (REMOTE) BrazilAbout Remobi:We are building the world's greatest community of remote technologists!Today, organizations that understand the value of remote working will reap the rewards. It doesn’t just provide team members with a healthier work-life balance, it gives you the opportunity to access the brightest minds in the world.Our...

  • Data Engineer Senior

    2 semanas atrás


    São Paulo, Brazil, BR Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career!Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools,...

  • Senior Data Engineer

    2 semanas atrás


    Brazil, BR Sigma Software Group Tempo inteiro

    Our growing data teams are seeking a savvy Senior Data Engineer to join them and help build and evolve the next generation of Audience & Identity data platforms that handle data at scale and use state-of-the-art technologies to unlock the data’s business value.If you’re excited about working with cutting-edge technology in a fast-paced environment,...


  • Brazil, BR BetterEngineer Tempo inteiro

    Better Engineers. Better Results. BetterEngineers connects accomplished Software Engineers across the Americas with our portfolio of high-growth and newsworthy technology companies in the United States. Senior Engineers in the BetterEngineers network work remotely with some of the most interesting tech companies in the world. Join us today and experience a...

  • Data Engineer

    2 semanas atrás


    Brazil, BR Infosys Tempo inteiro

    Infosys Brasil está em busca de profissionais com o perfil de Data Engineer Pl e ou Sr (Databricks) Descrição da vaga:Este profissional terá um papel chave no envolvimento com a equipe de negócios e funcional no fornecimento de soluções técnicas mais adequadas, em conformidade com os padrões de desenvolvimento do cliente e da Infosys. Também...

  • Data Engineer LATAM

    2 semanas atrás


    Brazil, BR Mark & Bridges Technology Inc. Tempo inteiro

    Data Engineer LATAMEstamos em busca de engenheiro de dados experiente para atuar em projetos estratégicos utilizando Azure Data Factory, Databricks, Synapse Analytics e PySpark. A posição é voltada para profissionais localizados na América Latina/Nearshore, com contratação em regime PJ.Responsabilidades:Atuação direta no time do cliente nos EUA,...

  • Data Engineer

    Há 2 horas


    São Paulo, Brazil, BR Pride Global Tempo inteiro

    As a Data Engineer, you will be a key member of our data team, focusing on designing, developing, and maintaining data solutions for our data warehouse, data lake, and Dataverse environments. with specific focus on sourcing data and developing data solutions. In this role you will be primarily responsible for the development of data workflows, views, and...

  • Data Engineer

    2 semanas atrás


    São Paulo, Brazil, BR TerraMagna Tempo inteiro

    Quem SomosCom 7 anos de história, a TerraMagna vive a missão de tornar a agricultura tudo aquilo que o mundo precisa que ela seja. Hoje atua como financiadora da cadeia de insumos do agronegócio. Possui mais de 1 bilhão de reais sob gestão e atua com clientes em todo o Brasil.Requisitos:Data Engineer pra trabalhar com Visão Computacional, ML, AI, dados...

  • Data Engineer

    4 meses atrás


    São Paulo, Brazil, BR CriticalRiver Inc. Tempo inteiro

    Title: Data EngineerLocation: Sao Paulo BrazilRemote: Need to Support in USA PST time zoneDuration: Fulltime Interview process:Live Hacker rank test on Advanced SQL questions.Technical screeningClient live Hacker rank interview. Summary:Need a minimum of 6 years experience in SQL, with hands-on experience in writing complex queries and optimizing database...

  • Senior Software Engineer

    2 semanas atrás


    Brazil, BR Remobi Tempo inteiro

    Senior Software Engineer / Golang DeveloperAbout Remobi:We are building the world's greatest community of remote technologists!Today, organizations that understand the value of remote working will reap the rewards. Because it doesn’t just provide team members with a healthier work-life balance, it gives you the opportunity to access the brightest minds...

  • Lead Data Engineer

    2 semanas atrás


    Rio de Janeiro, Brazil, BR Smart Energy Water Tempo inteiro

    Company OverviewChoice Technologies, an SEW company, is a business focused in providing revenue assurance solutions for energy utilities, with more than 24 years of experience. Our products (Smart iNTL, Smart iCash, Smart iGEOs and Smart iField) provide a complete solution to mitigate non-technical losses (energy fraud), calculate Energy Balance and increase...

  • Data Engineer

    2 semanas atrás


    São Paulo, Brazil, BR Tata Consultancy Services Tempo inteiro

    Come to one of the biggest IT Services companies in the world!! Here you can transform your career!Why to join TCS? Here at TCS we believe that people make the difference, that's why we live a culture of unlimited learning full of opportunities for improvement and mutual development. The ideal scenario to expand ideas through the right tools,...

  • Azure Data Engineer

    Há 3 horas


    Brazil, BR Luxoft Tempo inteiro

    We are seeking a highly skilled Azure Data Engineer with DevSecOps experience to join our team. The ideal candidate will have strong expertise in Azure cloud services, data engineering, and DevSecOps practices, with a focus on automation, security, and the full software development lifecycle. You will be responsible for developing, maintaining, and...

  • Senior Network Engineer

    1 semana atrás


    Brazil, BR Hcltech Tempo inteiro

    Senior Network EngineerHíbrido - SPSomos a HCLTech, uma das maiores empresas de tecnologia e DSA que mais crescem no mundo, com mais de 227.000 profissionais espalhados em 60 países, impulsionando o progresso através de recursos líderes do segmento e centrados nas áreas de Digital, Engenharia e Nuvem.A força motriz por trás desse trabalho, nosso...

  • Senior Data Engineer

    Há 3 horas


    São Paulo, Brazil, BR AB InBev Tempo inteiro

    Sobre a AB InBevA AB InBev é a cervejeira líder global e uma das cinco maiores empresas de produtos de consumo do mundo, com mais de 500 marcas de cerveja. Estamos presentes nos principais mercados da América do Norte, América Latina, Europa, Ásia e África, sempre buscando inovação e excelência.Sobre o ABI Growth GroupCriado em 2022, o Growth Group...


  • Brazil, BR Birlasoft Tempo inteiro

    Data Engineer Databricks [AI Solutions]Job Overview: The Data Engineer will be instrumental in designing and implementing advanced data pipelines within the Databricks environment and will play a crucial role in pre-sales activities, gathering information to develop comprehensive solutions and proposals for clients.Responsibilities and Duties:Develop and...

  • Data Engineer

    2 semanas atrás


    Brazil, BR Pyramid Consulting, Inc Tempo inteiro

    Advanced proficiency in complex (ANSI-compliant) SQL queries (required)In-depth knowledge of data warehousing and data modeling techniques (required)Expertise in analytics and reporting (required)Background in Python programming & Airflow orchestrationExperience with Spark/EC2/S3 (desirable)Familiarity with Snowflake/Databricks (advantageous)