Senior Big Data Engineer Position

Há 2 dias

São Paulo, São Paulo, Brasil beBeeDataSpecialist Tempo inteiro US$100.000 - US$150.000
Job Title

We are seeking a highly skilled Big Data Specialist to join our team. This role will involve migrating a data lake from AWS to OCI using Python and leveraging OCI services such as Object Storage, Data Flow, Function, ADW, and SQL Endpoint.

Key Responsibilities
  • Manage access to buckets and objects
  • Automate deployments
  • Develop scripts for copying data between AWS and OCI
  • Migrate AWS codes to OCI
  • Configure Data Flow
  • Run SQL endpoint queries
  • Perform post-migration validation

The ideal candidate will have experience with Spark optimization, including partitioning, joining, and caching.

Requirements
  • 5+ years of experience in big data engineering
  • Strong understanding of Python and its ecosystem
  • Experience with Spark and big data processing
  • Knowledge of OCI services and APIs

This is a full-time position that offers a competitive salary and benefits package. If you are passionate about working with big data, please apply now

  • Big Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil beBeeDataEngineer Tempo inteiro R$55.000 - R$85.000

    Job OpportunityAs a data engineer, you will play a crucial role in building innovative big data technologies and platforms to drive business impact. We are seeking a talented individual who can collaborate with partners and stakeholders to understand customer needs and develop effective solutions.The data team is responsible for designing, testing, coding,...

  • Big Data Engineer Pleno

    1 semana atrás

    São Paulo, São Paulo, Brasil Semantix Tempo inteiro

    Impactar positivamente bilhões de vidas é nosso propósito e também pode ser o seu Fundada em 2010, no Brasil, a Semantix é referência em Big Data, Analytics e Inteligência Artificial.Somos um time de apaixonados por inovação, com diversas origens e diferentes graus de experiência. O que nos une, é a motivação compartilhada de transformar a...

  • Senior Cloud Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil beBeeData Tempo inteiro US$100.000 - US$120.000

    Big Data Engineer Job SummaryWe are seeking an experienced Big Data Engineer to build and implement scalable data pipelines in cloud technologies.Develops and maintains large-scale data pipelines, builds out new integrations to support increasing data volume and complexity, and collaborates with analytics teams to improve data models and pipelines feeding...

  • Senior Data Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil 2K Tempo inteiro

    We're looking for a talented and passionate Senior Data Engineer to join Wildlife's Marketing Tech team in São Paulo, Brazil. Wildlife is one of the largest gaming companies in the world. Besides building amazing games, part of the reason it has been super successful has been its ability to acquire new users more efficiently than other companies. As a...

  • Senior Data Engineer Manager

    1 semana atrás

    São Paulo, São Paulo, Brasil beBeeEngineering Tempo inteiro

    Senior Data Engineer Manager PositionWe are seeking an experienced Senior Data Engineer Manager to lead our team of data engineers. This is a unique opportunity to shape the future of data engineering and drive innovation within our organization.The ideal candidate will have a strong background in data engineering, experience managing teams, and excellent...

  • Big Data Engineer

    4 semanas atrás

    São Paulo, São Paulo, Brasil Volare Serviços Médicos e Consultoria Tempo inteiro

    Big Data Engineer Our Client is a US technology company l ooking for a highly motivated Data Engineer with a passion for data, to build and implement data pipelines in cloud technologies, including SnapLogic and AWS. Responsibilities: Develops and maintains scalable data pipelines in SnapLogic and builds out new ETL and API integrations to support continuing...

  • Big Data Engineer

    1 semana atrás

    São Paulo, São Paulo, Brasil Volare Serviços Médicos e Consultoria Tempo inteiro

    Big Data Engineer Our Client is a US technology company l ooking for a highly motivated Data Engineer with a passion for data, to build and implement data pipelines in cloud technologies, including SnapLogic and AWS. Responsibilities: Develops and maintains scalable data pipelines in SnapLogic and builds out new ETL and API integrations to support continuing...

  • Big Data Engineer

    4 semanas atrás

    São Paulo, São Paulo, Brasil Volare Serviços Médicos e Consultoria Tempo inteiro

    Big Data Engineer Our Client is a US technology company l ooking for a highly motivated Data Engineer with a passion for data, to build and implement data pipelines in cloud technologies, including SnapLogic and AWS. Responsibilities: Develops and maintains scalable data pipelines in SnapLogic and builds out new ETL and API integrations to support continuing...

  • Big Data Engineer

    3 semanas atrás

    São Paulo, São Paulo, Brasil Volare Serviços Médicos e Consultoria Tempo inteiro

    Big Data Engineer Our Client is a US technology company l ooking for a highly motivated Data Engineer with a passion for data, to build and implement data pipelines in cloud technologies, including SnapLogic and AWS. Responsibilities: Develops and maintains scalable data pipelines in SnapLogic and builds out new ETL and API integrations to support continuing...

  • Senior Data Engineer

    1 dia atrás

    São Paulo, São Paulo, Brasil Curotec Tempo inteiro R$90.000 - R$120.000 por ano

    This is a remote position, 1 years Contract.We are seeking a Senior Data Engineer to support the ingestion, processing, and synchronization of data across our analytics platform. This role focuses on using Python Notebooks to ingest data via APIs into Microsoft Fabric's Data Lake and Data Warehouse, with some data being synced to a Synapse Analytics database...