Senior Data Engineer

2 meses atrás

Brasil Sigma Software Group Tempo inteiro

Our growing data teams are seeking a savvy Senior Data Engineer to join them and help build and evolve the next generation of Audience & Identity data platforms that handle data at scale and use state-of-the-art technologies to unlock the data’s business value.

If you’re excited about working with cutting-edge technology in a fast-paced environment, we’d love to have you on board. Don’t hesitate — join our team today

Customer

Our client has superior end-to-end technology, a premium marketplace, and best-in-market advisory services that power the world’s advertising businesses of the largest media and entertainment companies. For example, Fox, NBC Universal, Viacom in the USA, Sky, Channel 4, RTE, and Mediaset in Europe.

Project

We are working on a solution to simplify segment ingestion, audience construction, and activation for targeted placements and campaigns. Our platform bridges the gap by combining buyers’ consumer insights with sellers’ media viewership and geographic data, addressing identity fragmentation and unifying audiences across devices.

Responsibilities

  • Design, implement, and maintain scalable data pipelines to ingest and transform large volumes of data from various sources
  • Optimize streaming and batch data processing and storage solutions for performance and scalability
  • Monitor system performance, troubleshoot issues, and implement solutions to ensure high availability and reliability
  • Collaborate with product managers and other stakeholders to understand data requirements and deliver data solutions that meet business needs
  • Communicate technical concepts and solutions effectively to non-technical stakeholders
  • Stay abreast of emerging technologies and best practices in data engineering. Identify opportunities to improve data processes, tools, and infrastructure to enhance efficiency and effectiveness
  • Document data pipelines, processes, and systems to ensure clarity and maintainability
  • Share knowledge and best practices with team members to foster a culture of learning and collaboration
  • Work with and mentor Junior Data Engineers

Requirements

  • 5+ years of hands-on experience in the Software Development field and/or Big Data
  • Strong experience in Scala or Java
  • Proficient in Databricks with a solid foundation in Spark
  • Experience with CI/CD tools (Jenkins)
  • Familiarity with shell, ability to work well with command lines, and a solid knowledge of data structures, algorithms, and system design
  • Work experience with ETL pipelines
  • Strong skills in SQL and experience with at least one of the major database platforms
  • At least an Upper-Intermediate level of English

Would be a plus

  • Experience with Python, and Golang
  • Experience in the media or advertising industry
  • Experience with graph databases (GraphX, Neo4J) and related analysis
  • Experience with Kubernetes, Terraform, and ELK
  • Experience with Snowflake
  • AWS certification and previous experience with AWS services such as S3, EMR, EKS, Glue, EC2, IAM, CloudFormation, Athena, Lambda, and others

  • Senior Data Engineer

    4 semanas atrás

    Brasil ECF Data, LLC Tempo inteiro

    Unlock Your Data PotentialAt ECF Data, LLC, we're on the lookout for a skilled Senior Data Engineer to join our team. As a key player in our data-driven organization, you'll be responsible for designing, building, and managing complex data pipelines that drive business growth.We're a Portuguese consulting company with a strong focus on Technology and...

  • Senior Data Engineer

    4 semanas atrás

    Brasil Data Privacy Tempo inteiro

    About the RoleWe are seeking a highly skilled Data Platform Engineer to join our team at TRM Labs. As a key member of our Data Platform team, you will be responsible for designing, building, and maintaining scalable and highly available data infrastructure for our products and services.Key ResponsibilitiesDesign and implement complex data pipelines to...

  • Senior Data Engineer

    2 semanas atrás

    Brasil Luxoft Tempo inteiro

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at Luxoft. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining scalable data pipelines on Azure Databricks and Google Cloud Platform (GCP) using PySpark and Python.Key Responsibilities:Design and implement...

  • Senior Data Engineer

    1 semana atrás

    Brasil HCLTech Tempo inteiro

    Job Title: Senior Data EngineerJob Summary:We are seeking a highly skilled Senior Data Engineer to join our team at HCLTech. As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining data pipelines using cloud-native tools in a modern data stack.Responsibilities:• Design and implement end-to-end data pipelines using...

  • Senior/Staff Software Engineer, Data Platform

    3 semanas atrás

    Brasil Data Privacy Tempo inteiro

    About the RoleWe're seeking a highly skilled Senior/Staff Software Engineer to join our Data Platform team. As a key member of our team, you will be responsible for designing, developing, and deploying scalable and reliable data infrastructure for our products and services.Key ResponsibilitiesDesign and develop highly available and scalable data services to...

  • Senior/Staff Software Engineer, Data Platform

    2 semanas atrás

    Brasil Data Privacy Tempo inteiro

    About TRM LabsTRM Labs is a mission-driven company that aims to build a safer financial system for billions of people. We deliver a blockchain intelligence data platform to financial institutions, crypto companies, and governments to fight cryptocurrency fraud and financial crime.Job DescriptionWe are seeking a highly skilled Software Engineer to join our...

  • Data Engineer Specialist

    1 semana atrás

    Brasil MG Info | Data Driven Culture Tempo inteiro

    Job Title: Data Engineer SpecialistWe are seeking a highly qualified Senior Data Engineer to join our team at MG Info | Data Driven Culture. This professional will be involved in the development and management of critical components of a data platform.The ideal candidate will have strong proficiency in Python, Apache Airflow, REST APIs, KEDA, Azure DevOps,...

  • Senior Data Engineer

    4 semanas atrás

    Brasil AgileEngine, LLC Tempo inteiro

    Job DescriptionWe are seeking a highly skilled Senior/Lead BI Engineer to join our team at AgileEngine, LLC.Key Responsibilities:Assess and analyze large datasets, including census, eligibility, medical, and Rx claims data.Map and merge data dictionaries to ensure accurate and efficient data analysis.Develop and implement automated tests to ensure data...

  • Senior Data Engineer for Zazmic Inc

    4 semanas atrás

    Brasil Zazmic Inc Tempo inteiro

    Job Title: Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer to join our dynamic data team at Zazmic Inc. As a Senior Data Engineer, you will play a critical role in designing, developing, and optimizing scalable data pipelines using Apache Spark and Google Cloud Platform (GCP).Key Responsibilities:Architect, build, and maintain...

  • Senior Data Engineer

    3 semanas atrás

    Brasil AgileEngine, LLC Tempo inteiro

    Job DescriptionWe are seeking a highly skilled Senior/Lead BI Engineer to join our team at AgileEngine, LLC.Key Responsibilities:Assess and analyze large datasets, including census, eligibility, medical, and Rx claims data.Map incoming data dictionaries to our data dictionary and merge information into the dataset for analysis.Collaborate with the QA team to...

  • Senior Data Engineer

    3 semanas atrás

    Brasil AgileEngine, LLC Tempo inteiro

    {"h1": "Data Analysis and Engineering Expert","p": "We are seeking a highly skilled Data Analysis and Engineering Expert to join our team at AgileEngine, LLC. As a Senior Data Engineer, you will be responsible for assessing census/eligibility, medical, and Rx claims data, mapping incoming data dictionaries to our data dictionary, merging information into the...

  • Senior Data Engineer

    4 semanas atrás

    Brasil AgileEngine, LLC Tempo inteiro

    Job DescriptionWe are seeking a highly skilled Data Engineer to join our team at AgileEngine, LLC. As a Data Engineer, you will be responsible for designing, building, and maintaining complex data pipelines to connect different sources of data together.Key ResponsibilitiesCommunicate with cross-functional teams to understand data needs and business...

  • Senior Data Engineer

    Há 7 dias

    Brasil Acendeo Tempo inteiro

    About AcendeoWe are a growing team driven by innovation and experimentation. As a team member, you'll work closely with your cross-functional partners across Product Management, Data Engineering, and Platform Engineering to build next-generation products.English Skills RequiredC1 or C2 level of English is a must for this role.About the RoleAs a Senior Data...

  • Senior Data Engineer

    Há 5 dias

    Brasil Sigma Software Tempo inteiro

    Job Title: Senior Data Engineer - Scala ExpertJob Summary:We are seeking a highly skilled Senior Data Engineer to join our dynamic team at Sigma Software. As a key player in our data engineering team, you will be responsible for designing, implementing, and maintaining robust data pipelines and infrastructure.Key Responsibilities:Design and implement...

  • Data Engineer

    4 semanas atrás

    Brasil Planahead Tempo inteiro

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Plan Ahead. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement data pipelines using cutting-edge tools and technologies.Develop and maintain high-performance data...

  • Data Engineer

    3 semanas atrás

    Brasil Planahead Tempo inteiro

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Planahead. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement data pipelines using cutting-edge tools and technologiesDevelop and maintain large-scale data systems,...

  • Data Engineer

    3 semanas atrás

    Brasil Planahead Tempo inteiro

    Job Title: Data EngineerWe are seeking a highly skilled Data Engineer to join our team at Planahead. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data systems.Key Responsibilities:Design and implement data pipelines using cutting-edge tools and technologies.Collaborate with cross-functional teams to ensure...

  • Senior Data Management Engineer

    4 semanas atrás

    Brasil Luby Tempo inteiro

    About LubyLuby is a dynamic and innovative company that is passionate about technology and data. We are seeking a highly skilled Senior Data Management Engineer to join our team and help us drive our data modernization project forward.Job SummaryThe Senior Data Management Engineer will be responsible for building and maintaining data pipelines, performing...

  • Data Engineer

    3 semanas atrás

    brasil Premier Group Recruitment Tempo inteiro

    Data Engineer Contract $32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...

  • Data Engineer

    3 semanas atrás

    brasil Premier Group Recruitment Tempo inteiro

    Data EngineerContract$32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...