Lead Data Pipeline Architect

Há 6 dias

Rio de Janeiro, Brasil beBeeDataEngineer Tempo inteiro

Job Opportunity: Data Engineer

We are seeking an experienced professional to lead the design, development, and maintenance of data pipelines. This role involves ingesting threat intelligence data from various sources into our data ecosystem.

Responsibilities:

  • Develop and implement data validation processes to ensure accuracy, completeness, and consistency.
  • Collaborate with analysts to understand data requirements and create effective solutions.
  • Design and develop automation scripts for data export processes to external systems or partners.
  • Enhance existing data pipelines for improved performance and scalability.
  • Monitor data pipelines, troubleshoot issues, and ensure continuous data availability and integrity.
  • Document technical specifications, data flows, and procedures for data pipeline maintenance and support.

Requirements:

  • Proven experience as a Data Engineer or similar role, focusing on data ingest, validation, and export automation.
  • Strong proficiency in Python programming language.
  • Experience with data pipeline orchestration tools, such as Apache Airflow or NiFi.
  • Familiarity with cloud platforms like Snowflake, AWS, Azure, or Google Cloud Platform.
  • Expertise in data validation techniques and tools for ensuring data quality.
  • Proficiency in building and deploying images using containerization technologies like Docker and Kubernetes.
  • Excellent problem-solving skills and attention to detail.
  • Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Benefits:

The ideal candidate will have experience designing, developing, and maintaining data pipelines, implementing data validation processes, and collaborating with analysts.

Key Skills:

  • Python programming
  • Apache Airflow or NiFi
  • Snowflake or AWS
  • Data validation
  • Docker and Kubernetes

  • Azure Data Architect Lead

    1 semana atrás

    Rio de Janeiro, Rio de Janeiro, Brasil beBeeDataarchitect Tempo inteiro R$100.000 - R$120.000

    Job Title: Senior Data ArchitectAbout the RoleWe are seeking an experienced Data Architect with deep expertise in Azure Databricks, Unity Catalog, PySpark, and Delta Live Tables. In this role, you will lead the design, architecture, and delivery of advanced data solutions that drive business growth.Key Responsibilities:Lead solution architecture and design...

  • Senior Data Architect

    2 semanas atrás

    Rio de Janeiro, Rio de Janeiro, Brasil beBeeData Architect Tempo inteiro US$57.264 - US$120.384

    Job OverviewWe are seeking a skilled professional to lead our Data Architecture initiatives. This role involves owning the design and governance of our Lakehouse platform and representing that vision to clients.Main ResponsibilitiesClient Engagement & Solution DevelopmentAct as the primary technical liaison during key engagements, translating business goals...

  • Cloud Data Architect

    2 semanas atrás

    Rio de Janeiro, Rio de Janeiro, Brasil beBeeData Tempo inteiro R$80.000 - R$150.000

    Cloud Data Architect PositionWe are seeking an experienced Cloud Data Architect to lead the design and implementation of cloud-based data solutions.Key Responsibilities:Design, develop, and deploy scalable cloud-based data architectures on Azure.Collaborate with cross-functional teams to build and maintain metadata-driven data solutions for large...

  • Cloud Data Architect

    Há 6 dias

    Rio de Janeiro, Brasil beBeeData Tempo inteiro

    Cloud Data Architect Position We are seeking an experienced Cloud Data Architect to lead the design and implementation of cloud-based data solutions. Key Responsibilities: Design, develop, and deploy scalable cloud-based data architectures on Azure. Collaborate with cross-functional teams to build and maintain metadata-driven data solutions for large...

  • Chief Data Pipeline Architect

    Há 4 dias

    Rio de Janeiro, Brasil beBeeDataEngineer Tempo inteiro

    Job Overview As a senior data engineer, you will play a critical role in designing and maintaining scalable data pipelines on cloud-based infrastructure to ensure high-performance, quality, and security standards are met. This senior-level position involves collaborating with data scientists and analysts to integrate data from multiple sources in support of...

  • Data Architect

    Há 3 dias

    Rio de Janeiro, Brasil buscojobs Brasil Tempo inteiro

    Overview At Aravita, we're not just building a product; we're building the future of AI-enabled fresh food operations. We are a fast-paced, innovative startup on a mission to solve complex problems with data. Our culture is built on collaboration, curiosity, and a passion for technology. We're looking for a brilliant Data Engineer / Cloud...

  • Sr. Data Architect

    Há 6 dias

    Rio de Janeiro, Brasil buscojobs Brasil Tempo inteiro

    Overview Native / Bilingual English is required for this role (read / written / spoken). Please upload your CV Resume in English. Monthly salary : $5,000 - $6,000 Our partner is a company where data drives insight and architecture drives impact. They provide advanced analytics and real-time data pipelines that power the foodservice supply chain. Along with...

  • Cloud Data Architect

    1 semana atrás

    Região Geográfica Intermediária de São Paulo, Brasil beBeeData Tempo inteiro R$90.000 - R$120.000

    Job Title: Cloud Data ArchitectWe're seeking a seasoned data architect to lead the development of our modern cloud-based platforms.Key Responsibilities:Design and implement data platforms using Microsoft Fabric (Synapse, OneLake).Create scalable pipelines with Azure Data Factory.Optimize data performance using Delta/Parquet, partitioning, and Spark.Develop...

  • Senior Data Architect

    Há 4 dias

    Rio de Janeiro, Brasil beBeeData Tempo inteiro

    We are seeking a skilled data architect to design, develop, and maintain large-scale data pipelines that process regulatory data from around the world. The engineer will work hands-on with Python, SQL, and related tools to create efficient data flows, migrate processes to more maintainable platforms, and ensure our data is accurate, reliable, and ready for...

  • Data Architect Specialist

    2 semanas atrás

    Rio de Janeiro, Rio de Janeiro, Brasil beBeeData Tempo inteiro R$60.000 - R$72.000

    Cloud Data ArchitectThis is a strategic technical role responsible for designing and evolving scalable, high-performance data systems in the cloud.We are seeking a skilled Cloud Data Architect to define architecture and guide engineering across our entire data platform - from ingestion to analytics.The successful candidate will work closely with...