Sr Python Data Engineer

3 semanas atrás

Cuiabá, Brasil Softensity Inc Tempo inteiro

Senior Python Data Engineer

About the Project

Responsibilities

- Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas, Polars).
- Develop and expose RESTful APIs using FastAPI or similar frameworks.
- Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel reports.
- Contribute to a spec-driven development workflow (using GitHub Copilot, Claude, etc.) to scaffold and generate API/data pipeline code.
- Optimize report generation logic for speed and scalability, currently targeting sub-20 second response times.
- Integrate with messaging and storage mechanisms (e.G., Service Bus, Storage Accounts).
- Collaborate on infrastructure-as-code automation using Bicep (or similar IaC tools).
- Participate in design discussions for future migration to Snowflake and/or a data lake architecture.
- Contribute to CI/CD pipelines using GitHub Actions.

Required Skills and Experience

- Strong proficiency in Python for data processing (must have expertise in Pandas, nice to have: Polars, openpyxl).
- Experience building backend services or APIs using frameworks like FastAPI.
- Solid understanding of data modeling principles (Star Schema) and handling normalized datasets.
- Familiarity with enterprise messaging patterns and data integration from various sources (API-based and file-based).
- Experience working with GitHub and CI/CD pipelines (GitHub Actions or similar).
- Infrastructure-as-Code experience with Bicep or comparable tools (Terraform, AWS CDK).
- Comfort with spec-driven development and leveraging AI tools like GitHub Copilot for scaffolding.

Softensity is an equal-opportunity employer. All qualified applicants are considered without regard to gender, identity, or personal background.

  • Sr Python Data Engineer

    3 semanas atrás

    Cuiabá, Brasil Softensity Inc Tempo inteiro

    Senior Python Data Engineer About the Project Responsibilities Design, build, and maintain high-performance data processing pipelines using Python libraries (Pandas, Polars). Develop and expose RESTful APIs using FastAPI or similar frameworks. Consume and process normalized Parquet files from multiple upstream sources to generate dynamic Excel...

  • Sr Data Engineer

    Há 6 dias

    Cuiabá, Brasil Luxoft Tempo inteiro

    Before you apply, please get familiar with Luxoft Luxoft locations:Logeek Magazine:Luxoft Alumni Club:Responsibilities:Design and implement scalable data pipelines using Databricks and KafkaBuild and maintain real-time streaming solutions for high-volume dataCollaborate with cross-functional teams to integrate data flows into broader systemsOptimize...

  • Data Engineer

    3 semanas atrás

    Cuiabá, Brasil HeartCentrix Solutions Tempo inteiro

    We are seeking a highly skilled Python Data Engineer with an AI / ML focus to join our client’s growing data & analytics team in Brazil. This role is ideal for someone who loves building scalable data pipelines, operationalizing machine learning workflows, and partnering closely with data scientists to bring models into production. You will design,...

  • Sr Data Engineer

    Há 6 dias

    Cuiabá, Brasil Luxoft Tempo inteiro

    Before you apply, please get familiar with Luxoft Luxoft locations: Logeek Magazine: Luxoft Alumni Club: Responsibilities: Design and implement scalable data pipelines using Databricks and Kafka Build and maintain real-time streaming solutions for high-volume data Collaborate with cross-functional teams to integrate data flows into broader...

  • Sr Data Engineer

    1 semana atrás

    Cuiabá, Brasil Luxoft Tempo inteiro

    Before you apply, please get familiar with Luxoft - Luxoft locations: https://career.Luxoft.Com/locations/ - Logeek Magazine: https://career.Luxoft.Com/logeek-magazine/ - Luxoft Alumni Club: https://career.Luxoft.Com/alumni/ Responsibilities: - Design and implement scalable data pipelines using Databricks and Kafka - Build and maintain real-time streaming...

  • Data Engineer

    Há 3 dias

    Cuiabá, Brasil Zunzun Solutions Tempo inteiro

    Summary: We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on...

  • Data Engineer

    Há 19 horas

    Cuiabá, Brasil Zunzun Solutions Tempo inteiro

    Summary: We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines. In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions. You will play a key role in modernizing our data platform on...

  • Data Engineer

    Há 8 horas

    Cuiabá, Brasil Zunzun Solutions Tempo inteiro

    Summary: We are seeking a highly skilled Data Engineer (Azure Databricks) to design, implement, and optimize enterprise-grade data pipelines.In this role, you will leverage Azure Databricks, Azure Data Factory, SQL Server, and Python to enable scalable, governed, and performant data solutions.You will play a key role in modernizing our data platform on the...

  • Data Engineer

    Há 6 dias

    Cuiabá, Brasil Ascendion Tempo inteiro

    Overview: We are seeking a highly skilled Data Engineer to support the development of personalized search capabilities and data assimilation initiatives within the organization.This is a remote role aligned to the EST time zone.The ideal candidate will have strong experience working with Python, databricks, and Kafka, and will contribute directly to ongoing...

  • Data Engineer

    Há 6 dias

    Cuiabá, Brasil Ascendion Tempo inteiro

    Overview: We are seeking a highly skilled Data Engineer to support the development of personalized search capabilities and data assimilation initiatives within the organization. This is a remote role aligned to the EST time zone. The ideal candidate will have strong experience working with Python, databricks, and Kafka, and will contribute directly to...