Senior Python Engineer
Há 1 mês
Hello
We are actively looking for a Senior Python Engineer. If you or your consultant are actively looking for a new job please share your profile.
Role: Senior Python Engineer
Duration: 2+ Months
Location: Brazil ( REMOTE )
Mandatory Skills : Experience with Airflow on Kubernetes or KEDA
We are seeking a highly skilled Senior Python Engineer to join our Technical Data Delivery team. In this role, you will be pivotal in developing and managing critical components of our data platform, including Data APIs (REST APIs), Apache Airflow, and data engineering artifacts such as data ingestion and data curation pipelines. This position requires strong proficiency in Python, Apache Airflow, REST APIs, Azure DevOps, and Azure cloud services.
Responsibilities:
1. Collaborate closely with the Data Engineering Lead to devise effective data ingestion strategies aligned with business objectives.
2. Define and implement robust data ingestion patterns and processes to ensure efficient and reliable data flow into the organization's data platform.
3. Contribute to the development and deployment of Apache Airflow on Azure Kubernetes Service (AKS).
4. Develop and maintain reusable data engineering or ETL pipelines and codebases using Python, Airflow, REST APIs, PySpark, Databricks, and the Azure cloud platform.
5. Design and implement robust data APIs using Python frameworks such as FastAPI or Flask, and deploy them on Azure App Service.
6. Work with cross-functional teams to understand data requirements and provide scalable data engineering solutions.
7. Design and implement batch and streaming data architectures leveraging Azure cloud services like Azure Data Factory and Azure Databricks.
8. Ensure adherence to software engineering best practices, including version control, testing, and continuous integration/continuous deployment (CI/CD) processes.
9. Participate in code reviews, technical discussions, and knowledge-sharing sessions within the team.
Skills (must have):
1. Strong experience in Python programming language.
2. Extensive experience with Apache Airflow and its deployments on Azure Kubernetes Service.
3. Extensive experience with Apache Airflow developments like Dynamic DAGs and Airflow Rest API’s
4. Hands-on experience with Python frameworks like FastAPI or Flask and deploying REST APIs on Azure App Service.
5. Sound understanding of software engineering development practices, including version control, testing, and continuous integration/continuous deployment (CI/CD) with proven experience using Azure DevOps
6. Demonstrated experience of core data engineering concepts and principles.
7. Proficiency in designing and implementing reusable and scalable data engineering pipelines and codebase.
8. Solid experience in designing batch and streaming data architectures using Azure cloud platform services.
9. Effective problem-solving skills and the ability to troubleshoot complex data engineering issues.
10. Commitment to continuous learning and staying updated with industry trends and best practices in data engineering.
11. Strong communication and collaboration skills with English language proficiency. Ability to work effectively in a team environment, provide training, and document processes.
Skills (nice to have):
1. Proficiency in writing PySpark code for data processing and transformation.
2. Familiarity with Python tools like Pytest, tox, poetry.
3. Familiarity with Python testing frameworks like pytest and build tools like tox and poetry.
Thank You
Satti Reddy
-
Data Engineer
Há 3 dias
Brazil, BR Insight Global Tempo inteiroMust-haves:5+ years of data engineer experience Experience working with AWS architecture (RDS, Glue, EMR, EC2, S3, Postgres, EMR, etc..)Snowflake data warehousingPython & SQL codingDay to Day:Insight Global is looking for 3 remote data engineers to join the analytics organization at a global medical device client. We are establishing a new data analytics...
-
Senior Software Engineer
2 semanas atrás
Brazil, BR Orange Quarter Tempo inteiroOrange Quarter have partnered with the fastest growing scale-up in the world and I’m helping them hire Senior Software Engineers for their teams in LATAM as they accelerate towards IPO.They have a market leading HR and Payroll platform that is used by over 35,000 businesses across 150+ countries.This role entails being part of a truly cross-functional...
-
Senior Back-End Developer
Há 3 dias
Brazil, BR Flatiron Software Co Tempo inteiroJob Title: Senior Backend Developer (Python)About:Flatiron is a global remote software development company with engineers located around the world. We unite experts from diverse backgrounds and experiences in a collaborative culture to deliver exceptional products and services for our clients. As a forward-thinking software engineering company, we provide...
-
Senior Data Engineer
2 semanas atrás
São Paulo, Brazil, BR BL Consultants Tempo inteiroSenior Data Engineer - São Paulo, Brazil - RemoteAbout the Role: Our client is looking for a Senior Data Engineer to be a technical powerhouse to help us scale our data infrastructure, automation and tools to meet growing business needs.You’re excited about this opportunity because you will…Work with business partners and stakeholders to understand data...
-
Senior DevOps Engineer
1 semana atrás
Brazil, BR TLR Ventures Tempo inteiroThis is a freelance contractor position.TLR Ventures is biotech startup backed by Jackson Laboratories, a leading global biomedical lab. We are developing a transformational technology platform for biomedical research that’s going to make a big difference in how drugs are discovered and how diseases are cured. The platform is a scientific data collection...
-
Senior Backend Engineer
Há 3 dias
Brazil, BR Station70 Tempo inteiroJob Type: Full-time, Remote (US CST, EST, MST time zones)About Station70Station70 is redefining digital asset security through our wallet-agnostic platform for scalable and secure digital asset key backup and recovery. Named after the protective bunkers used in the CASTLE nuclear tests, we offer trusted infrastructure for safeguarding assets against wallet...
-
Senior Machine Learning Engineer
2 meses atrás
Brazil, BR ATRA Tempo inteiroVaga: Machine Learning Engineer - MLOps Descrição da Vaga: Estamos à procura de um(a) Machine Learning Engineer especializado(a) em Google Cloud para integrar nosso time. O profissional terá a principal função de operacionalizar modelos criados pelas áreas usuárias, utilizando práticas de MLOps para garantir a eficiência e escalabilidade das...
-
Senior Software Engineer
Há 3 dias
Brazil, BR Remobi Tempo inteiroSenior Software Engineer / Golang DeveloperAbout Remobi:We are building the world's greatest community of remote technologists!Today, organizations that understand the value of remote working will reap the rewards. Because it doesn’t just provide team members with a healthier work-life balance, it gives you the opportunity to access the brightest minds...
-
DevOps Engineer Sênior
Há 1 mês
São Paulo, Brazil, BR Bankly - Banking as a Service Tempo inteiroVenha fazer parte do time Bankly!Somos uma plataforma de tecnologia financeira e Banking as a Service (BaaS) que tem o objetivo de alavancar tecnologia e quebrar barreiras para empoderar pessoas e empresas na transformação do ecossistema financeiro.Como DevOps Engineer Sênior você terá como principais responsabilidades:Definir, criar e gerenciar...
-
Data Engineer
2 meses atrás
Brazil, BR Premier Group Recruitment Tempo inteiroData EngineerContract$32 - $36 per hour. Premier Group has engaged exclusively with a growing marketing agency who are seeking a Data Engineer to join them on a 3-month contract, with a huge possibility of extension. The company is an English speaking business, so you must have strong English, both written and verbal, as well as your own Sociedade...
-
Desenvolvedor Senior Python
3 meses atrás
Brazil, BR Tata Consultancy Services Tempo inteiroVenha para uma das maiores empresas de Serviços TI do mundo!! Aqui você pode transformar sua carreira!Por que fazer parte da TCS? Aqui na TCS acreditamos que as pessoas fazem a diferença, por isso vivemos uma cultura de aprendizado ilimitado cheio de oportunidades de melhorias e desenvolvimento mútuo. O cenário ideal para expandir ideias através das...
-
Senior Python Developer Who Saves Lives
3 meses atrás
Brazil, BR Areti Health Tempo inteiroHelp build cutting-edge technology that saves lives! Hiring anywhere in Latin America.Areti Health is a venture-funded Silicon Valley startup that dramatically boosts clinical trial patient recruitment using generative AI. Our technology allows pharmaceutical, biotech, and medical device companies to bring their therapies to patients faster thereby saving...
-
Data Engineer
3 semanas atrás
Brazil, BR Pivotal Solutions Tempo inteiroThis is a contract position that will start immediately through the end of the year, with possible extension through next year depending on performance.Responsibilities:Collaborate with data analysts, engineers, business stakeholders, data scientists, and other team members to understand detailed data requirementsWrite code in Python and Airflow to build...
-
Senior Software Engineer
3 semanas atrás
Brazil, BR Interlix Staffing Tempo inteiroCompany OverviewOur client is a startup revolutionizing the agricultural industry by providing cutting-edge AI-driven risk management solutions for specialty crop growers (fruits, vegetables and tree nuts). Their mission is to protect farmers from unforeseen risks in an increasingly volatile climate. As an AI company at their core, they're dedicated to...
-
Labview Engineer
3 semanas atrás
Brazil, BR Luxoft Tempo inteiroProject Description:About the Client:Our client is one of the market leading producers of Integrated Circuits Lithography and Metrology machines, based in the Netherlands.About the Project:Automation of control processes for IC Lithography machines. Automation runs on Linux servers that are communicating with the machine via C++ code interfaces. Python...
-
Senior Software Engineer
3 semanas atrás
Brazil, BR Fraud Deflect Tempo inteiroCompany DescriptionFraud Deflect aims to revolutionize Friendly Fraud and Chargeback Management by bringing Trust, Integrity, and Transparency to an industry often clouded in secrecy. We strive to eliminate unethical practices and provide a clear pathway for our clients.Role DescriptionWe are seeking a highly skilled Senior Python Software Engineer to join...
-
Senior Software Engineer
Há 3 dias
Brazil, BR Iteris Consultoria e Software Tempo inteiroSomos a Iteris!Uma empresa de tecnologia que desde 2009 integra o humano e o tecnológico visando agregar cada vez mais valor para a transformação digital dos nossos clientes. Já somos mais de 900 pessoas apaixonadas por tecnologia e unidas por laços reais, que acreditam que a excelência nos resultados é a soma das qualidades técnicas e...
-
DevOps Engineer Senior
Há 1 mês
Brazil, BR SuperFrete Tempo inteiroEstamos contratando DevOps Engineer SeniorResponsabilidades:- Projetar, implementar e manter infraestruturas escaláveis e seguras utilizando Terraform e outras ferramentas de infraestrutura como código (IaC) no ambiente AWS;- Configurar e gerenciar pipelines de CI/CD para garantir uma integração e entrega contínua eficaz;- Trabalhar com serviços...
-
Senior Automation Engineer
2 meses atrás
Brazil, BR Foretellix Tempo inteiro***Please send your CV in EnglishWhat will you do:Be involved in changing the world: develop a validation framework and content for ensuringquality of a revolutionary system for autonomous vehicles.Find solutions and innovative approaches for complex problems in the SW Validation domain.Develop advanced methods for efficient analyzing of large volumes of...
-
Senior Data Engineer
Há 1 mês
Brazil, BR Acendeo Tempo inteiroAbout the CompanyYou will join a growing team driven by innovation, experimentation, and a deep understanding of our customers. As a team member, you'll work closely with your cross-functional partners across Product Management, Data Engineering, and Platform Engineering as we build next-generation products.English Skills a MUST C1 or C2 onlyAbout the...