Senior HPC Data Engineer

4 semanas atrás

Porto Alegre, Rio Grande do Sul, Brasil N-iX Tempo inteiro
About the Role

We are looking for an experienced Data Service Module Engineer to develop and deploy the data service module for the HPC modeling project. This role focuses on implementing high-performance data storage and retrieval systems using HDF5 or similar, with parallel and concurrent I/O capabilities. The ideal candidate will have expertise in designing scalable data services optimized for HPC or distributed workflows, ensuring low latency and high throughput.

Key Responsibilities
  • Module Development and Deployment: Design and implement the data service module using HDF5 for efficient data storage and retrieval.
  • Develop parallel and concurrent I/O mechanisms to optimize performance for large-scale datasets.
  • Ensure the module is tightly integrated with HPC and visualization workflows.
Performance Optimization
  • Optimize I/O operations for CPU/GPU-based workflows to minimize bottlenecks.
  • Implement caching, compression, and other strategies to enhance performance.
Data Management
  • Design data structures and schemas suitable for storing 3D grid data and other simulation outputs.
  • Ensure data integrity and consistency during concurrent read/write operations.
Testing and Validation
  • Develop and execute test cases to validate module performance and reliability under various load conditions.
  • Conduct benchmarking to ensure scalability across different hardware configurations.
Documentation and Support
  • Document the architecture, APIs, and usage guidelines for the data service module.
  • Provide technical support to the development and visualization teams for data integration.
Qualifications Education

Bachelor's or Master's degree in Computer Science, Software Engineering, or related fields.

Experience
  • 3+ years of experience in developing and deploying data services for HPC or similar systems.
  • Proven expertise with HDF5 or similar, in parallel I/O operations. Equivalent experience in distributed systems is also applicable.
Technical Skills
  • Programming: Strong proficiency in (at least one): C++, Python, GoLang, or Fortran.
  • HDF5 Expertise: In-depth knowledge of HDF5 APIs and advanced features like parallel HDF5.
  • Parallel I/O: Experience with MPI I/O, POSIX I/O, or similar frameworks for concurrent/parallel data access.
  • Performance Optimization: Skills in profiling and optimizing I/O operations for large datasets.
  • Proficiency in SQL and experience with any RDMS
  • Knowledge of at least one orchestration and scheduling tool (e.g., Airflow, Prefect, Dagster) is a plus.
Soft Skills
  • Strong problem-solving skills and ability to work in a multidisciplinary team.
  • Excellent communication skills for cross-team collaboration and documentation.
Preferred Qualifications
  • Familiarity with data formats used in scientific computing, 3D visualization, and simulation workflows.
We offer
  • Flexible working format - remote, office-based or flexible
  • A competitive salary and good compensation package
  • Personalized career growth
  • Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more)
  • Active tech communities with regular knowledge sharing
  • Education reimbursement
  • Memorable anniversary presents
  • Corporate events and team buildings
  • Other location-specific benefits
  • Not applicable for freelancers
Employment details
  • Seniority level: Not Applicable
  • Employment type: Full-time
  • Job function: Information Technology and Engineering
  • Industries: Software Development

Referrals increase your chances of interviewing at N-iX by 2x

Get notified about new Senior Data Engineer jobs in Brazil .

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

  • Senior data engineer

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil Netvagas Tempo inteiro

    About AzionWe are a global leader in the application and security industry. Our platform allows companies to operate with agility, reducing latency and increasing the reliability of their applications. We are focused on simplifying application building and looking for passionate and innovative individuals to join our teamAt Azion you will have the...

  • Senior data engineer

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil Netvagas Tempo inteiro

    About Azion We are a global leader in the application and security industry. Our platform allows companies to operate with agility, reducing latency and increasing the reliability of their applications. We are focused on simplifying application building and looking for passionate and innovative individuals to join our team At Azion you will have the...

  • Middle/Senior Data Engineer for Support

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil N-iX Tempo inteiro

    1 week ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. Role Overview As a Data Engineer on our Support Team, you will be responsible for maintaining robust data pipelines to support our products and services. You will work closely with cross-functional teams to understand data requirements and implement...

  • Senior Data Engineer Data Science

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil Azion Technologies, Inc. Tempo inteiro

    Sobre o Cargo Estamos buscando um Senior Data Engineer para integrar nossa equipe de Data Intelligence no nosso escritório de Porto Alegre/RS . Projetar, implementar e otimizar processos de ELT para ingerir, transformar e integrar dados de diversas fontes em uma plataforma de dados centralizada; Desenvolver e implementar estratégias de monitoramento...

  • AI Engineer

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil AgileEngine Tempo inteiro

    Join to apply for the AI Engineer (Senior) ID34931 role at AgileEngine 4 weeks ago Be among the first 25 applicants Join to apply for the AI Engineer (Senior) ID34931 role at AgileEngine AgileEngine is one of the Inc. 5000 fastest-growing companies in the US and a top-3 ranked dev shop according to Clutch. We create award-winning custom software...

  • Senior Software Engineer, Data Product

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil TRM Labs Tempo inteiro

    TRM is on a mission to build a safer financial system for billions of people. We deliver a blockchain intelligence data platform to financial institutions, crypto companies, and governments to fight cryptocurrency fraud and financial crime. We consider our business — and our profit — as a way to move towards our mission sustainably and at scale. The...

  • Senior Data Analyst

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil NTConsult Tempo inteiro

    **Local: Porto Alegre/RS**:**Experiência: Sênior**:**Tipo de Contratação Não Informado**:**Publicada em: 17/06/2025**:**The project**: We are looking for an exceptional Senior Data Analyst, specializing in Power BI, for this fully remote opportunity to work with a US-based client in the Bank Investments services industry.**Responsabilities**:1. Design...

  • Senior Data Analyst

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil NTConsult Tempo inteiro

    **The project**: We are looking for an exceptional Senior Data Analyst, specializing in Power BI, for this fully remote opportunity to work with a US-based client in the Bank Investments services industry.**Responsabilities**:1. Design and implement effective data models using Power BI to support interactive reporting and data analysis.3. Develop, optimize,...

  • Rust Developer

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil AgileEngine Tempo inteiro

    OverviewJoin to apply for the Rust Developer (Senior) ID41402 role at AgileEngine . AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best...

  • Senior Software Engineer

    4 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeePython Tempo inteiro

    Are you a highly skilled Python developer looking for a new challenge? Our client is seeking an experienced software engineer to join their team as a Python Developer. This fully remote position will involve collaborating with a U.S.-based team to design and develop APIs and data-driven applications. The ideal candidate will have 5+ years of experience in...