Senior HPC Data Engineer

3 semanas atrás

Brasília, Brasil N-iX Tempo inteiro

Job Overview We are looking for an experienced Data Service Module Engineer to develop and deploy the data service module for the HPC modeling project. This role focuses on implementing high-performance data storage and retrieval systems using HDF5 or similar, with parallel and concurrent I/O capabilities. The ideal candidate will have expertise in designing scalable data services optimized for HPC or distributed workflows, ensuring low latency and high throughput. Project: Digital asset operations platform Project: Global biopharmaceutical company Module Development and Deployment Design and implement the data service module using HDF5 for efficient data storage and retrieval. Develop parallel and concurrent I/O mechanisms to optimize performance for large-scale datasets. Ensure the module is tightly integrated with HPC and visualization workflows. Performance Optimization Optimize I/O operations for CPU/GPU-based workflows to minimize bottlenecks. Implement caching, compression, and other strategies to enhance performance. Data Management Design data structures and schemas suitable for storing 3D grid data and other simulation outputs. Ensure data integrity and consistency during concurrent read/write operations. Testing and Validation Develop and execute test cases to validate module performance and reliability under various load conditions. Conduct benchmarking to ensure scalability across different hardware configurations. Documentation and Support Document the architecture, APIs, and usage guidelines for the data service module. Provide technical support to the development and visualization teams for data integration. Qualifications Education: Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related fields. Experience: 3+ years of experience in developing and deploying data services for HPC or similar systems. Proven expertise with HDF5 or similar, in parallel I/O operations. Equivalent experience in distributed systems is also applicable. Technical Skills: Programming: Strong proficiency in (at least one): C++, Python, GoLang, or Fortran. HDF5 Expertise: In-depth knowledge of HDF5 APIs and advanced features like parallel HDF5. Parallel I/O: Experience with MPI I/O, POSIX I/O, or similar frameworks for concurrent/parallel data access. Performance Optimization: Skills in profiling and optimizing I/O operations for large datasets. Proficiency in SQL and experience with any RDMS. Might be a plus: knowledge of at least one orchestration and scheduling tool, for example, Airflow, Prefect, Dagster, etc. Soft Skills: Strong problem-solving skills and ability to work in a multidisciplinary team. Excellent communication skills for cross-team collaboration and documentation. Preferred Qualifications: Familiarity with data formats used in scientific computing, 3D visualization, and simulation workflows. We Offer Flexible working format - remote, office-based or flexible A competitive salary and good compensation package Professional development tools (mentorship program, tech talks and trainings, centers of excellence, and more) Active tech communities with regular knowledge sharing #J-18808-Ljbffr

  • Senior Data Engineer

    2 semanas atrás

    Brasília, Brasil Pride Global Tempo inteiro

    We're Hiring: Senior Data Engineer | Remote from Brazil | Fluent English required | Location: Remote – Brazil onlyContact: TemporaryAre you passionate about building scalable data platforms and cutting-edge MLOps solutions? Do you want to work with a top-tier US company revolutionizing e-commerce and circular fashion?We're looking for a Senior Data...

  • Senior data engineer

    2 semanas atrás

    Brasília, Brasil Pride Global Tempo inteiro

    We're Hiring: Senior Data Engineer | Remote from Brazil | Fluent English required|Location: Remote – Brazil onlyContact:TemporaryAre you passionate about building scalable data platforms and cutting-edge MLOps solutions? Do you want to work with a top-tier US company revolutionizing e-commerce and circular fashion?We're looking for aSenior Data...

  • Senior Data Engineer

    2 semanas atrás

    Brasília, Brasil Oowlish Tempo inteiro

    About the Role We are looking for a Senior Data Engineer to join our team and take ownership of building and maintaining scalable data pipelines and infrastructure. This role is ideal for someone who thrives in a dynamic environment, can work independently, and enjoys collaborating across teams to deliver reliable solutions. You will be the primary data...

  • Senior data engineer

    4 semanas atrás

    Brasília, Brasil Prosigliere Tempo inteiro

    Looking for an experienced Senior Data Engineer to join the Prosigliere team.DescriptionWe are seeking a highly skilled and motivated Data Engineer to join our growing team. The ideal candidate will be instrumental in designing, building, and maintaining robust and scalable data pipelines and infrastructure. This role requires a deep understanding of data...

  • Senior Data Engineer

    3 semanas atrás

    Brasília, Brasil Oowlish Technology Tempo inteiro

    Overview Oowlish, one of Latin America's rapidly expanding software development companies, is seeking experienced technology professionals to enhance our diverse and vibrant team. As a valued member of Oowlish, you will collaborate with premier clients from the United States and Europe, contributing to pioneering digital solutions. Our commitment to creating...

  • Inviting Senior Data Engineers

    Há 6 dias

    Brasília, Brasil Fusemachines Tempo inteiro

    2 days ago Be among the first 25 applicants About Fusemachines Fusemachines is a 10+ year old AI company, dedicated to delivering state‑of‑the‑art AI products and solutions to a diverse range of industries. Founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, our company is on a steadfast mission to democratize AI and...

  • Senior Data Engineer

    1 semana atrás

    Brasília, Brasil FullStack Labs Tempo inteiro

    Join to apply for the Senior Data Engineer - Remote - Latin America role at FullStack Labs About FullStack: FullStack is the most transparent IT talent network, connecting highly skilled individuals with top global companies and Silicon Valley startups for remote, on-demand projects. We focus on building a trusted, high-performance network where talent can...

  • Cloud Data Engineer

    4 semanas atrás

    Brasília, DF, Brasil Microtalent is becoming INSPYR Global Solutions Tempo inteiro €50

    Employment type: Direct Hire – Full-time, with all benefits required by Brazil law Salary range: Competitive and negotiable based on experience Language: Bilingual (Advanced English – excellent verbal and written communication skills required) The Senior Cloud Data Engineer leads the design, architecture, and implementation of secure, scalable data...

  • Senior Data Platform Engineer

    4 semanas atrás

    Brasília, Brasil Truelogic Software Tempo inteiro

    Senior Data Platform Engineer (OpenMetadata) - GTM Data Leader | Brazil Join to apply for the Senior Data Platform Engineer (OpenMetadata) - GTM Data Leader | Brazil role at Truelogic Software Senior Data Platform Engineer (OpenMetadata) - GTM Data Leader | Brazil 2 days ago Be among the first 25 applicants Join to apply for the Senior Data Platform Engineer...

  • Senior Data Platform Engineer

    Há 14 horas

    Brasília, Brasil Elios Talent Tempo inteiro

    Data Platform Engineer – Senior Key Highlights ️ Lead the design and expansion of large-scale data platform components supporting analytics, experimentation, and machine learning workloads ⚡ Architect and optimize ETL, streaming, metadata, and federated querying systems at scale Drive performance tuning, observability, and reliability efforts across...