High-Performance Data Services Specialist

Há 21 horas

Porto Alegre, Rio Grande do Sul, Brasil beBeeDataScientist Tempo inteiro R$96.885 - R$146.312
Job Title:

Senior HPC Data Engineer

About the Role

We are seeking an experienced professional to develop and deploy high-performance data storage and retrieval systems for our large-scale modeling project.

Key Responsibilities:
  • Design and implement scalable data services optimized for high-performance computing or distributed workflows.
  • Develop parallel and concurrent I/O mechanisms to optimize performance for large datasets.
  • Ensure tight integration with high-performance computing and visualization workflows.
Performance Optimization
  • Optimize input/output operations for CPU/GPU-based workflows to minimize bottlenecks.
  • Implement caching, compression, and other strategies to enhance performance.
Data Management
  • Design data structures and schemas suitable for storing 3D grid data and other simulation outputs.
  • Ensure data integrity and consistency during concurrent read/write operations.
Testing and Validation
  • Develop and execute test cases to validate module performance and reliability under various load conditions.
  • Conduct benchmarking to ensure scalability across different hardware configurations.
Documentation and Support
  • Document the architecture, APIs, and usage guidelines for the data service module.
  • Provide technical support to development and visualization teams for data integration.
Qualifications:Education

Bachelor's or Master's degree in Computer Science, Software Engineering, or related fields.

Experience
  • Minimum 3 years of experience in developing and deploying data services for high-performance computing or similar systems.
  • Proven expertise with HDF5 or similar technologies in parallel I/O operations.
Technical Skills
  • Programming: Strong proficiency in C++, Python, GoLang, or Fortran.
  • HDF5 Expertise: In-depth knowledge of HDF5 APIs and advanced features like parallel HDF5.
  • Parallel I/O: Experience with MPI I/O, POSIX I/O, or similar frameworks for concurrent/parallel data access.
  • Performance Optimization: Skills in profiling and optimizing I/O operations for large datasets.
  • Proficiency in SQL and experience with relational database management systems.
  • Knowledge of at least one orchestration and scheduling tool (e.g., Airflow, Prefect, Dagster).
Soft Skills
  • Strong problem-solving skills and ability to work in a multidisciplinary team.
  • Excellent communication skills for cross-team collaboration and documentation.

  • Data Engineer

    2 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeDataEngineer Tempo inteiro R$80.000 - R$98.000

    Job Title: Data Engineer - High-Performance Data SolutionsAbout the RoleWe are a human-centered organization dedicated to building innovative products and fostering a culture of collaboration, talent attraction, and integration.We deliver high-quality work for our clients, taking calculated risks and seeking optimal solutions to complex problems with our...

  • High-Volume Data Ingestion Specialist

    Há 5 dias

    Porto Alegre, Rio Grande do Sul, Brasil beBeeDataEngineer Tempo inteiro R$1.000.000 - R$1.500.000

    Job OverviewWe aim to craft exceptional products by adopting a human-centric approach. As a key team member, you will contribute to the development of data ingestion pipelines for processing merchant transactions.Key ResponsibilitiesCollaborate with engineers to design and implement solutions for high-volume, real-time or batch data ingestion while ensuring...

  • High-Performance Data Solutions Architect

    2 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeDataEngineer Tempo inteiro €80.000 - €125.000

    Job Title: Senior Data EngineerAt the forefront of technological innovation, we are seeking a highly skilled Senior Data Engineer to lead our data team in designing and implementing scalable, high-performance, secure, and reliable data pipelines.The ideal candidate will possess extensive experience in developing and optimizing complex data pipelines, with a...

  • High-Tech Data Specialist

    Há 2 dias

    Porto Alegre, Rio Grande do Sul, Brasil beBeeDataEngineer Tempo inteiro

    Job Opportunity: Data EngineerWe're seeking an experienced Data Engineer to join our team and contribute technical expertise to the buildout of data ingestion pipelines.Implement robust measures to guarantee data quality, reliability, and optimal pipeline performance.Share knowledge with a team of engineers and foster a culture of agile, data-driven...

  • Remote Part-time Data Collection Specialist

    2 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil Lifewood Data Technologies Sdn Bhd Tempo parcial

    **Job Title**: Remote Part-Time Data Collection Specialist**Location**: Remote (Based in Brazil)**Job Summary**:Join our innovative international project as a Remote Part-Time Data Collection Specialist We are seeking enthusiastic individuals who are eager to work from the comfort of their home while contributing to a dynamic team. This role offers flexible...

  • High-Performance System Specialist

    2 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeInfrastructure Tempo inteiro R$60.000 - R$120.000

    Job Summary:As a Site Reliability Engineer, you will be responsible for ensuring the efficiency and resilience of our services.Your goal is to maintain high standards of reliability and guarantee continuous operations. To achieve this, you will monitor system availability, latency, performance, and capacity to ensure efficient service delivery. You will also...

  • High Performance System Reliability Specialist

    Há 5 dias

    Porto Alegre, Rio Grande do Sul, Brasil beBeeHighPerformance Tempo inteiro R$120.000 - R$180.000

    Ensuring the efficiency and reliability of services is paramount. We are seeking a dynamic and challenge-driven professional to join our team.About The PositionThis role entails ensuring high standards of system availability, latency, performance, and capacity while managing incidents and collaborating on software architectures.System monitoring and incident...

  • High-Volume Data Expert

    2 semanas atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeDataEngineer Tempo inteiro R$80.000 - R$120.000

    We are seeking an experienced Data Engineer to contribute technical expertise to the ongoing buildout of data ingestion pipelines for processing merchant transactions. The ideal candidate will have a strong understanding of designing and implementing solutions for high-volume, real-time, or batch data ingestion using Python.Key Responsibilities:Design and...

  • High-Performance Infrastructure Specialist

    1 semana atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeReliability Tempo inteiro US$90.000 - US$120.000

    Job Description">We are seeking a highly skilled Site Reliability Engineer to join our team. The successful candidate will be responsible for ensuring the efficiency and resilience of our services, maintaining high standards of reliability, and guaranteeing continuous, uninterrupted operations.">The ideal candidate will have a strong knowledge of Linux and...

  • High-Performance Cloud Operations Specialist

    1 semana atrás

    Porto Alegre, Rio Grande do Sul, Brasil beBeeEngineer Tempo inteiro R$80.000 - R$100.000

    About the RoleWe are looking for a dynamic and challenge-driven professional to join our team as a Cloud Operations Engineer. As an engineer, you will be responsible for ensuring the efficiency and resilience of our services, maintaining high standards of reliability, and guaranteeing continuous, uninterrupted operations.Key ResponsibilitiesEnsure the...