Data Optimization Specialist

Há 3 dias

São Leopoldo, Rio Grande do Sul, Brasil beBeeEngineering Tempo inteiro US$150.000 - US$250.000

As an Engineering Expert, you will play a key role in leading cutting-edge projects focused on optimizing data product runtime architecture within the Business Data Cloud. Your expertise will drive innovation and strategic decision-making.

Key Responsibilities
  • Analyze and refine Spark workloads to enhance performance and resource efficiency by tuning parameters, re-engineering data processing pipelines, and ensuring optimal execution strategies.
  • Implement monitoring solutions to detect and diagnose performance bottlenecks or failures in Spark applications, solving complex runtime issues related to resource allocation, execution errors, or data inconsistencies.
  • Collaborate with various parts of the organization to design and build a framework that allows Business Data Cloud workloads to scale effortlessly and elastically across distributed data environments.
  • Produce documentation on best practices for Spark optimization and conduct training sessions or workshops to enhance the broader team's expertise in performance tuning and support strategies.
  • Lead AI-automation initiatives to improve workload scalability and efficiency.
  • Mentor engineering teams and lead cross-functional global collaborations.
Required Skills and Qualifications
  • Deep familiarity with principles of distributed computing, including concurrency, fault tolerance, and network latency, which are essential for optimizing distributed data processing.
  • Comprehensive knowledge of Apache Spark architecture, including its core components like the driver, executor, and cluster manager, as well as Spark's execution model.
  • Expertise in performance profiling and tuning Spark applications, including optimizing resource allocation, parallelism, and shuffling processes to reduce execution time and improve efficiency.
  • Skilled in writing efficient SQL queries and transformations using SparkSQL and DataFrames, optimizing operations to reduce computation overhead.
  • Strong debugging skills to identify and resolve runtime issues, optimize code paths, and rectify configuration or environment-related problems.
  • Experience with tools such as Spark's web UI, Ganglia, Grafana, or Prometheus for monitoring application status and diagnosing performance bottlenecks.
  • Comprehensive understanding of AI, ML trends and their applications in the distributed Data query execution space.
  • Excellent leadership, mentorship, and communication skills.
  • Strategic vision and deep analysis skills to foresee industry advancement.
Benefits and Opportunities

SAP values diversity, equity, and inclusion. We offer a collaborative, caring team environment with opportunities for learning and development, recognition for individual contributions, and a variety of benefit options.

Inclusion and Legal Notice

SAP is an equal opportunity workplace committed to accessibility and accommodations for applicants with physical and/or mental disabilities. Qualified applicants will receive consideration for employment without regard to age, race, religion, national origin, ethnicity, age, gender, sexual orientation, gender identity or expression, protected veteran status, or disability.

  • Senior Data Processing Specialist

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeSpecialist Tempo inteiro R$125.000 - R$175.000

    Senior Data Processing SpecialistWe are seeking a seasoned Senior Data Processing Specialist to lead the optimization of our data product runtime architecture within SAP Business Data Cloud. The successful candidate will drive innovation and strategic decision-making by analyzing and refining Spark workloads to enhance performance and resource efficiency.The...

  • Optimize Data Architectures

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeDataOptimization Tempo inteiro US$80.000 - US$123.456

    Unlock Career Growth as a Data Optimization SpecialistAs a Data Optimization Specialist, you will spearhead projects aimed at refining data product runtime architecture within the SAP Business Data Cloud. Your expertise will inform strategic decision-making and drive technological advancements.Key Responsibilities:Analyze and refine Spark workloads to...

  • Data Services Specialist

    Há 21 horas

    São Leopoldo, Rio Grande do Sul, Brasil beBeeDataServices Tempo inteiro R$60.000 - R$113.000

    Job SummaryA Data Services Specialist is required to manage client relationships, develop reporting strategies and provide expert guidance on data management solutions.Key Responsibilities:

  • Chief Data Architecture Specialist

    Há 4 dias

    São Leopoldo, Rio Grande do Sul, Brasil beBeeData Tempo inteiro R$98.765 - R$123.456

    Lead Data EngineerThis role involves delivering high-impact data initiatives by leveraging Oracle expertise and modern AWS cloud capabilities.Maintain a robust infrastructure that combines design, development, and integration of data pipelines, models, and solutions.Optimize database environments for optimal performance, scalability, and availability.Develop...

  • Data Architecture Specialist

    1 semana atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeDistributed Tempo inteiro R$106.080 - R$141.600

    Expert Data Architect - Distributed ComputingThis challenging role involves leading teams to optimize data product runtime architecture within a cloud-based environment. As an expert in distributed computing, you will drive strategic decision-making and technological advancements.Key Responsibilities:Refine Spark workloads to enhance performance and resource...

  • Senior Data Architect

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeData Tempo inteiro R$63.420 - R$101.505

    Spearheading Data Optimization InitiativesAbout the Role:We are seeking a seasoned data expert to spearhead efforts in optimizing data product runtime architecture within our Business Data Cloud. This role will involve driving innovative projects and supporting performance and scalability solutions.Key Responsibilities:Analyze and refine Spark workloads,...

  • Chief Data Architect

    Há 6 dias

    São Leopoldo, Rio Grande do Sul, Brasil beBeeData Tempo inteiro R$1.000.000 - R$1.200.000

    Our organization seeks a seasoned data engineering leader to spearhead the design, development, and maintenance of complex data pipelines and integration solutions utilizing Oracle and AWS technologies. As a key member of our team, you will leverage your expertise in data governance, security, and quality standards to ensure the highest level of performance,...

  • Big Data Performance Expert

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeData Tempo inteiro R$80.000 - R$100.000

    High-Performance Engineer PositionBecoming a key contributor driving innovative projects that optimize data product runtime architecture within SAP Business Data Cloud is essential for optimizing the performance of big data processing.Responsibilities:Independently analyze and refine Spark workloads, applying advanced performance tuning techniques to achieve...

  • Facilities Data Manager

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeDataManagement Tempo inteiro

    Job Title: Data Management Facility SpecialistWe are seeking an experienced Data Management Facility Specialist to join our team. The successful candidate will be responsible for managing data related to facilities and real estate, ensuring high-quality data and providing guidance to the team.The ideal candidate will have a strong background in business...

  • Chief Performance Optimization Engineer

    2 semanas atrás

    São Leopoldo, Rio Grande do Sul, Brasil beBeeEngineering Tempo inteiro R$125.000 - R$175.000

    Key Engineering Expert roles focus on delivering high-quality solutions that drive business outcomes.">Job Description:We are seeking an experienced Key Engineering Expert to lead our efforts in optimizing data product runtime architecture within SAP Business Data Cloud. This individual will be responsible for driving innovation and strategic decision-making...