AI Engineer

2 semanas atrás

São Paulo, São Paulo, Brasil Perform Tempo inteiro
Overview

Join to apply for the AI Engineer - RAG Platform role at Perform

We're hiring an AI Engineer to design and build a production-grade RAG platform that powers our test autoscripting agent. This platform ingests our QA codebase and documentation, transforms them into embeddings, and serves relevant context (page objects, fixtures, helpers, examples) via a retrieval API—enabling high-quality LLM-generated tests. You'll own everything from ingestion to evaluation, including keeping the index fresh via Jenkins and optimizing for token cost and latency.

This role is ideal for someone who thrives in the intersection of LLM tooling, backend engineering, and developer productivity.

What You'll Do
  • Build and maintain a local RAG platform , including:
  • Loaders for Git, Confluence, Drive.
  • Code-aware chunking (AST/semantic) and embedding pipelines.
  • Vector indexing in ChromaDB with metadata and reranking.
  • FastAPI (or similar) retrieval service for the autoscripting agent.
  • Implement metadata filters (e.g., layer=page-object|fixture|helper|test, Git SHA, feature tags) and import-based neighbor expansion to optimize context.
  • Optimize for cost and performance : tune k values, context lengths, reranker thresholds, and cache frequent retrievals.
  • Build retrieval evaluation and telemetry : track recall, faithfulness, token usage, compile success of generated code, and wire alerts into Jenkins CI.
  • Manage access to Claude 4 Sonnet and other model APIs; help deploy self-hosted endpoints if needed (keys, quotas, audit logs).
  • Write runbooks and train the SDET team on how to use and troubleshoot the RAG system.
Tech Stack (Initial Plan)

Embeddings: mxbai-embed-large-v1 (text), bge-code-base (code)

Reranker: mxbai-rerank-base-v2

Vector store: ChromaDB (local)

Pipeline orchestration: LangChain (router by MIME/type)

Retrieval API: FastAPI

Evaluation: Telemetry + basic metrics (compile/run, cost, retrieval quality)

What You Bring
  • 4+ years in ML/AI or platform-oriented backend engineering , including 2+ years building LLM
  • Development withinRAG applications .
  • Strong experience with LangChain , vector DBs (ChromaDB, Qdrant, pgvector), and code-aware embeddings (BGE-code or similar).
  • Solid Python skills (FastAPI or Flask) and comfort reading Java to inform chunking and context design.
  • Experience with Jenkins , secrets management, and basic observability tooling (Grafana, Prometheus, LangSmith, or RAGAS).
  • Comfortable working with OpenAI/Anthropic APIs or deploying self-hosted endpoints, including handling keys, rate limits, and safety controls.
It is an asset if you have:
  • Experience with Claude-specific practices , structured prompting, and cost control techniques.
  • Familiarity with retrieval evaluation tools like RAGAS or LangChain Evaluators, plus A/B testing for prompt or routing strategies.
  • Understanding of security and compliance for developer-facing AI tools (PII handling, audit logging).
  • The SDET team focuses on test quality and final review of autoscripted code.
  • The Automation Agent Engineer tunes prompts and retrieval logic.
  • You own the RAG platform : indexing, retrieval quality, LLM orchestration, and CI integration.
Seniority level
  • Not Applicable
Employment type
  • Other
Job function
  • Engineering and Information Technology

Is this job not a good fit? Explore similar roles that may interest you.

#J-18808-Ljbffr

  • Ai Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil Elios Talent Tempo inteiro

    AI Engineer – Applied AI | Python | ML & Deep LearningWe're hiring an AI Engineer to help build and deploy intelligent systems that power smart features, automate workflows, and drive better decision-making across our platform.This role is ideal for someone eager to apply cutting-edge AI in production environments and grow within a high-impact engineering...

  • AI Infrastructure Engineer

    Há 3 dias

    São Paulo, São Paulo, Brasil BayRock Labs Tempo inteiro

    Join to apply for the AI Infrastructure Engineer role at BayRock Labs 4 days ago Be among the first 25 applicants Join to apply for the AI Infrastructure Engineer role at BayRock Labs Get AI-powered advice on this job and more exclusive features. About BayRock LabsAt BayRock Labs, we pioneer innovative tech solutions that drive business transformation....

  • Lead AI Engineer

    Há 7 dias

    São Paulo, São Paulo, Brasil beBeeArtificial Tempo inteiro R$80.000 - R$150.000

    We're on a mission to revolutionize the field of Artificial Intelligence by creating cutting-edge digital agents that can think, learn and adapt.About the Role:This is an exciting opportunity for skilled Engineers who are passionate about building sophisticated AI systems from scratch.You will be responsible for designing and developing multi-agent systems...

  • AI Solutions Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil beBeeArtificial Tempo inteiro US$80.000 - US$120.000

    ServiceNow DeveloperWe are seeking a talented and proactive AI Solutions Engineer with expertise in AI-driven capabilities.This role is key to enhancing our ServiceNow platform with AI-powered solutions, driving automation, efficiency, and intelligent workflows for our business and clients.Design, develop, and implement solutions using ServiceNow AI...

  • Visual AI Engineer – AI-Driven Automation

    2 semanas atrás

    São Paulo, São Paulo, Brasil CloudWalk, Inc. Tempo inteiro

    Join to apply for the Visual AI Engineer – AI-Driven Automation & Content Generation role at CloudWalk, Inc. 1 day ago Be among the first 25 applicants Join to apply for the Visual AI Engineer – AI-Driven Automation & Content Generation role at CloudWalk, Inc. About CloudWalk:We are not just another fintech unicorn. We are a pack of dreamers, makers,...

  • AI Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil MLabs Tempo inteiro

    Overview Our client is an innovative AI company building the future of healthcare with AI doctors and nurses, starting in Latin America. Their mission is to reinvent healthcare delivery in the region with AI-native medical solutions. This is an urgent hiring process for a high-impact role. You will have a unique opportunity to be the first AI Engineer in...

  • AI Engineer

    Há 2 dias

    São Paulo, São Paulo, Brasil MLabs Tempo inteiro

    Overview Our client is an innovative AI company building the future of healthcare with AI doctors and nurses, starting in Latin America. Their mission is to reinvent healthcare delivery in the region with AI-native medical solutions. This is an urgent hiring process for a high-impact role. You will have a unique opportunity to be the first AI Engineer in the...

  • Data/AI Engineer

    3 semanas atrás

    São Paulo, São Paulo, Brasil ablel Tempo inteiro

    We are seeking a fractional Data/AI Specialist to support our sales team on a per-hour/contract basis. This role will primarily serve as a Sales Engineer, providing technical expertise during sales calls and helping prospects understand the value of our data-driven solutions.Key ResponsibilitiesJoin sales calls as the technical expert, answering customer...

  • Data/AI Engineer

    2 semanas atrás

    São Paulo, São Paulo, Brasil ablel Tempo inteiro

    We are seeking a fractional Data/AI Specialist to support our sales team on a per-hour/contract basis . This role will primarily serve as a Sales Engineer , providing technical expertise during sales calls and helping prospects understand the value of our data-driven solutions. Key Responsibilities Join sales calls as the technical expert, answering...

  • Ai/Ml Engineer

    3 semanas atrás

    São Paulo, São Paulo, Brasil JSR Tech Consulting Tempo inteiro

    Long term, contract to hire position with a major financial firm.REMOTE.Payrate: 80 - 100 / hrRequisition for Senior Machine Learning Engineer (Generative AI Focus)Position Overview: We are seeking a highly skilled and experienced Senior Machine Learning Engineer to join our dynamic team. In the rapidly evolving world