Ai Qa

2 semanas atrás

Ituiutaba, Brasil Bloom Talent Partners Tempo inteiro

Company Description
Bloom Talent Partners specializes in IT recruitment, focusing primarily on the Latam market. We bridge the gap between employers and exceptional IT professionals, ensuring a perfect cultural fit. For companies, we provide tailored IT talent, tech expertise, efficient hiring, and a personalized approach. For candidates, we offer tailored growth, tech insights, an effortless search, and supportive guidance to align with their career goals.
Role Description
Position: AI QA
Location: Brazil
About the role: As the AI QA, you will be the analytical architect of the quality and performance ecosystem that powers Made. You will be responsible for answering the critical question, "How do we measure the quality of a generative AI's output at scale?" This role is not about traditional QA, it is about creating a new discipline. You will design the frameworks, metrics, and systems that ensure our AI agents are not only responsive but also strategically valuable, creatively inspiring, and personality-aligned. You will be a key leader in a new function, working at the intersection of data science, product strategy, and creative insight to build a truly revolutionary product.
Key Responsibilities:
? Design and implement a comprehensive AI quality framework from the ground up, blending automated testing with expert human evaluation.
? Develop detailed Quality Scorecards to systematically assess the performance of our AI agents and their skills, such as content ideation or comment replies.
? Codify and maintain the distinct personalities and communication styles of our AI agents.
? Analyze and interpret the outputs from our Automated Quality Engine, identifying systemic trends, model weaknesses, and opportunities for improvement.
? Collaborate closely with our Creator-in-Residence (CIR) to translate their qualitative, expert feedback into actionable data and new test cases for the automated system.
? Partner with Product and Engineering teams to provide clear, data-driven feedback that directly informs model fine-tuning and the development of new AI skills.
? Own the curation and management of our "Golden Datasets", the benchmark collections of ideal prompts and responses that define excellence for our platform.
Key Requirements:
? Bachelor's degree in Computer Science, Information Systems, Computer Engineering, or a related field.
? Minimum of 3 years of experience with QA
? Experience with web application testing frameworks and tools (e.g., Jest, Cypress, Selenium).
? Experience with automated tests in Python.
? Experience with LangChain, LangGraph or similar tools. LangSmith is a plus.
? Experience with RAG and fine-tuning of LLMs.
? Experience in Git and code reviews.
? Understanding of database systems (e.g., MySQL, PostgreSQL, MongoDB) and writing optimized SQL queries.
? Understanding of A/B testing, data analysis, and performance metrics.
? Exceptional problem-solving abilities and attention to detail.
? Effective communication skills and the ability to work in a fast-paced, collaborative environment. Plus requirements:
? Familiarity with evaluation metrics for generative AI (e.g., coherence, factuality, diversity, style alignment, and hallucination detection).
? Experience with machine learning libraries such as TensorFlow or PyTorch.
? Experience in Python for web development.
? Experience with FastAPI.

  • Ai Safety Specialist

    1 semana atrás

    Ituiutaba, Brasil Bebeeredteamer Tempo inteiro

    We are seeking highly analytical and detail-oriented professionals to help us rigorously test and evaluate AI-generated content.The ideal candidate will identify vulnerabilities, assess risks, and ensure compliance with safety, ethical, and quality standards.The key responsibilities of this role include conducting Red Teaming exercises to identify...

  • Qa/Red Teaming Expert

    2 semanas atrás

    Ituiutaba, Brasil Innodata Inc. Tempo inteiro

    Job Description: We are seeking highly analytical and detail-oriented professionals with hands-on experience in Red Teaming, Prompt Evaluation , and AI/LLM Quality Assurance .The ideal candidate will help us rigorously test and evaluate AI-generated content to identify vulnerabilities, assess risks, and ensure compliance with safety, ethical, and quality...

  • Senior Backend Engineer

    1 semana atrás

    Ituiutaba, Brasil Yuna.Io Tempo inteiro

    Company Description Yuna's mission is to radically transform the way mental health support is accessed and delivered.By providing 24/7 instant and private support at the push of a button, we aim to address the significant gaps in mental health care accessibility.In a system where waiting times for therapists can reach up to 48 days and many people can't even...

  • Senior Backend Engineer

    Há 4 dias

    Ituiutaba, Brasil Yuna.Io Tempo inteiro

    Company Description Yuna's mission is to radically transform the way mental health support is accessed and delivered. By providing 24/7 instant and private support at the push of a button, we aim to address the significant gaps in mental health care accessibility. In a system where waiting times for therapists can reach up to 48 days and many people can't...

  • Senior Backend Engineer

    1 semana atrás

    Ituiutaba, Brasil Yuna.Io Tempo inteiro

    Company DescriptionYuna's mission is to radically transform the way mental health support is accessed and delivered. By providing 24/7 instant and private support at the push of a button, we aim to address the significant gaps in mental health care accessibility. In a system where waiting times for therapists can reach up to 48 days and many people can't...

  • Linguistics Expert

    2 semanas atrás

    Ituiutaba, Brasil Innodata Inc. Tempo inteiro

    About the role: We are seeking a highly analytical and detail-oriented Thai linguist to support AI training initiatives and linguistic content creation.This role is ideal for someone with a strong academic background in linguistics (syntax, semantics, pragmatics, morphology, phonology, sociolinguistics, etc.) and a passion for language, technology, and clear...

  • Linguistics Expert

    2 semanas atrás

    Ituiutaba, Brasil Innodata Inc. Tempo inteiro

    About the role:We are seeking a highly analytical and detail-oriented Thai linguist to support AI training initiatives and linguistic content creation. This role is ideal for someone with a strong academic background in linguistics (syntax, semantics, pragmatics, morphology, phonology, sociolinguistics, etc.) and a passion for language, technology, and clear...

  • Content Rater

    Há 9 horas

    Ituiutaba, Brasil Lite Tech Tempo inteiro

    Job Opportunity – Content Rater (On-site, PJ Contract) Location: Floor 5 - Av. Maria Coelho Aguiar, 215 - Bloco C - Jardim São Luís, São Paulo - SP, ********* Salary: R$ ***** (PJ) On-site, Full-time (PROJECT DURATION 3 MONTHS)What you will doEvaluate and rate content (text, images, audio, video) following clear guidelines.Review AI-generated responses...

  • Sr Marketing Data Analyst

    Há 10 horas

    Ituiutaba, Brasil Improvado Tempo inteiro

    Improvadois an AI-powered, unified platform designed for marketing teams in medium to large-scale enterprises and agencies, who are looking to automate complex marketing intelligence and reporting to make decisions with ease. Improvado gathers, organizes, and untangles marketing data to deliver instant insights through BI and AI, helping to eliminate...

  • Content Specialist

    1 semana atrás

    Ituiutaba, Brasil Bebeelinguist Tempo inteiro

    About UsWe are seeking a highly analytical and detail-oriented linguist to support AI training initiatives and linguistic content creation.Key Responsibilities:Create or edit linguistically-rich content including grammar guides, syntactic analyses, usage explanations, or examples for NLP pipelines.Identify and resolve issues related to ambiguity, bias, and...