
Quality Assurance for AI Agent Evaluation Analyst
Há 5 dias
AI Agent Evaluation Analyst - AI Trainer at Mindrift. We connect domain experts with AI projects to ethically shape the future of GenAI. This is a flexible, project-based opportunity that is remote and part-time.
Responsibilities- Reviewing evaluation tasks and scenarios for logic, completeness, and realism
- Identifying inconsistencies, missing assumptions, or unclear decision points
- Helping define clear expected behaviors (gold standards) for AI agents
- Annotating cause-effect relationships, reasoning paths, and plausible alternatives
- Thinking through complex systems and policies to ensure agents are tested properly
- Working closely with QA, writers, or developers to suggest refinements or edge case coverage
- Excellent analytical thinking: can reason about complex systems, scenarios, and logical implications
- Strong attention to detail: can spot contradictions, ambiguities, and vague requirements
- Familiarity with structured data formats: can read, not necessarily write JSON/YAML
- Can assess scenarios holistically: what’s missing, what’s unrealistic, what might break?
- Good communication and clear writing (in English) to document findings
- Experience with policy evaluation, logic puzzles, case studies, or structured scenario design
- Background in consulting, academia, olympiads (logic/math/informatics), or research
- Exposure to LLMs, prompt engineering, or AI-generated content
- Familiarity with QA or test-case thinking (edge cases, failure modes, what could go wrong)
- Understanding of scoring or evaluation in agent testing (precision, coverage)
- Get paid for your expertise, with rates that can go up to $15/hour depending on your skills and project needs
- Flexible, remote, freelance project that fits around your commitments
- Participate in an advanced AI project and build relevant experience
- Influence how future AI models understand and communicate in your field
- Internship
- Part-time
- Other
- IT Services and IT Consulting
-
Região Geográfica Intermediária de São Paulo, Brasil Mindrift Tempo inteiroOverview At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. The Mindrift platform, launched and powered by Toloka, connects domain experts with cutting-edge AI projects from innovative tech clients. Our mission is to unlock the potential of GenAI by tapping into...
-
Quality Assurance
1 semana atrás
São Paulo, Brasil Superbet Tempo inteiroOverview Join to apply for the Quality Assurance (QA) role at Superbet . It’s an exciting time to join Superbet as we enter new markets and develop new technologies. We’re looking for a new colleague to partner with our team to bring people initiatives to life and help deliver top-notch customer experiences. Job Summary We’re looking for a Quality...
-
Quality Assurance Analyst
2 semanas atrás
São Paulo, Brasil INDI Staffing Services Tempo inteiroAt INDI, we're passionate about empowering individuals and businesses worldwide. Our cutting-edge recruiters connect leading companies with top talent, fostering a dynamic environment where innovation thrives. Join us in shaping the future of work.Overview of the role:We are looking for a Quality Assurance Analyst to join our Analyst team and participate in...
-
Quality Assurance Analyst
3 semanas atrás
São Paulo, Brasil Novartis Brasil Tempo inteiroOverviewQuality Assurance Analyst role at Novartis Brasil. ResponsibilitiesOversight of all production and testing activities, ensuring compliance with cGxP, data integrity, and eCompliance within the Novartis Quality Management System. Support exception investigations; review and approval of production, QC, and AS&T records; MBR review. Support OpEx...
-
Quality Assurance Analyst
Há 7 dias
Sao Paulo, Brasil Novo Nordisk Tempo inteiroQuality Assurance Analyst **Category**:Quality**Location**:São Paulo, São Paulo, BR- **The Position** - As a Quality Assurance Analyst at Novo Nordisk, you will play a pivotal role with the following key responsibilities: - Operationalize the Quality Management System (QMS) and uphold Good Warehousing and Distribution Practices (GWDP) with Sanitary...
-
Quality Assurance Sr. Analyst
Há 22 horas
São José dos Campos, Brasil Johnson & Johnson Tempo inteiro**Job Function**: Quality **Job Sub Function**: Customer/Commercial Quality **Job Category**: Professional **All Job Posting Locations**: São José dos Campos, São Paulo, Brazil, São Paulo, Brazil Our expertise in Innovative Medicine is informed and inspired by patients, whose insights fuel our science-based advancements. Visionaries like you work on...
-
Software Quality Assurance Analyst
2 semanas atrás
São Paulo, São Paulo, Brasil Handoff Tempo inteiro US$600.000 - US$1.200.000 por anoWhy join us?Handoff is the AI agent that runs a construction company.We help remodelers automate estimating, streamline operations, and win more work - backed by real-time cost data, intuitive design, and workflows that "speak contractor." With over 10,000 monthly active users and $6B in annualized project volume already flowing through our platform, we're...
-
Freelance Software Developer
Há 3 dias
São Paulo, Brasil Mindrift Tempo inteiroFreelance Software Developer (Rust) - Quality Assurance (AI Trainer)At Mindrift, innovation meets opportunity. We believe in using the power of collective intelligence to ethically shape the future of AI. The Mindrift platform connects specialists with AI projects from major tech innovators. Our mission is to unlock the potential of Generative AI by tapping...
-
Quality Assurance Analyst
3 semanas atrás
São Paulo, Brasil Fiserv Tempo inteiroYou deserve to do what you love, and love what you do - a career that works as hard for you as you do. At Fiserv, we are more than 40,000 #FiservProud innovators delivering superior value for our clients through leading technology, targeted innovation and excellence in everything we do. You have choices - if you strive to be a part of a team driven to create...
-
Quality Assurance Analyst Jr
Há 7 dias
Sao Paulo, Brasil Novo Nordisk Tempo inteiroQuality Assurance Analyst Jr **Category**:Quality**Location**:São Paulo, São Paulo, BR- **The Position** - As a Quality Assurance Analyst Jr. at Novo Nordisk, you will take on the following key responsibilities: - Operationalize the Quality Management System (QMS) and uphold Good Warehousing and Distribution Practices (GWDP) with Sanitary Technical...