Ai Agent Evaluation Analyst
Há 7 dias
About MindriftThis opportunity is only for candidates currently residing in the specified country.Your location may affect eligibility and rates.Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity.We believe in using the power of collective human intelligence to ethically shape the future of AI.What We DoThe Mindrift platform, launched and powered by Toloka, connects domain experts with cutting-edge AI projects from innovative tech clients.Our mission is to unlock the potential of GenAI by tapping into real-world expertise from across the globe.Who We're Looking ForWe're looking for curious and intellectually proactive contributors, the kind of person who double-checks assumptions and plays devil's advocate.OpportunityThis is a flexible, project-based opportunity well-suited for:Analysts, researchers, or consultants with strong critical thinking skillsStudents (senior undergrads / grad students) looking for an intellectually interesting gigPeople open to a part-time and non-permanent opportunityAbout the ProjectWe're on the hunt for QAs for autonomous AI agents for a new project focused on validating and improving complex task structures, policy logic, and agent evaluation frameworks.Throughout the project, you'll have to balance quality assurance, research, and logical problem-solving.What You'll Be DoingReviewing evaluation tasks and scenarios for logic, completeness, and realismIdentifying inconsistencies, missing assumptions, or unclear decision pointsHelping define clear expected behaviors (gold standards) for AI agentsAnnotating cause-effect relationships, reasoning paths, and plausible alternativesThinking through complex systems and policies as a human would to ensure agents are tested properlyWorking closely with QA, writers, or developers to suggest refinements or edge case coverageHow to Get StartedApply to this post, qualify, and get the chance to contribute to a project aligned with your skills, on your own schedule.Shape the future of AI while building tools that benefit everyone.RequirementsExcellent analytical thinking: Can reason about complex systems, scenarios, and logical implicationsStrong attention to detail: Can spot contradictions, ambiguities, and vague requirementsFamiliarity with structured data formats: Can read, not necessarily write JSON/YAMLAbility to assess scenarios holistically: What's missing, what's unrealistic, what might break?Good communication and clear writing (in English) to document your findings.Preferred ExperienceExperience with policy evaluation, logic puzzles, case studies, or structured scenario designBackground in consulting, academia, olympiads (e.g. logic/math/informatics), or researchExposure to LLMs, prompt engineering, or AI-generated contentFamiliarity with QA or test-case thinking (edge cases, failure modes, "what could go wrong")Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.)BenefitsGet paid for your expertise, with rates that can go up to $15/hour depending on your skills, experience, and project needsTake part in a flexible, remote, freelance project that fits around your primary professional or academic commitmentsParticipate in an advanced AI project and gain valuable experience to enhance your portfolioInfluence how future AI models understand and communicate in your field of expertise#J-*****-Ljbffr
-
Senior Ai Software Engineer
2 semanas atrás
Uberlândia, Brasil Workana Tempo inteiroWorkana is the largest remote work platform for talents in Latin America.Our new segment, Workana Premium, focuses on matching the most exceptional professionals with leading and innovative companies around the globe.Enjoy competitive compensation, dedicated support, and the flexibility of remote work within a dynamic environment that fosters collaboration...
-
[C] Senior Ai Engineer
2 semanas atrás
Uberlândia, Brasil Latamcent Tempo inteiroOverviewSenior AI EngineerFull-Time | Remote from Latin America | Required Overlap: 9AM - 3PM PST (6 hours)EkLine is seeking a Senior AI Engineer to lead the development and optimization of AI-powered features for our documentation platform.You will design and deploy AI agents, fine-tune large language models (LLMs), and build ML-driven tools for content...
-
Hausa Language Specialist
2 semanas atrás
Uberlândia, Brasil Invisible Expert Marketplace Tempo inteiroHausa Language Specialist – AI Trainer Apply for the Hausa Language Specialist – AI Trainer role at Invisible Expert Marketplace . Are you an experienced Hausa language professional eager to shape the future of AI? Review and annotate Hausa content for training data. Assess AI-generated outputs for accuracy and fluency. Identify and document error...
-
Content Writer
1 semana atrás
Uberlândia, Brasil Innodata Inc. Tempo inteiroJob Description: We are seeking highly analytical and detail-oriented professionals with hands-on experience in Red Teaming, Prompt Evaluation , and AI/LLM Quality Assurance . The ideal candidate will help us rigorously test and evaluate AI-generated content to identify vulnerabilities, assess risks, and ensure compliance with safety, ethical, and quality...
-
Content Editor/Author/Writer
Há 4 dias
Uberlândia, Brasil Innodata Inc. Tempo inteiroJob TitleContent Editor/Author/Writer at Innodata Inc.Job DescriptionWe are seeking highly analytical and detail-oriented professionals with hands-on experience in Red Teaming, Prompt Evaluation, and AI/LLM Quality Assurance.The ideal candidate will help us rigorously test and evaluate AI-generated content to identify vulnerabilities, assess risks, and...
-
Content Editor/Author/Writer
1 semana atrás
Uberlândia, Brasil Innodata Inc. Tempo inteiroJob Title Content Editor/Author/Writer at Innodata Inc. Job Description We are seeking highly analytical and detail‑oriented professionals with hands‑on experience in Red Teaming, Prompt Evaluation, and AI/LLM Quality Assurance. The ideal candidate will help us rigorously test and evaluate AI‑generated content to identify vulnerabilities, assess risks,...
-
Data Analyst
1 semana atrás
Uberlândia, Brasil Peoplebank Tempo inteiro2 days ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. Leading multinational software company with over 150,000 customers and millions of users across their product range, whose goal and is to change the way people work together with customer experience at the forefront of everything they do. Central modern...
-
AI Sales Product Specialist
2 semanas atrás
Uberlândia, Brasil Zendesk, Inc. Tempo inteiroAI Sales Product Specialist - LATAM page is loaded## AI Sales Product Specialist - LATAMremote type: Fully Flexiblelocations: Remote, São Paulo, Brazil: São Paulo, Braziltime type: Full timeposted on: Posted 4 Days Agojob requisition id: R32179## Job DescriptionAs an AI Sales Specialist, you'll be responsible for driving the growth and adoption of our full...
-
Especialista Sênior
Há 4 horas
Uberlândia, Brasil Neospace Ai Tempo inteiroAbout NeoSpaceNeoSpaceis an innovative startup shaping the future of technology with cutting-edge artificial intelligence solutions.We develop specialized AI models to optimize processes and transform our clients' experience.Our goal is to simplify people's lives and increase business efficiency by creating smarter and more accessible products and...
-
Especialista sênior
Há 7 dias
Uberlândia, Minas Gerais, Brasil NeoSpace AI Tempo inteiro R$60.000 - R$120.000 por anoAbout NeoSpaceNeoSpaceis an innovative startup shaping the future of technology with cutting-edge artificial intelligence solutions. We develop specialized AI models to optimize processes and transform our clients' experience. Our goal is to simplify people's lives and increase business efficiency by creating smarter and more accessible products and...
