AI Agent Evaluation Analyst

3 semanas atrás

Região Geográfica Imediata de Criciúma, Brasil Mindrift Tempo inteiro

Overview Mindrift connects domain experts with cutting‑edge AI projects, unlocking the potential of GenAI by tapping real‑world expertise from across the globe. Who we’re looking for We seek curious, intellectually proactive contributors who double‑check assumptions, play devil’s advocate, and thrive amid ambiguity and complexity. About the project We need QAs for autonomous AI agents in a new project that validates and improves complex task structures, policy logic, and agent evaluation frameworks. The role balances quality assurance, research, and logical problem‑solving. Responsibilities Review evaluation tasks and scenarios for logic, completeness, and realism. Identify inconsistencies, missing assumptions, or unclear decision points. Help define clear expected behaviors (gold standards) for AI agents. Annotate cause‑effect relationships, reasoning paths, and plausible alternatives. Think through complex systems and policies as a human would to ensure agents are tested properly. Work closely with QA, writers, or developers to suggest refinements or edge‑case coverage. Requirements Excellent analytical thinking: reason about complex systems, scenarios, and logical implications. Strong attention to detail: spot contradictions, ambiguities, and vague requirements. Familiarity with structured data formats: can read, not necessarily write JSON/YAML. Ability to assess scenarios holistically: identify missing or unrealistic elements. Good communication and clear writing (in English) to document findings. Desired skills Experience with policy evaluation, logic puzzles, case studies, or structured scenario design. Background in consulting, academia, olympiads (e.g. logic/math/informatics), or research. Exposure to LLMs, prompt engineering, or AI‑generated content. Familiarity with QA or test‑case thinking (edge cases, failure modes, "what could go wrong"). Some understanding of how scoring or evaluation works in agent testing (precision, coverage, etc.). Benefits Pay rates up to $15/hour depending on skills, experience, and project needs. Flexible, remote, freelance project that fits around primary professional or academic commitments. Participate in an advanced AI project and gain valuable experience to enhance your portfolio. Influence how future AI models understand and communicate in your field of expertise. Seniority level Internship Employment type Part‑time Job function Other Industries IT Services and IT Consulting Referrals increase your chances of interviewing at Mindrift by 2x. We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr

  • Multimodal AI Evaluation Analyst

    Há 2 horas

    Patos de Minas, MG, Brasil iMerit Technology Tempo inteiro

    Job Description: Multimodal AI Evaluation Analyst Target Language: English Company: iMerit Scholars Position Type: - Remote; Contract ($5/hour); ~10-40 hours per week Position Overview: iMerit Scholars is rapidly expanding our network of skilled AI Evaluation Analysts to help advance next-generation multimodal AI systems. This opportunity begins with a short...

  • AI Research Engineer

    2 semanas atrás

    Região Geográfica Intermediária de Juiz de Fora, Brasil Tether Operations Limited Tempo inteiro

    Join Tether and Shape the Future of Digital Finance At Tether, we’re pioneering a global financial revolution with innovative blockchain solutions that enable seamless, secure, and transparent digital transactions worldwide. About Tether Our product suite includes the trusted stablecoin USDT , energy-efficient Bitcoin mining solutions, advanced data...

  • [EMEA] - Lead AI Developer

    3 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil Ciandt Tempo inteiro

    We are tech transformation specialists, uniting human expertise with AI to create scalable tech solutions. With over 7,400 CI&Ters around the world, we’ve built partnerships with more than 1,000 clients during our 30-year history. Artificial Intelligence is our reality. We are looking for AI-first engineers who use Generative AI as a foundation of software...

  • German Language Specialist – AI Trainer

    3 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil Invisible Expert Marketplace Tempo inteiro

    German Language Specialist – AI Trainer Join Invisible Expert Marketplace and shape the future of AI by providing high‑quality German language training data. Your expertise will enrich AI systems that serve German speakers worldwide. Responsibilities Challenge advanced language models on German grammar, syntax, morphology, phonology, semantics, and...

  • AI Red Team Engineer

    3 semanas atrás

    Região Geográfica Intermediária de Sorocaba, Brasil LILT AI Tempo inteiro

    AI Red Team Engineer - Brazilian Portuguese 1 day ago Be among the first 25 applicants About LILT AI is changing how the world communicates — and LILT is leading that transformation. We’re on a mission to make the world’s information accessible to everyone, regardless of the language they speak. We use cutting‑edge AI, machine translation, and...

  • Business Analyst II

    3 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil Experian Group Tempo inteiro

    Overview We\'re looking for a Business Analyst with expertise in Generative AI, automation, and process improvement to join our Operational Excellence team supporting Compliance. You will focus on applying AI technologies like GPTs and bots to improve workflows and enhance productivity. Responsibilities Lead process discovery and identify automation...

  • Analyst – SAP IBP

    3 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil HCLTech Tempo inteiro

    Get AI-powered advice on this job and more exclusive features. International Opportunity for Analyst – SAP IBP! HCLTech is hiring for a high-impact international project in Mexico City, and we’re looking for experienced talent in SAP IBP (Integrated Business Planning) to help drive digital transformation in supply chain operations. Role: Applications...

  • Talent Technical Assessment Analyst

    3 semanas atrás

    Região Geográfica Intermediária de Juiz de Fora, Brasil BairesDev Tempo inteiro

    Talent Technical Assessment Analyst - Remote Work | REF# Join to apply for the Talent Technical Assessment Analyst - Remote Work | REF# role at BairesDev . At BairesDev®, we’ve been leading the way in technology projects for over 15 years, delivering cutting‑edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our...

  • Data Scientist II

    3 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil Microsoft Tempo inteiro

    Overview We are hiring multiple Data Scientists across LATAM to join the Microsoft 365 team. These are remote positions, allowing you to work from the comfort of your home! As part of our team, you will help shape one of Microsoft’s fastest‑growing cloud services. Your work will directly impact Copilot, enabling personalized, context‑aware experiences...

  • UX Research Analyst

    2 semanas atrás

    Região Geográfica Imediata de Criciúma, Brasil Pulse Labs Tempo inteiro

    This is a 3-month contract role. Contract extension is not guaranteed, but may occur dependent on performance, company work volume, and budget. Pulse Labs is empowering insights and elevating experiences for the world's top technology companies. Backed by investors including Google and Amazon, we're revolutionizing product development delivering...