Multimodal Ai Evaluator

iMerit seeks highly skilled professionals to evaluate the performance of advanced artificial intelligence systems.As a Multimodal AI Evaluator, you will be responsible for assessing the accuracy and appropriateness of AI outputs across various modalities, including text, image, video, and multimodal interactions.Key Responsibilities:Evaluate outputs generated by large language models (LLMs) and other AI systems.Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.Identify subtle errors, hallucinations, or biases in AI responses.Apply domain expertise and logical reasoning to resolve ambiguous or unclear outputs.Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency across the evaluation team.Collaborate with Project Managers and Quality Leads to meet accuracy, reliability, and turnaround benchmarks.Required Skills & Qualifications:Strong critical reading, observational, and evaluative skills across different modalities.Ability to articulate nuanced judgments with precision and clarity.Excellent English comprehension (CEFR B2 or above), additional languages a plus.Familiarity with LLMs, generative AI, and multimodal systems.Strong attention to detail and ability to apply guidelines consistently.Awareness of cultural and linguistic nuances, including potential bias and harm in AI outputs.Comfort with evolving workflows, rapid feedback cycles, and complex quality frameworks.Requirements:Bachelor's degree/diploma or equivalent educational qualification.1+ years of experience in data annotation, LLM evaluation, content moderation, or related AI/ML domains.Demonstrated experience working with data annotation tools and software platforms.Strong understanding of language and multimodal communication, instruction following in image generation, fact-checking, narrative coherence in video, etc.Ability to adapt quickly to changing project directions and fast-paced work environments.Previous experience creating or annotating complex data specifically for Large Language Model (LLM) training.Prior exposure to generative AI prompt engineering or LLM fine-tuning workflows is a plus.