Multimodal Ai Evaluation Specialist

As a Multimodal AI Evaluation Specialist, you will play a critical role in assessing the performance of advanced language models and multimodal systems.Your expertise will be invaluable in evaluating the accuracy, quality, and cultural alignment of model outputs across various modalities.Job SummaryWe are seeking an experienced professional to join our team as a Multimodal AI Evaluation Specialist.In this role, you will work with complex guidelines to assess the output of large language models (LLMs) and vision models (LVMs).Your evaluations will inform the development and fine-tuning of these models, ensuring they meet the highest standards of accuracy and reliability.Key Responsibilities:Evaluate LLM outputs across multiple modalities (text, image captions, video descriptions, and multimodal prompts).Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.Identify subtle errors, hallucinations, or biases in AI responses.Apply domain expertise and logical reasoning to resolve ambiguous or unclear outputs.Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency across the evaluation team.Escalate unclear cases and contribute to refining evaluation guidelines.Required Skills and QualificationsTo succeed in this role, you will need strong critical reading, observational, and evaluative skills across different modalities.You should be able to articulate nuanced judgments with precision and clarity, have excellent English comprehension (CEFR B2 or above), and familiarity with LLMs, generative AI, and multimodal systems.You will also need a Bachelor's degree/diploma or equivalent educational qualification, 1+ years of experience in data annotation, LLM evaluation, content moderation, or related AI/ML domains, demonstrated experience working with data annotation tools and software platforms, and a strong understanding of language and multimodal communication.