Content Evaluator

4 weeks ago Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.OverviewJob Opportunity: Content Quality Assurance Specialist.We are seeking highly analytical and detail-oriented professionals with hands-on experience in testing and evaluating AI-generated content to identify vulnerabilities, assess risks, and ensure compliance with quality standards.ResponsibilitiesConduct rigorous testing of large language models (LLMs) to identify adversarial outputs.Evaluate and stress-test AI prompts across multiple domains (e.g., finance, healthcare, security) to uncover potential failure modes.Develop and apply test cases to assess accuracy, bias, toxicity, hallucinations, and misuse potential in AI-generated responses.Collaborate with data scientists and safety researchers to report risks and suggest mitigations.Perform manual QA and content validation across model versions, ensuring factual consistency, coherence, and guideline adherence.Create evaluation frameworks and scoring rubrics for prompt performance and safety compliance.Document findings, edge cases, and vulnerability reports with high clarity and structure.RequirementsProven experience in AI red teaming, LLM safety testing, or adversarial prompt design.Familiarity with NLP tasks and ethical considerations in generative AI.Strong background in Quality Assurance, content review, or test case development for AI/ML systems.Understanding of LLM behaviors, failure modes, and model evaluation metrics.Excellent critical thinking, pattern recognition, and analytical writing skills.Ability to work independently, follow detailed evaluation protocols, and meet tight deadlines.Next StepsTo proceed further in the evaluation process, you will need to complete two assessments: a linguistic and analytical skills assessment and an English proficiency test (C1 or C2 level required).Once both assessments are successfully completed, you will be eligible for onboarding.You will also receive an invitation to our internal job platform.Please take a few minutes to register and complete your profile.If interested, kindly share your resume at ******.SenioritiesEntry levelEmployment typeFull-timeJob functionEducation and TrainingIndustriesBusiness Consulting and ServicesReferrals increase your chances of interviewing at Innodata Inc. by 2xGet notified about new Evaluator jobs in Salvador, Bahia, Brazil.Related rolesSoftware Engineer, Platform - Salvador, BrazilBackend Software Engineer (Node, Python, Ruby on Rails or Go)Front-End Developer - Remote Work | REF#****Salvador, Bahia, Brazil R$9,******-R$9,******** month agoWe're unlocking community knowledge in a new way.Experts add insights directly into each article, started with the help of AI.#J-*****-Ljbffr