Worldwide AI Evaluators and Quality Experts
AI, Data Annotation & Quality Assurance · Worldwide · Full-time · Flexible, based on project requirements
Summary
evaluation project starting as early as October 13, 2026. We are seeking 2,000 experts across two specialized tracks: Factuality Evaluation and Core Quality Evaluation. Selected candidates will help improve AI systems by verifying factual claims, conducting web research, comparing AI-generated responses, and assessing accuracy, helpfulness, clarity, and overall quality. Candidates must demonstrate strong English proficiency, critical-thinking skills, attention to detail, and the ability to follow detailed evaluation guidelines. All candidates must pass the required certification assessment before entering production. Eligibility, project requirements, and available placements apply.
Responsibilities
RESPONSIBILITIES • Review AI-generated responses for factual accuracy, reliability, and quality. • Break responses into individual claims and verify them using credible online sources. • Identify inaccurate, misleading, unsupported, or unverifiable statements. • Compare AI model responses using structured side-by-side evaluation methods. • Assess accuracy, helpfulness, clarity, instruction-following, and overall response quality. • Provide clear, evidence-based explanations for evaluation decisions. • Follow detailed project guidelines and maintain consistent evaluation standards. • Complete required onboarding and certification before production.
Minimum qualifications
• Fluent English with strong reading comprehension and writing skills. • Strong critical-thinking, analytical, and problem-solving abilities. • Excellent attention to detail and ability to follow complex instructions. • Ability to conduct independent research and make evidence-based judgments. • Reliable internet access and the ability to work remotely. • Availability to complete the required certification assessment. • Ability to meet project quality standards and follow applicable workflows.
Preferred qualifications
• Experience with AI response rating, data annotation, fact-checking, or content quality assurance. • Strong online research and source-verification skills. • Experience comparing AI responses across multiple quality dimensions. • Familiarity with structured rating guidelines and multi-step evaluation workflows.
