AI Evaluators and Quality Experts, United States

Summary

Priceless Center LLC is recruiting qualified professionals based in the United States for a three-month remote AI evaluation project, with the earliest anticipated start date of October 13, 2026. We are seeking analytical, detail-oriented English-speaking experts for two specialized project tracks: Factuality Evaluation and Core Quality Evaluation. Factuality Evaluators will review AI-generated responses, verify individual claims through credible online sources, and assess factual accuracy. Core Quality Evaluators will compare responses from different AI models and assess accuracy, helpfulness, clarity, instruction-following, and overall quality. Candidates must meet the relevant project requirements, complete the required certification assessment, and achieve the required passing score before entering production.

Responsibilities

• Review AI-generated responses across a wide range of topics. • Identify and separate individual claims that can be verified. • Research claims using credible, authoritative online sources. • Determine whether claims are accurate, inaccurate, or unverifiable. • Compare responses from two or more AI models side by side. • Evaluate accuracy, helpfulness, clarity, instruction-following, and overall response quality. • Provide clear, evidence-based explanations for evaluation decisions. • Follow detailed guidelines and multi-step workflows. • Maintain consistent judgments and high-quality evaluation records.

Minimum qualifications

Must be based in the United States. • Fluent in English, with strong reading comprehension and writing skills. • Strong critical-thinking and analytical abilities. • Excellent attention to detail. • Ability to understand and follow detailed instructions. • Ability to complete the required certification assessment. • For Factuality, strong web research and source-verification skills. • For Core Quality, hands-on experience using large language models (LLMs) and evaluating their outputs.

Preferred qualifications

• Previous experience in AI evaluation, data annotation, or AI response rating. • Experience with fact-checking, online research, or content quality assurance. • Ability to compare AI responses across multiple quality dimensions. • Experience working with multiple AI models and evaluation tools. • Strong written reasoning and evidence-based decision-making skills.