Quality Evaluator – Fluent English (Colombia)
Anyone AI · Remote · Colombia - Fully Remote ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Part-time contract
- Fluent English required
- Up to 3 months
- AI evaluation focus
Anyone AI seeks English-fluent contractors to evaluate and compare responses from advanced AI models. You will assess accuracy, helpfulness, and clarity to help train frontier AI systems. This is a remote, part-time independent contractor position with a commitment of up to three months.
What you'll do
- Interact with AI models across various prompts and topics
- Compare multiple AI-generated responses side by side
- Assess accuracy, helpfulness, and instruction following
- Follow detailed rating guidelines and workflows
- Write clear explanations for evaluation decisions
- Apply evaluation criteria consistently
What you'll need
- Fluency in English with strong writing skills
- Extensive hands-on experience using LLMs
- Strong analytical judgment for quality assessment
- Excellent attention to detail
- Ability to follow complex instructions carefully
- Comfort switching between AI tools and workflows
Nice to have
- Experience in AI response evaluation or data annotation
- Background in editing, research, or fact-checking
- Familiarity with evaluation rubrics
- Experience with side-by-side comparison tasks
How you'll work
- Remote location
- Independent contractor status
- Part-time commitment
- Contract duration up to 3 months