Quality Evaluator – Fluent English (Brazil)
Anyone AI · Remote · Brazil ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Part-time contract
- Fluent English required
- Up to 3 months
Anyone AI creates high-quality training and evaluation data for frontier AI models. As a Quality Evaluator, you will interact with large language models, compare their outputs side by side, and provide written explanations to enhance system reliability.
What you'll do
- Interact with various AI models across diverse prompts and topics
- Compare multiple AI-generated responses to determine the best fit
- Assess accuracy, clarity, helpfulness, and instruction following
- Follow detailed rating guidelines and multi-step evaluation workflows
- Write clear explanations to support your evaluation decisions
- Apply evaluation criteria consistently across all assigned tasks
What you'll need
- Fluency in English with strong reading and writing skills
- Extensive hands-on experience using large language models
- Strong analytical judgment for assessing response quality
- Excellent attention to detail and ability to follow complex instructions
- Ability to explain evaluation decisions clearly and convincingly
- Comfort switching between different AI tools and workflows
Nice to have
- Previous experience in AI response evaluation or data annotation
- Background in editing, research, fact-checking, or analytical writing
- Familiarity with evaluation rubrics and side-by-side comparison tasks
How you'll work
- Fully remote position
- Independent contractor status
- Part-time commitment
- Contract duration up to 3 months