AI Quality & Evaluation Lead
… · Remote · Poland ·
COMPENSATION
Salary available to subscribersListed on the employer's job board (Ashby)
- Fully remote
- 13-week contract
- AI quality focus
- LLM judge validation
- Senior level
Samsung Food is building an AI coaching experience within Samsung Health, where all guidance is generated rather than written. This engagement establishes the evaluation layer for output quality, focusing on tone, structure, and coaching logic. You will author failure taxonomies, codify them into rubrics, and validate automated judges against human grades before handing the system to internal owners.
What you'll do
- Create a failure taxonomy with 5-10 categories based on hand-graded traces
- Develop binary pass/fail criteria with clear definitions and examples
- Write annotation guidelines that require no interpretation
Loading…