AI Evaluators: Assessing A Shopping Assistant
Terac · Remote · United States ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- 20+ hours/week
- $50 per hour
- AI evaluation
- E-commerce focus
Join Terac to evaluate the accuracy and helpfulness of a new AI shopping assistant. You will analyze real-world e-commerce queries to identify logical errors and improve model response quality.
This position requires building structured rubrics and verifiers to judge future interactions. It is an ongoing remote opportunity suited for detail-oriented professionals in QA or data evaluation.
What you'll do
- Review real user interaction traces with the AI assistant
- Identify logical failures, inaccuracies, or poor recommendations
- Create structured rubrics to judge response quality
- Build verifiers for consistent evaluation of future responses
- Commit to at least 20 hours of work per week
What you'll need
- Experience in data evaluation, QA, or AI training
- Strong analytical skills for spotting subtle text errors
- Familiarity with e-commerce search and shopping experiences
- Ability to maintain a sustained workload of 20+ hours weekly
- Comfortable analyzing text interactions in depth
Nice to have
- Prior experience in prompt engineering
- Background in complex data annotation
- Experience in software testing
How you'll work
- Fully remote engagement
- Requires 20+ hours per week
- Work performed on internal custom platform