AI Benchmark Engineer | Native Language Specialist - Chinese (Hong Kong) - Remote
LILT (Production) · Remote · Hong Kong (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native Chinese (HK)
- Freelance / Contract
- Fully remote
- Python + Shell
- AI Benchmarking
You will build rigorous evaluation suites for Terminal-Bench, testing how AI handles non-English data processing and locale edge cases. The work involves creating realistic task environments in your native language to measure multilingual capabilities accurately.
What you'll do
- Design high-quality coding tasks in native language to test AI agents
- Build realistic task environments using local datasets and files
- Identify failure points in AI performance through prompting tests
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty by analyzing execution logs against model tiers
- Participate in multi-layer human quality control and audit processes
What you'll need
- 1+ years of industry software engineering experience
- Native or near-native fluency in Chinese (Hong Kong)
- High proficiency in English for communication and documentation
- Strong skills in Python and standard shell scripting
- Experience with Terminal/CLI-based development workflows
- Deep understanding of Unicode, encoding, and locale conventions
Nice to have
- Background from leading technology companies
- Degree from top-tier engineering university
- Familiarity with coding agents and AI tools
Stack
- Python
- Shell Scripting
- Terminal-Bench
- Claude Haiku/Sonnet/Opus
How you'll work
- Independent contractor role with flexible schedule
- Fully remote work location
- Requires 15-20 hours per week availability
- No fixed hours or mandatory check-ins
- Not suitable as primary full-time income source
What you get
- Competitive rates with prompt payment
- Access to diverse AI and language projects
- Join global community of linguists and experts
- Flexible workload based on project demand
Hiring process
- Submit application with CV in English
- Complete GenAI assessment to evaluate skills
- Finalize onboarding and profile setup