AI Benchmark Engineer | Native Language Specialist - Chinese (Taiwan) - Remote
LILT (Production) · Remote · Taiwan (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native Chinese (Taiwan)
- Freelance / Contract
- Fully remote
- Python + Shell
- AI Benchmarking
Join LILT as an independent contractor to build rigorous evaluation suites for large language models. You will create realistic task environments using native Chinese datasets, focusing on encoding, locale edge cases, and terminal workflows without relying on English translations.
The role requires strong Python and shell scripting skills to develop reference implementations and deterministic verification scripts. You will collaborate with a global team to calibrate task difficulty and maintain high-quality standards through a multi-layer review process.
What you'll do
- Design high-signal coding tasks in native Chinese to test multilingual model robustness
- Build realistic task environments using local datasets and files
- Identify failure points in AI performance through native language prompting
- Write reliable reference implementations and deterministic verifier scripts
- Analyze execution logs to calibrate task difficulty across model tiers
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in Chinese (Taiwan) with deep grammatical understanding
- High proficiency in English for communication and documentation
- At least one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Extensive experience with Terminal/CLI-based development workflows
- Deep understanding of Unicode normalization and encoding/decoding robustness
- Familiarity with locale-dependent conventions like collation and casing
Nice to have
- Background from leading technology companies or top-tier engineering universities
- Experience with bidirectional text handling or font fallbacks
- Familiarity with coding agents and their operational limits
Stack
- Python
- Shell Scripting
- Terminal-Bench
- Unicode
How you'll work
- Fully remote freelance position with flexible scheduling
- No fixed hours or micromanagement, work when you want
- Requires minimum 2 hours per day or 15-20 hours per week availability
- Independent contractor status (1099), responsible for own taxes
- Not suitable as a primary income source due to fluctuating demand
What you get
- Competitive rates with prompt payment processing
- Access to diverse AI and language technology projects
- Opportunity to join a global community of linguists and experts
- Flexible portfolio building across various industries
Hiring process
- Submit application with an updated CV in English
- Complete a GenAI assessment to evaluate technical skills
- Finalize onboarding and profile setup in the system