AI Benchmark Engineer | Native Language Specialist - Indonesian - Remote
LILT (Production) · Remote · Indonesia (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Freelance contract
- Native Indonesian
- Python + Shell
- AI benchmarking
Join LILT as an AI Benchmark Engineer to design rigorous evaluation suites for multilingual software challenges. You will create high-quality tasks in Indonesian to test how AI models handle non-English data, encoding edge cases, and terminal workflows without relying on English translations.
What you'll do
- Design realistic task environments using datasets in Indonesian
- Identify failure points where AI struggles with native language prompts
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty against various model tiers like Haiku and Sonnet
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in Indonesian with deep grammatical knowledge
- High proficiency in English for communication and documentation
- At least one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Experience with Terminal/CLI-based development workflows
- Understanding of Unicode normalization and locale-dependent conventions
Nice to have
- Background from leading technology companies or top-tier universities
- Familiarity with coding agents and their operational limits
- Knowledge of bidirectional text handling and font rendering issues
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Fully remote freelance position
- Flexible schedule with no fixed hours or micromanagement
- Requires reliable availability for accepted tasks
- Minimum commitment of 2 hours per day or 15-20 hours per week
- Contractors manage their own tax obligations
What you get
- Competitive rates with prompt payment processing
- Access to diverse AI and language technology projects
- Opportunity to join a global community of experts
- Freedom to choose workload volume based on preference
Hiring process
- Submit application with an updated CV in English
- Complete a GenAI assessment to evaluate technical skills
- Finalize onboarding and profile setup in the system