AI Benchmark Engineer | Native Language Specialist - Korean - Remote
LILT (Production) · Remote · Korea (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Freelance contract
- Native Korean required
- Python + Shell
- Flexible schedule
Join LILT as an AI Benchmark Engineer to design and validate evaluation suites for coding agents. You will create high-quality tasks in Korean that test how well AI models handle complex locale, encoding, and text processing challenges without relying on English translations.
What you'll do
- Design realistic task environments using Korean datasets and files
- Identify failure points where AI struggles with native language prompts
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty against various model tiers like Haiku and Sonnet
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in Korean with deep grammatical understanding
- High proficiency in English for communication and documentation
- At least one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Extensive experience with Terminal and CLI-based development workflows
- Deep knowledge of Unicode normalization and encoding robustness
- Familiarity with locale-dependent conventions and safe string operations
Nice to have
- Background from leading technology companies or top-tier engineering universities
- Experience with bidirectional text handling and font fallbacks
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding agents
How you'll work
- Remote freelance opportunity with flexible hours
- No fixed hours, check-ins, or micromanagement
- Requires reliable availability for accepted tasks
- Typical commitment of 15 to 20 hours per week
- Contractors manage their own tax obligations
What you get
- Competitive rates with prompt payment processing
- Access to diverse and innovative AI projects
- Join a global community of linguists and experts
- Opportunity to expand professional portfolio
Hiring process
- Submit application with updated CV in English
- Complete GenAI assessment to evaluate skills
- Finalize onboarding and profile setup in system