AI Benchmark Engineer | Native Language Specialist - Spanish (Mexico) - Remote
LILT (Production) · Remote · Mexico (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native Spanish (Mexico)
- Fully remote freelance
- Python + Shell scripting
- AI benchmarking
- Flexible schedule
Join LILT as an AI Benchmark Engineer to design and validate Terminal-Bench tasks for multilingual software challenges. You will create high-quality assets in Spanish to measure model robustness without relying on English translations. This role focuses on identifying failure points in AI handling of non-English data and complex locale edge cases.
What you'll do
- Design realistic task environments using datasets in native Spanish
- Identify failure points where AI struggles with native language prompts
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty against various model tiers like Haiku and Sonnet
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in Spanish with deep grammatical understanding
- High proficiency in English for communication and documentation
- At least one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Experience with Terminal/CLI workflows and coding agents
- Knowledge of Unicode normalization and encoding robustness
Nice to have
- Background from leading technology companies or top-tier universities
- Understanding of locale-dependent conventions like collation and casing
- Experience with bidirectional text handling or font fallbacks
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding agents
How you'll work
- Fully remote freelance opportunity
- Flexible schedule with no fixed hours or micromanagement
- Requires minimum 2 hours per day or 15-20 hours per week
- Independent contractor responsible for own taxes
- Not suitable as primary income source due to fluctuating demand
What you get
- Competitive rates with prompt payments
- Access to diverse AI and language technology projects
- Join a global community of linguists and experts
- No health insurance or paid time off provided
Hiring process
- Submit application with updated CV in English
- Complete GenAI assessment to evaluate skills
- Finalize onboarding and profile setup in system