AI Benchmark Engineer | Native Language Specialist - Spanish (Spain) - Remote
LILT (Production) · Remote · Spain (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native Spanish (Spain)
- Freelance / Contract
- Fully remote
- Python + Shell
- AI Benchmarking
Join LILT as an independent contractor to build rigorous evaluation suites for coding agents. You will create realistic task environments using Spanish datasets, ensuring assets remain in the target language to accurately measure multilingual handling without English translation aids.
The work involves identifying AI failure points, writing deterministic verifier scripts, and calibrating task difficulty against various model tiers. You will participate in a multi-layer quality control process to ensure benchmark integrity and grammatical accuracy.
What you'll do
- Design high-signal tasks to test multilingual software challenges
- Build realistic environments using native language datasets
- Identify failure points where AI struggles with native prompts
- Write reliable, deterministic verifier scripts for task validation
- Analyze execution logs to calibrate task difficulty levels
- Participate in four-layer human quality control and audit processes
What you'll need
- 1+ years of industry experience in software engineering
- Native or near-native fluency in Spanish (Spain)
- High proficiency in written and spoken English
- Strong skills in Python and standard shell scripting
- Extensive experience with Terminal/CLI development workflows
- Deep understanding of Unicode normalization and encoding robustness
- Familiarity with locale-dependent conventions and text I/O
Stack
- Python
- Shell Scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Independent contractor role (1099)
- Fully remote with flexible schedule
- No fixed hours or micromanagement
- Requires 15-20 hours per week availability
- Not suitable as primary income source
- Contractors responsible for own tax obligations
What you get
- Competitive rates with prompt payments
- Freedom to choose working hours
- Access to diverse AI and language projects
- Join a global community of experts
Hiring process
- Submit application with CV in English
- Complete GenAI skills assessment
- Finalize onboarding and profile setup