AI Benchmark Engineer | Native Language Specialist - Spanish (Argentina) - Remote
LILT (Production) · Remote · Argentina (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native Spanish (Argentina)
- Fully remote freelance
- Python & Shell scripting
- AI Benchmarking
- Flexible schedule
You will design and validate evaluation tasks for large language models, focusing on multilingual robustness in terminal workflows. The work requires creating high-quality assets in Spanish that test encoding, locale, and data processing edge cases without relying on English translations.
This freelance opportunity allows you to work independently on cutting-edge AI projects. You will collaborate with a global community to ensure benchmark integrity through rigorous quality control processes.
What you'll do
- Design realistic task environments using datasets in Spanish
- Identify failure points in AI handling of native language prompts
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty against various model tiers
- Participate in a four-layer human quality control process
What you'll need
- Native or near-native fluency in Spanish (Argentina)
- High proficiency in English
- At least one year of software engineering industry experience
- Strong skills in Python and standard shell scripting
- Experience with Terminal/CLI-based development workflows
- Understanding of Unicode normalization and encoding robustness
Nice to have
- Background from leading technology companies
- Degree from top-tier engineering university
- Familiarity with coding agents
- Knowledge of locale-dependent conventions like collation
Stack
- Python
- Shell scripting
- Terminal-Bench
- Claude Haiku
- Claude Sonnet
- Claude Opus
How you'll work
- Fully remote freelance contract
- Flexible hours with no fixed schedule
- Minimum 15-20 hours per week recommended
- Independent contractor responsible for own taxes
What you get
- Competitive rates with prompt payments
- Access to diverse AI and language projects
- Join a global community of experts
- No micromanagement or mandatory check-ins
Hiring process
- Submit application with CV in English
- Complete GenAI skills assessment
- Finalize onboarding and profile setup