AI Benchmark Engineer | Native Language Specialist - French (France) - Remote
LILT (Production) · Remote · France (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Freelance / Contract
- Native French required
- Python & Shell scripting
- Flexible schedule
Join LILT as an AI Benchmark Engineer to design rigorous evaluation suites for large language models. You will create high-quality tasks in French to test multilingual robustness, focusing on prompt effects and locale edge cases without relying on English translations.
This independent contractor role allows you to work on cutting-edge AI projects that measure how models handle complex terminal workflows and non-English data processing. You will collaborate with a global community of experts to ensure benchmark integrity and fairness.
What you'll do
- Design realistic task environments using French datasets and files
- Identify failure points where AI struggles with native language prompts
- Write deterministic verifier scripts and reference implementations
- Analyze execution logs to calibrate task difficulty levels
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in French with deep grammatical knowledge
- High proficiency in English for communication and documentation
- Minimum one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Extensive experience with Terminal/CLI-based development workflows
- Understanding of Unicode normalization and encoding robustness
- Familiarity with locale-dependent conventions and text I/O operations
Nice to have
- Background from leading technology companies or top-tier universities
- Working familiarity with coding agents and AI evaluation tools
- Knowledge of bidirectional text handling and font fallbacks
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Fully remote independent contractor position
- Flexible hours with no fixed schedule or micromanagement
- Requires reliable availability for accepted tasks
- Typical commitment of 15-20 hours per week
- Not suitable as a primary full-time income source
What you get
- Competitive rates with prompt payment processing
- Freedom to choose workload and working hours
- Access to diverse and innovative AI projects
- Opportunity to expand professional portfolio and skills
Hiring process
- Submit application with an updated CV in English
- Complete a GenAI assessment to evaluate technical skills
- Finalize onboarding and profile setup in the system