AI Benchmark Engineer | Native Language Specialist - French (Belgium) - Remote
LILT (Production) · Remote · Belgium (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native French required
- Fully remote freelance
- Python + Shell scripting
- AI benchmarking focus
- Flexible schedule
You will create high-quality evaluation tasks that test how AI models handle non-English data processing, prompt language effects, and complex locale edge cases. The work involves building realistic task environments in French and writing deterministic verifier scripts to ensure benchmark integrity.
What you'll do
- Design realistic task environments using French datasets and files
- Identify failure points where AI struggles with native language prompts
- Write reliable, deterministic verifier scripts for task validation
- Analyze execution logs to calibrate task difficulty levels
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in French with deep grammatical knowledge
- High proficiency in English for communication and documentation
- At least one year of industry experience in software engineering
- Strong skills in Python, standard shell scripting, and data processing
- Extensive experience with Terminal/CLI-based development workflows
- Deep understanding of Unicode normalization and encoding robustness
- Familiarity with locale-dependent conventions and text I/O operations
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding agents
How you'll work
- Independent contractor role with no fixed hours
- Remote work from anywhere, subject to geographic restrictions
- Requires minimum 2 hours per day or 15-20 hours per week
- No health insurance, paid time off, or retirement benefits provided
- Contractor is responsible for their own tax obligations
Hiring process
- Submit application with an updated CV in English
- Complete a GenAI assessment to evaluate technical skills
- Finalize onboarding and profile setup in the system