AI Benchmark Engineer | Native Language Specialist - French (Canada) - Remote
LILT (Production) · Remote · Canada (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Fully remote
- Freelance / Contract
- Native French (Canada)
- Python + Shell
- AI Benchmarking
Join LILT as an independent contractor to build rigorous evaluation suites that test how AI handles non-English data and complex locale issues. You will create high-quality tasks in French (Canada) to measure model performance without relying on English translations, ensuring genuine multilingual capability.
What you'll do
- Design realistic task environments using datasets in French (Canada)
- Identify failure points where AI struggles with native language prompts
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty from Easy to Very Hard against model tiers
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in French (Canada) with deep grammar knowledge
- High proficiency in English for communication and documentation
- At least one year of industry software engineering experience
- Strong skills in Python, standard shell scripting, and data processing
- Experience with Terminal/CLI workflows and coding agents
- Understanding of Unicode normalization and encoding robustness
Nice to have
- Background from leading technology companies or top-tier universities
- Knowledge of locale-dependent conventions like collation and casing
- Experience with bidirectional text handling or font fallbacks
Stack
- Python
- Shell Scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Independent contractor role with flexible schedule
- No fixed hours or micromanagement
- Requires reliable availability for task delivery
- Minimum commitment of 15-20 hours per week recommended
- Contractors handle their own tax obligations
What you get
- Competitive rates with prompt payments
- Work on cutting-edge AI and language technology projects
- Access to a global community of linguists and experts
- Opportunity to expand portfolio across diverse domains
Hiring process
- Submit application with updated CV in English
- Complete a GenAI assessment to evaluate skills
- Finalize onboarding and profile setup in the system