AI Benchmark Engineer | Native Language Specialist - German (Switzerland) - Remote
LILT (Production) · Remote · Switzerland (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native German required
- Fully remote freelance
- Python + Shell scripting
- AI benchmarking focus
- Flexible schedule
You will create high-quality evaluation tasks that test how AI models handle non-English data processing, locale edge cases, and terminal workflows. The work involves building realistic environments using German datasets and writing deterministic verifier scripts to ensure benchmark integrity.
What you'll do
- Design realistic task environments using German datasets and files
- Identify failure points where AI struggles with native language prompts
- Write reliable, deterministic verifier scripts for task validation
- Analyze execution logs to calibrate task difficulty levels
- Participate in a four-layer human quality control process
What you'll need
- Native or near-native fluency in German with deep grammatical knowledge
- High proficiency in English for communication and documentation
- Strong skills in Python, standard shell scripting, and data processing
- Experience with Terminal/CLI-based development workflows
- Familiarity with coding agents and prompt engineering concepts
- Understanding of Unicode normalization and encoding robustness
- Knowledge of locale-dependent conventions like collation and casing
Nice to have
- Background from leading technology companies or top-tier universities
- Experience with bidirectional text handling or font fallbacks
Stack
- Python
- Shell scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Remote freelance contract (1099)
- Flexible hours with no fixed schedule
- Minimum 15-20 hours per week expected
- Supplemental income rather than primary job
- Contractor responsible for own taxes
What you get
- Work when you want with no micromanagement
- Prompt payments and competitive rates
- Access to diverse AI and language projects
- Join a global community of experts
Hiring process
- Submit application with CV in English
- Complete GenAI skills assessment
- Finalize onboarding and profile setup