AI Benchmark Engineer | Native Language Specialist - German (Germany) - Remote
LILT (Production) · Remote · Germany (Remote) ·
COMPENSATION
€•• – ••k
Listed on the employer's job board (Ashby)
- Native German required
- Freelance / Contract
- Fully remote
- Python + Shell
- AI Benchmarking
Join LILT as an independent contractor to design rigorous evaluation suites for Terminal-Bench tasks. You will create high-quality assets in German to test how well AI models handle complex locale, encoding, and prompt language challenges without relying on English translations.
This position is ideal for supplementing income rather than serving as a primary job. It requires consistent availability to deliver quality work on demand, with flexible scheduling and no fixed hours or micromanagement.
What you'll do
- Design realistic task environments using German datasets and files
- Identify failure points in AI performance through native language prompting
- Write deterministic verifier scripts and reference implementations
- Calibrate task difficulty from Easy to Very Hard against model tiers
- Participate in a four-layer human quality control and audit process
What you'll need
- Native or near-native fluency in German with deep grammatical knowledge
- High proficiency in English for communication and documentation
- At least one year of industry experience in software or prompt engineering
- Strong skills in Python, standard shell scripting, and data processing
- Extensive experience with Terminal/CLI-based development workflows
- Familiarity with coding agents and multilingual text processing pitfalls
Nice to have
- Background from leading technology companies or top-tier engineering universities
- Understanding of Unicode normalization and encoding robustness
- Knowledge of locale-dependent conventions like collation and casing
Stack
- Python
- Shell Scripting
- Terminal-Bench
- Coding Agents
How you'll work
- Fully remote freelance opportunity
- Flexible schedule with no fixed hours or check-ins
- Requires minimum 2 hours per day or 15-20 hours per week
- Independent contractor responsible for own taxes
- Not suitable as a primary income source due to fluctuating demand
What you get
- Competitive rates with prompt payment processing
- Access to diverse AI and language technology projects
- Join a global community of linguists and subject matter experts
- No micromanagement or rigid office hours
Hiring process
- Submit application with an updated CV in English
- Complete a GenAI assessment to evaluate technical skills
- Finalize onboarding and profile setup in the system