QA Engineer
Titan AI · Remote · United States ·
COMPENSATION
$110k – $135k
Listed on the employer's job board (Ashby)
- Fully remote (US)
- 7+ years QA experience
- Python + AI testing
- Build QA from scratch
- Fintech compliance focus
You will establish the initial QA practice at Titan, creating evaluation frameworks and regression baselines for AI-driven banking tools. As an individual contributor, you will write test cases, build automated suites, and integrate quality gates into deployment pipelines.
The position requires hands-on work with Python and AI evaluation tooling to ensure model outputs meet strict financial compliance standards. You will triage bugs and produce audit-ready documentation for SOC 2 Type II requirements.
What you'll do
- Design and execute evaluation frameworks for LLM and agentic AI outputs
- Define behavioral contracts and own regression baselines for model behavior
- Write and maintain automated test suites for APIs and frontend surfaces
- Set up and enforce quality gates within CI/CD pipelines
- Triage production bugs and create reproduction cases for regression tests
- Produce test artifacts and documentation meeting SOC 2 Type II standards
What you'll need
- Seven or more years of experience in software QA engineering
- At least two years personally testing AI or ML systems
- Fluency in Python and experience with pytest, Playwright, or Selenium
- Hands-on experience with RAGAS, DeepEval, LangSmith, or similar tools
- Ability to trace failures from application layer to infrastructure
- Experience integrating QA gates into CI/CD pipelines end to end
Nice to have
- Experience in fintech, banking, or other regulated environments
- Familiarity with document processing pipelines and multi-agent architectures
- Knowledge of RAG validation techniques
- Experience with observability tooling such as Arize or Langfuse
Stack
- Python
- pytest
- Playwright
- Selenium
- RAGAS
- DeepEval
- LangSmith
- Azure
- REST APIs
How you'll work
- Fully remote within the United States
- Occasional travel to client sites and team offsites required
What you get
- Competitive base salary
- Meaningful equity package