Embedded AI Engineer, On-Device Models
Deepgram · Remote · USA - Remote ·
COMPENSATION
$219k – $274k
Listed on the employer's job board (Ashby)
- Fully remote
- Senior or Staff level
- C, C++, Rust
- Edge AI optimization
- Non-NVIDIA hardware
Join the Partner Platform Engineering team to enable Deepgram’s voice AI models on diverse hardware platforms. You will work at the lowest layer of the edge stack, optimizing models for devices with tight memory, compute, and power constraints where standard toolchains fall short.
What you'll do
- Write and optimize custom kernels for non-NVIDIA accelerators, SoCs, DSPs, and NPUs
- Perform target-side quantization, operator fusion, and memory layout optimization
- Integrate with vendor NPU/DSP toolchains and extend edge inference runtimes
- Build performance-critical runtime code for embedded Linux, bare-metal, and RTOS
- Establish benchmarking for latency, accuracy, power, and memory footprint
- Partner with silicon vendors on SDK integration and low-level tuning
- Provide hardware constraint feedback to Applied ML and Research teams
What you'll need
- Experience delivering production systems on resource-constrained hardware
- Strong proficiency in C, C++, and/or Rust for performance-critical code
- Hands-on experience with model optimization like quantization and pruning
- Familiarity with edge inference runtimes such as ONNX Runtime or TFLite
- Understanding of CPU/GPU/NPU architectures and memory hierarchies
- Experience with bare-metal, RTOS, or embedded Linux environments
- Ability to scope ambiguous problems and explain technical tradeoffs
Nice to have
- Experience with real-time audio processing on embedded platforms
- Depth in ML optimization techniques like mixed-precision inference
- Background in hardware evaluation and systematic benchmarking
- Experience shipping AI features in consumer products at scale
- Familiarity with secure on-device deployment practices
Stack
- C
- C++
- Rust
- ONNX Runtime
- TensorRT
- TFLite
- ExecuTorch
- FreeRTOS
- Zephyr
- Embedded Linux
How you'll work
- Fully remote position
- AI-first mindset required for daily workflows
- Fast-paced environment with rapid technological change