Applied ML Engineer - Edge Devices
Deepgram · Remote · USA - Remote ·
COMPENSATION
$155k – $245k
Listed on the employer's job board (Ashby)
- Fully remote
- Senior or Staff level
- Edge ML deployment
- Python + PyTorch
- Non-NVIDIA hardware
Join the Partner Platform Engineering team to adapt Deepgram’s high-performance speech models for diverse hardware environments. You will work one layer above the metal, ensuring models run efficiently on edge servers, embedded platforms, and non-NVIDIA accelerators without altering the core hardware paradigm.
This applied engineering role bridges the gap between model research and embedded kernel development. You will own the process of reshaping operators, adjusting precision, and validating performance on real devices to deliver repeatable, production-grade voice AI solutions.
What you'll do
- Port speech models to non-NVIDIA and edge platforms by adapting structure and parameters.
- Make serving-side decisions on quantization, precision, and operator substitution for device constraints.
- Validate every port on real hardware using accuracy, latency, throughput, and memory benchmarks.
- Build automated deployment pipelines for model packaging, conversion, and versioning.
- Collaborate with Embedded AI Engineers to specify needs and adapt models for custom kernels.
- Partner with silicon vendors to align toolchains and model formats for working deployments.
- Provide feedback to research teams on edge constraints to improve future model portability.
What you'll need
- Production experience deploying ML models to edge or non-NVIDIA hardware is required.
- Working knowledge of quantization tradeoffs like INT8, FP16, and mixed precision calibration.
- Experience with edge inference runtimes such as ONNX Runtime, TFLite, or OpenVINO.
- Ability to modify model graphs, swap unsupported operators, and adjust architecture parameters.
- Strong proficiency in Python and PyTorch with a focus on reproducibility and testing.
- Comfort building automation tools for model conversion and deployment workflows.
- Clear communication skills to scope ports on unfamiliar platforms and drive results.
Nice to have
- Experience with speech, audio, or streaming real-time models.
- Familiarity with low-level kernels like CUDA, Metal, NEON, or vendor DSP code.
- Knowledge of model security, signing, and encrypted storage on deployed devices.
- Exposure to multiple accelerator families including Qualcomm, Apple, ARM, or Intel.
- Track record of building internal tooling to accelerate model porting processes.
Stack
- Python
- PyTorch
- ONNX Runtime
- TFLite
- ExecuTorch
- OpenVINO
- CUDA
- Metal
How you'll work
- Fully remote position
- AI-first mindset required for daily operations
- Fast-paced environment with rapid technological changes
- Expectation to actively use and experiment with advanced AI tools