Architect, On-Device Inference

About Sarvam Sarvam is building the bedrock of Sovereign AI for India. The company is developing India’s full-stack sovereign AI platform, building across research, models, infrastructure and applications with a singular focus on making AI genuinely work for India. Sarvam works with leading enterprises and public institutions and is backed by Lightspeed, Peak XV, and Khosla Ventures. Sarvam partners with India’s leading brands, including Tata Capital, SBI Life, CRED, IDFC, and LIC.

About

the Role Own the technical architecture of Sarvam’s on-device products end-to-end - from model export and chipset-specific runtime selection up through the OS-layer voice input integration on Windows, macOS, and Android. You set the standards every other engineer on

the team works against, you are the technical interface to OEM partners (Qualcomm, Intel, NVIDIA, AMD, Apple), and you are accountable for hitting the published latency, footprint, and accuracy targets across every supported chipset. This is a player-coach role. We expect you to write code, debug at the kernel/driver layer when needed, and review every workbook before publication - but your highest-leverage work is in setting the architecture and unblocking

the team. What You’ll Do End-to-end latency and footprint budgets across all targets - Memory + runtime SLAs on NPUs / CPUs / GPUs. Runtime selection strategy per chipset: when OpenVINO vs. ONNX Runtime+EP, when TensorRT vs. CUDA-direct, when QNN vs. LiteRT, when CoreML vs. CPU fallback. The decision matrix is your deliverable. Model export pipeline: how models go from PyTorch to every target runtime, with shared infrastructure where possible and per-runtime customization where necessary. xPU selector and graceful-degradation logic: probing host capabilities, driver-version compatibility, fallback paths when the user picks an unavailable backend. OEM technical relationships: you are the technical face to Qualcomm, Intel, NVIDIA, AMD, and Apple counterparts. You explain perf wins and losses, escalate driver issues, and influence their roadmaps where we have leverage. Tech Lead duties for the optimization team: hiring bar, technical roadmap, weekly architecture reviews, mentorship of the three senior engineers.