About the role
We are looking for a Backend Engineer with 1 - 5 years of experience to own our critical guardrails and observability tooling. We need an engineer, ideally from a high-growth startup, who is excited to build for massive scale and is comfortable talking directly with developers at our largest enterprise customers like Nvidia, Adobe and NASA.
What you will do:
- Build and scale our core product, ensuring world-class performance and reliability for 1B+ monthly requests.
- Own the backend code for guardrails and logging, ensuring all calls are traceable and errors are surfaced correctly.
- Enhance our observability stack (Datadog, Splunk, Prometheus) to handle high-volume traffic with minimal latency.
- Develop in Python and/or Rust to build and design CPU-level guardrails against common attacks.
- Engage directly with developers at large enterprise customers several times a week to solve their challenges.
About LiteLLM
LiteLLM provides an open source Python SDK and Python FastAPI Server that allows calling 100+ LLM APIs (Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic) in the OpenAI format
