Senior Backend Engineer

About the Company

LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services.

Responsibilities

• Build and scale the product, ensuring performance, reliability, and continuous improvement
• Ensure all guardrail and policy enforcement calls are properly logged and traceable
• Build and design CPU-level guardrails to cover common attacks on LLM APIs, MCP servers, and Agents
• Identify and fix areas where silent failures occur in guardrail creation and registration
• Work with observability integrations including Datadog, Splunk, Prometheus, and OpenTelemetry
• Enhance observability integrations to handle 1B+ requests/mo. with minimal latency and no memory leaks
• Collaborate cross-functionally on backend engineering priorities regarding performance, reliability, and security

Requirements

• Bachelor’s or Master’s in Computer Science or related field
• 1+ years of experience with Python and backend frameworks like FastAPI or Flask
• Understanding of logging best practices, error handling, and secure backend development
• Exposure to monitoring, logging, or metrics platforms (Datadog, Splunk, Prometheus, OpenTelemetry)
• Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.)
• Driven to deliver high-quality backend code with strong auditing and debugging capabilities

Benefits

• Competitive salary
• Health, dental, and vision benefits