LiteLLM is the world's most popular AI Gateway, trusted by top companies like Adobe, Netflix, and NASA. Our platform empowers developers by providing secure, reliable access to LLMs and adjacent services, and we're looking for a Backend Engineer (New Grad) to help us build rock-solid guardrails and observability tooling at scale.

About The Role
You’ll focus on making our guardrails and logging world-class. You will own the backend code that ensures all guardrail calls are consistently logged, errors are surfaced to users (not silently swallowed), and our observability instrumentation works for real-world, high-volume traffic. Your attention to detail in areas like latency metrics, logging traceability, and backend guardrail registration will directly impact user trust in our security and compliance features.

Responsibilities

• Ensure all guardrail and policy enforcement calls (e.g., applyguardrail) are properly logged and traceable through our SpendLogs and relevant database tables

• Identify and fix areas where silent failures occur in guardrail creation, registration, and policy application—ensuring robust error handling and transparency to end users

• Work with observability integrations, including Datadog, Splunk, Prometheus, and OpenTelemetry, to maintain accurate, configurable, and usable monitoring and logging for backend systems

• Refactor and enhance our Prometheus integration to allow configurable latency histogram buckets, ensuring metrics scale for high-traffic environments

• Collaborate cross-functionally on backend engineering priorities (performance, reliability, security)

What We’re Looking For

• Recent graduate (Bachelor’s or Master’s in Computer Science or related field)

• Solid experience with Python and backend frameworks (e.g. FastAPI, Flask)

• Understanding of logging best practices, error handling, and secure backend development

• Exposure to monitoring, logging, or metrics platforms (Datadog, Splunk, Prometheus, OpenTelemetry)

• Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.)

• Driven to deliver high-quality backend code with strong guardrails, auditing, and debugging capabilities

• Eagerness to tackle hard bugs and ensure system transparency for end users