LiteLLM Security: Your AI Gateway Is a Secrets Manager. Benchmark It Like One

July 2026

An AI gateway is a secrets manager wearing a router's clothes. LiteLLM, the most widely adopted of them, runs in production at Netflix, Lemonade and Rocket Money (InfoWorld, 2026), holding every model provider key those teams use behind a single proxy. It should be run at the standard you would apply to an actual secrets manager.

That is the thread running through a bad year for LiteLLM. Four different classes of attack hit it in 2026, and they had nothing in common but the ending. Each exploited a different flaw, and every one reached the same provider keys.

Four separate LiteLLM security failures reached the public record in 2026, one now on CISA's Known Exploited Vulnerabilities list, and they share no common bug. What they share is a common cause: the gateway concentrates every provider key, the salt that decrypts them, and a plaintext log of every prompt behind one always-decrypting process, without the containment a secrets manager brings to that concentration: least privilege that stops one foothold from reaching every key, and an audit trail that records what was read.

What a LiteLLM gateway holds

An AI gateway earns its place. It gives you one API surface across a dozen providers, tracks spend, enforces rate limits, and fails over when a provider goes down. That is real work, and it is why LiteLLM sits in production at scale. To do that work, it has to hold three things.

First, the real provider keys. The OpenAI, Anthropic, Bedrock and Azure keys, the ones that bill your accounts. LiteLLM writes them to a Postgres database, encrypted with a salt key you set through an environment variable (LiteLLM docs, 2026).

Second, the virtual keys it mints in front of those. Each one is scoped with a budget and a model allowlist, so a leaked virtual key is capped at its budget rather than opening your whole provider account. This is good security engineering, and it is worth keeping.

Third, everything that flows through. Every prompt and every response transits the proxy, and by default LiteLLM stores that traffic in its spend logs as plaintext (LiteLLM security FAQ, 2026).

So the gateway solves key sprawl by replacing many scattered secrets with one concentrated store. That is the improvement and the risk in a single act.

Why four different LiteLLM bugs all end in the same place

Four separate failures hit LiteLLM in 2026.

• CVE-2026-42271, command injection in the MCP test endpoints. CVSS 8.7, added to CISA's KEV catalogue on 8 June 2026 with confirmed active exploitation (CISA, 2026; NVD, 2026). It runs commands on the proxy host, which reaches every provider key the gateway stores.
• CVE-2026-47101, 47102 and 40217, a privilege-escalation chain disclosed by Obsidian Security. Rated CVSS 9.9, fully fixed in 1.83.14-stable, and a different issue from the KEV bug despite the two being frequently confused (Obsidian Security, 2026). It escalates a low-privilege user to proxy admin, exposing the provider keys the gateway holds.
• CVE-2026-42208, SQL injection, exploited within 36 hours of disclosure (The Hacker News, 2026). Attackers read straight from the database tables that store the provider keys (Sysdig, 2026).
• The March PyPI compromise. Attackers stole the project's publishing token through a compromised dependency in its own CI pipeline, pushed two malicious releases, and the payload went straight for cloud keys, API keys and Kubernetes tokens before the packages were pulled (BerriAI, 2026; Datadog Security Labs, 2026).

Four different bug classes. Command injection, privilege escalation, SQL injection, a poisoned dependency. Every one lands in the same place: the host that holds every provider key, the salt that decrypts them, and the plaintext logs of every prompt.

The architecture sets the blast radius, not the bug. This is not a story about one project's bad month, and it is not that LiteLLM is badly built. It is that concentration and compromise are the same coin. The property that makes the gateway useful, one place that holds everything so you do not have to, is the property that makes any single foothold reach everything. Four different bugs proved the point four different ways. What each one reached was decided long before the bug existed, by the decision to put every key in one place. That concentration is not hypothetical risk, either. Our own research clocked exposed cloud credentials on AI infrastructure being validated against live APIs in under 90 seconds (CyberDesserts, 2026). The window between a foothold and a drained account is measured in seconds, not hours.

You already know how to do this. You just did it somewhere else.

The gateway rarely gets benchmarked as a secrets manager, because it arrives as AI infrastructure and gets judged on AI-infrastructure things: does it route to every provider, does it track spend, does it speak the OpenAI API. The question that matters for the most credential-dense service in the stack is easy to skip, because the tool does not present as a secrets store.

Hold it against the tool you already trust for this job. HashiCorp Vault is not magic, and it is not immune. Its own threat model says plainly that an attacker with code execution on the host can compromise its data (HashiCorp, 2026), and Vault has had its own remote-code-execution flaws. The difference is what a compromise costs you. Vault issues short-lived, scoped credentials that expire on their own and can be revoked by lease, gates every secret read behind a policy, and writes each one to an audit log. Its seal keeps the decryption key out of storage, so a stolen disk or backup yields ciphertext rather than secrets. Steal a client's token and you get only what that identity was allowed to read, for as long as its lease lasts, with a record of it. Reaching everything means compromising the Vault process itself, which is a far higher bar than a bug in one app that talks to it. Vault makes total exposure the hard case.

Now LiteLLM. The same foothold pays out differently. It holds the long-lived provider account keys themselves, not short-lived derived ones, so what the attacker takes does not expire and cannot be revoked by lease. Its audit logging, an Enterprise feature, records admin actions like creating and deleting keys, not the moment a provider credential is read, and an attacker on the host reads those keys straight from the database or the environment, a path the proxy never sees. And the salt key that decrypts those credentials is a static value in an environment variable that, by LiteLLM's own documentation, cannot be rotated for the life of the database without breaking every stored key (LiteLLM docs, 2026), so it sits in the same blast zone as the data it protects. The prompt logs are plaintext by default. Neither system is immune, but when LiteLLM is the one that falls, it takes everything with it and leaves no trace.

There is a reason the gateway ends up here, and it is worth being fair about. A secrets manager can hold a credential it never has to use. The gateway has to use the provider key on every single call, so it cannot lock the key away the way Vault locks away a database password. It has to keep something usable, close to the request path, all the time.

That was a genuine constraint. It is now much less of one. The model providers have started shipping keyless authentication. OpenAI released workload identity federation on 26 May 2026, letting a workload exchange an identity token it already holds for a short-lived access token, with no long-lived key to store (OpenAI, 2026). Anthropic made the same capability generally available for the Claude API on 17 June 2026 (Anthropic, 2026). The gateway can broker one of those short-lived tokens per request rather than sitting on a permanent key to every account. The strongest protection is no longer out of reach. Moving to it is a deliberate architecture choice rather than the path of least resistance, so the guidance is simple: treat keyless, short-lived provider auth as the target state, and store a long-lived key only until you can get there.

What good AI gateway security looks like

You do not need a new product. You need to treat the gateway as what it already is, and three moves get you most of the way.

Stop storing the long-lived key. This is the keyless move above, made concrete: point the gateway at the provider's workload-identity federation, or a real secrets manager that mints short-lived tokens, so the worst a compromise yields is the traffic in flight rather than a permanent key to every account. It is the single change that shrinks the blast radius most.

If you must store keys, seal and scope. Keep the credential store separate from the process handling untrusted requests, so a bug in the HTTP layer does not automatically read the vault. Scope each provider key down at the provider side, so even a full proxy compromise inherits limited authority rather than account-wide power. The same discipline applies wherever a gateway concentrates authority for agentic workflows: the more a single control point holds, the more a single compromise takes.

Shrink what is concentrated. Turn off the plaintext prompt logging you do not need. Keep the admin interface and the test and preview endpoints, the ones the command injection abused, off any network an attacker can reach. Patch on a cadence that matches the crown jewels the gateway holds, not the developer convenience it first presents as. The gateway is one piece of a wider AI agent security problem, and it is the piece that holds the keys.

You did not install a convenience. You installed a secrets manager. Everything that follows depends on treating it like one.

References:

1. InfoWorld, "LiteLLM: An open-source gateway for unified LLM access." https://www.infoworld.com/article/3975290/litellm-an-open-source-gateway-for-unified-llm-access.html
2. CISA, "CISA Adds Two Known Exploited Vulnerabilities to Catalog" (8 June 2026). https://www.cisa.gov/news-events/alerts/2026/06/08/cisa-adds-two-known-exploited-vulnerabilities-catalog
3. NVD, CVE-2026-42271. https://nvd.nist.gov/vuln/detail/CVE-2026-42271
4. Obsidian Security, LiteLLM privilege-escalation chain disclosure (CVE-2026-47101, 47102, 40217).
5. The Hacker News, "LiteLLM CVE-2026-42208 SQL Injection Exploited within 36 Hours of Disclosure." https://thehackernews.com/2026/04/litellm-cve-2026-42208-sql-injection.html
6. Sysdig, "CVE-2026-42208: Targeted SQL injection against LiteLLM's authentication path." https://www.sysdig.com/blog/cve-2026-42208-targeted-sql-injection-against-litellms-authentication-path-discovered-36-hours-following-vulnerability-disclosure
7. BerriAI, "[Security]: litellm PyPI package (v1.82.7 + v1.82.8) compromised" (GitHub Issue #24518). https://github.com/BerriAI/litellm/issues/24518
8. Datadog Security Labs, "LiteLLM and Telnyx compromised on PyPI: Tracing the TeamPCP supply chain campaign." https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-teampcp-supply-chain-campaign/
9. LiteLLM documentation, salt key and provider credential storage. https://docs.litellm.ai/docs/proxy/docker_quick_start
10. LiteLLM security FAQ / spend logs documentation. https://docs.litellm.ai/docs/proxy/ui_logs
11. HashiCorp, "Vault Security Model." https://developer.hashicorp.com/vault/docs/internals/security
12. OpenAI, API changelog (workload identity federation, 26 May 2026). https://developers.openai.com/api/docs/changelog
13. Anthropic, "Workload Identity Federation" (Claude Platform docs, GA 17 June 2026). https://platform.claude.com/docs/en/manage-claude/workload-identity-federation
14. CyberDesserts, "Exposed Credentials Pipeline" (primary research). https://blog.cyberdesserts.com/exposed-credentials-pipeline/