Platform Monkey ctx

LiteLLM v1.110.0-rc.2 Release Analysis: Credential Injection Banned, Rust Router Repaired

· By
Sources

What This Release Actually Is

LiteLLM v1.110.0-rc.2 (published 2026-11-12 02:53 UTC) is a release candidate that exists for one reason: a security fix that could not wait for the stable train. The rc.2 delta over rc.1 is a single commit-class change — the backport of PR #51254, security(proxy): stop request body fields from conflicting with admin-set deployment fields — and that same fix landed on the stable lines within the same hour as v1.109.1 (02:07 UTC, via PR #51264) and v1.106.9 (01:06 UTC, via PR #51261). If you run LiteLLM in production on any supported line, that coordination, not the RC label, is the story of your week.

But the v1.110.0 line itself is the more consequential thing to evaluate, because it is the first minor developed on a single architecture: PR #50178 (refactor(router): delete the python router core, merged 2026-11-05) removed the Python router core entirely, leaving the Rust core as the only core. The v1.110.0 cycle has consequently been a regression-and-repair arc: the Rust core shipped with a hot path that was slower than the Python core it replaced, a native Rust Redis batch layer that leaked worker memory on every request, and a registry read path that could pair one version's deployment indices with another version's model list. Most of those wounds are now closed — inside this RC. That makes v1.110.0-rc.2 the most interesting kind of release to review: the one where the migration guide is a post-mortem.

The compare from v1.109.0 to v1.110.0-rc.2 spans 401 commits across 300 files (GitHub compare API), so this analysis concentrates on the changes that alter production behavior in ways an operator can observe: what requests get rejected, what requests silently change answer, what pods leak memory, and what bills differently.

2026-11-10 20:24 UTC   v1.103.11 published          (stable patch train, older line)
2026-11-10 20:29 UTC   v1.104.9  published          (stable patch train, older line)
2026-11-10 22:51 UTC   v1.108.5 published            (lts/1.108.x patch)
2026-11-11 00:06 UTC   v1.110.0-rc.1 published      (next minor, rc.1)
2026-11-11 00:56 UTC   v1.109.0 published            (stable minor)
2026-11-11 22:44 UTC   PR #51254 opened              (security: credential-confusion fix)
2026-11-11 23:07 UTC   PR #51263 opened              (backport -> rc/1.110.0)
2026-11-12 01:06 UTC   v1.106.9  published           (security backport, stable/1.106.x)
2026-11-12 01:12 UTC   v1.108.6  published           (lts patch + backports)
2026-11-12 02:07 UTC   v1.109.1  published           (security backport, stable/1.109.x)
2026-11-12 02:53 UTC   v1.110.0-rc.2 published      (security backport, rc/1.110.0)
                       ^ the same fix, three lines, one hour

Everything below is mined from the release bodies, the pull requests themselves, and the source tree at the v1.110.0-rc.2 tag. Where a number is a maintainer benchmark rather than an independent measurement, it says so.

The Headline Fix: Client-Side Credential Injection Is Now Banned (#51254)

The security backport in rc.2 closes a class of vulnerability that every multi-tenant LLM gateway eventually rediscovers: fields a client can put in a request body that the proxy will treat as deployment configuration. Before this fix, a caller could send tenant_id, client_id, client_secret, azure_username, azure_password, or azure_scope in the body of POST /v1/chat/completions, and the proxy would merge those values with the sign-in fields the admin had configured on the deployment. The documented failure mode was an opaque 500 from the Azure sign-in step — the confusing-behavior case. The dangerous case is what the ban-list source comments at the tag describe explicitly: an attacker-controlled endpoint or credential field "either exfiltrates the request payload (incl. messages + admin-set tokens) to the attacker's host, or coerces the proxy into authenticating against the attacker's host with admin secrets."

After the fix, the request is refused up front with a 401 that names the field, unless the value arrives in metadata or an admin explicitly opted in. The exact enforcement message, read from litellm/proxy/auth/auth_utils.py at the tag:

Rejected Request: {param} is not allowed in request body.
Clientside passthrough requires explicit admin opt-in via either
`general_settings.allow_client_side_credentials = true` (proxy-wide) or
`configurable_clientside_auth_params` on the deployment in your proxy config.yaml.
Relevant Issue: https://huntr.com/bounties/4001e1a2-7b7a-4776-a3ae-e6692ec3d997

That huntr.com/bounties/ reference in the shipped error string matters for triage: the ban-list architecture this fix extends came in through a disclosed vulnerability report, not proactive hardening. Treat the pre-fix behavior as exploitable, not merely sloppy.

The Six New Fields, and the Much Longer List They Joined

PR #51254 itself is a tight diff — auth_utils.py +93/−2 plus tests — adding the six Entra credential fields and two value-format rules to a ban list that already existed at scale. The full _BANNED_REQUEST_BODY_PARAMS tuple at the v1.110.0-rc.2 tag is worth reading as an inventory of every way a request body could previously steer the proxy:

Endpoint retargeting          api_base, base_url, aws_bedrock_runtime_endpoint,
                              aws_bedrock_control_plane_endpoint,
                              sagemaker_base_url, s3_endpoint_url, deployment_url,
                              nvcf_function_id, use_ssl, riva_offline

AWS identity selection        aws_sts_endpoint, aws_web_identity_token,
                              aws_role_name, aws_profile_name, aws_session_name,
                              aws_external_id, aws_session_tags
                              (aws_profile_name signs Bedrock/S3 requests as
                              ANY profile present on the proxy host)

Azure / Entra credentials     azure_ad_token (resolved via get_secret, then sent
                              as the bearer token) + the six new fields:
                              tenant_id, client_id, client_secret,
                              azure_username, azure_password, azure_scope

GCP / Vertex                  vertex_credentials, vertex_ai_credentials

Bedrock tenancy + isolation   aws_bedrock_project_id, workspace_id,
                              aws_workspace_id, anthropic_workspace_id,
                              anthropic-workspace-id, bedrock_tags,
                              agentcore_runtime_session_id
                              (every caller-chosen AgentCore session value
                              provisions another microVM)

Storage                       s3_bucket_name, s3_output_bucket_name,
                              allow_legacy_cloud_file_ids
                              (redirect Bedrock file uploads to attacker S3)

Router / transport choice     rust (picks the Rust transport + its callback
                              surface the admin did not choose), model_list,
                              user_config, fireworks_forward_user_id,
                              user_provider_credentials, github_copilot_*,
                              observability callback params (derived from the
                              canonical allowlist: new integrations are
                              banned by default), custom-pricing params

Two value-format rules ship with it. First, credential fields now refuse values written in the admin-config formats — os.environ/... and oidc/... secret references — outside deployment-management routes. The rejection message is direct: "Rejected Request: {param} holds a secret reference, which only a deployment or credential a proxy admin configures can carry." That closes the sharpest version of the injection: a client writing os.environ/LITELLM_MASTER_KEY-shaped references into a credential field and getting the proxy to resolve its own secrets on a request-shaped path.

Second, the ban now reaches the metadata path, not just the request-body root. The source comment is unusually honest about why: "a value under metadata.langfuse_host redirects the same Langfuse client and leaks the same credentials as the root-level langfuse_host, but the original check only walked the request-body root, so the metadata path was an unintentional bypass." Per the backport note in PR #51263, the rc/1.110.0 line previously had no separate metadata ban list at all; the backport adds one that leaves out only the six Entra names — meaning those six are the ones callers can legitimately relocate into metadata.

BEFORE #51254                                          AFTER #51254
----------------                                      ----------------
client                                                 client
  | body: tenant_id, client_secret, azure_ad_token       | body: tenant_id, client_secret
  v                                                      v
[proxy endpoint]                                       [proxy endpoint]
  |                                                      |
  | (no field-level gate)                                +-- is_request_body_safe()
  v                                                          | banned param present?
merge w/ admin-set deployment fields                          |-- yes -> 401, field named
  |                                                          |    (opt-in checked:
  +-- os.environ/ or oidc/ values                            |     allow_client_side_credentials
  |     get_secret()-resolved against                        |     OR configurable_clientside_auth_params
  |     the PROXY's secret store                             |     OR metadata-carried six)
  v                                                          +-- os.environ/ or oidc/ value in
provider call (attacker-retargetable)                           a credential field? -> reject
                                                          v
                                                      admin-configured deployment fields only
                                                          |
                                                          v
                                                      provider call (admin-pinned target)

The Opt-In Matrix

The migration question every operator has to answer: which of your callers legitimately need to pass these fields? The fix defines three escape hatches, and the YAML below shows all of them as they appear in a real config:

general_settings:
  # Proxy-wide: re-enables ALL banned fields for every deployment. Blunt.
  allow_client_side_credentials: false

  # Narrower: value allowlists for the S3-shaped params specifically
  allowed_client_side_s3_buckets:
    - "acme-agent-uploads"
  allowed_client_side_s3_endpoint_urls:
    - "https://s3.us-east-1.amazonaws.com"

model_list:
  - model_name: "azure/eu-gpt"
    litellm_params:
      model: "azure/eu-deployment"
      api_base: "https://eu.example.openai.azure.com"
      # Per-deployment: allow ONLY the named fields from request bodies
      configurable_clientside_auth_params:
        - "tenant_id"
        - "client_id"
        - "client_secret"

Note the asymmetry the S3 allowlists introduce: s3_bucket_name and s3_endpoint_url can be admitted by value (specific buckets, specific endpoints), while credential fields are admitted by name (the whole field or nothing). That is the right shape — an allowlist of secrets by value is not a thing — but it means the per-deployment opt-in is all-or-nothing per field name. Audit which keys actually use it.

What Breaks on Upgrade

For teams running Azure deployments where developers carried their own service-principal fields in request bodies (a pattern that worked because the proxy mixed them in), this is a forced rearchitecture of client config, not a config tweak. That is the honest migration toil: the fix is correct, and the blast radius is every client that leaned on the bug.

The Rust Router Core Grew Up Under Fire

v1.110.0 is the first minor where there is no fallback architecture. PR #50178 deleted the Python router core outright, which retroactively explains most of this cycle's churn: the Rust core's host-side glue (the Python code that mirrors Rust state and answers Python-side questions) was written in a hurry, and the v1.110.0 line has been paying for it. Three of the repairs are in this RC.

Mid-Stream Fallback Stops Inventing Answers (#50260, backport #50798)

LiteLLM's enable_mid_stream_fallback_continuation router setting does something genuinely valuable for agent workloads: when a streaming response dies mid-answer, the router can resume the conversation on a fallback deployment and hand the caller one continuous stream instead of a partial answer plus an error. The bug PR #50260 fixed is what that continuation did to request-level fallbacks: the mid-stream continuation check read only the router's fallback list, never the request's.

The failure reads like a correctness horror story, and the PR's user flow is worth internalizing: a caller sends "stream": true with a request-level fallback that carries its own messages — a restart the conversation on failure policy. The primary stream gets cut after "Alpha beta gamma". The continuation logic splices the primary's partial prefix onto the fallback's restart messages, and the caller reads "Alpha beta gamma omega." with no error at all. A silent wrong answer from a router is the worst failure class a gateway has; after the fix, that same request surfaces the primary's error after the partial text, because the check now reads the request's fallbacks first and only falls back to the router's list when the request doesn't set one — the same precedence the normal fallback walk already used.

stream cut mid-answer ("Alpha beta gamma")
        |
        v
+-----------------------------------+     BEFORE #50260: check reads
| mid-stream continuation check     |     ONLY router fallbacks
+-----------------------------------+        |
        |                                    v
        |  request has its own fallbacks?   router fallback: backup
        |  (they restart the conversation)  -> splice primary prefix
        |         |                          onto fallback messages
        v         v                          -> caller reads
   AFTER #50260: request fallbacks          "Alpha beta gamma omega."
   take precedence. A request fallback
   that brings its own `messages`
   cannot carry the prefill:
   -> surface the primary's error
      after the partial text
   (no fallback messages: continue
   on router fallbacks, as before)

Two architecture facts matter more than the bug. First, the fix is a one-argument change in the Rust core's continuation check — this logic now lives in litellm-rust/, and the regression was latent while the Python core served by default: the PR notes the test only started failing when #50178 made the Rust core the only core. Second, the caveats section is honest about a remaining gap: the Responses API continuation "never checks fallback targets in either core — it reads no fallback list at all." If you are building on /v1/responses streaming with mid-stream fallback expectations, this release does not yet have that story.

The Registry Race: Positions, Mirrors, and One New Lock (#50933, backport #50993)

This is the deepest fix in the release. The Rust core's registry API returns positions — indices into the deployment list — and the Python host then reads HostedModelList, its Python mirror of that list, at those positions. Before PR #50933, writes applied the changeset to the Rust core first and refreshed the mirror after, with no lock between the two. A reader on another thread inside that window paired one registry version's positions with the other version's list. The PR documents both observable failures:

      WRITER thread                              READER thread (before #50933)
      ------------                              --------------------------
[1] apply changeset to Rust core
        |
        |   <<<<< window: Rust state updated, Python mirror stale >>>>>
        |                                          [2] registry lookup ->
        |                                              Rust returns positions
        |                                              for the NEW list
        |                                          [3] read HostedModelList
        |                                              at those positions
        |                                              -> it is the OLD list
        v
[4] refresh Python mirror

  outcome A: append happened -> position > len(mirror) -> IndexError
            (raised in driver.finish AFTER upstream success)
  outcome B: removal happened -> positions shifted ->
            model-1's lookup returns model-0's deployments (key-0 included)
            -> wrong rows reach model-group info, usage headers, get_model_list

  AFTER #50933: the registry's existing RLock (_deployment_index_lock) is
  held across each router-core write AND the mirror read-back that follows;
  the 21 registry readers that pair lookups with mirror reads hold the same
  lock. Build steps (validate / provider / secrets) answer OUTSIDE the lock,
  so a secret-manager call can never block readers.

The fix reuses the registry's existing _deployment_index_lock (an RLock), passing it into HostedModelList so refresh, build phases, append, replace, and the single-delete path all hold it across the Rust write plus the Python read-back. The cost is characterized precisely: lookups already hold the GIL on the Python side, so the change adds "about one uncontended RLock acquire per lookup." The stress numbers are the kind operators should ask vendors for more often: an 8-thread completion load against an upsert loop with sys.setswitchinterval(1e-6) produced 6 IndexErrores in 4,000 calls on main and zero in three runs of 4,000 on the fix branch.

The caveats deserve equal billing, because they define the remaining exposure: a set_model_list replace-build still shows the new list in the mirror while the Rust core serves the old one until the build ends (closing that would mean holding the lock across secret resolution, so it is deferred), and an upsert is still two writes — remove, then add — so a read between them sees the group without that deployment, same as the old Python router. The long-term plan, per the PR, is for lookups to return deployment IDs instead of positions, which removes the paired-state problem as a class. Until then: treat deployment churn (frequent /model/new, /model/update traffic) as the trigger condition for this race, and be glad it is locked.

Host-Path Performance: The Rust Core Was Slower Than What It Replaced (#50641)

The most instructive numbers in this release are in PR #50641, because they quantify the cost of the single-core transition before anyone repaired it. Two host hot paths ran on every request and were written naively against the Rust bridge: the router's callback_filters step read every candidate deployment back into Python — 1,000 reads per pick at 1,000 deployments — and membership was tested with list scans, O(n²), about 7 ms per pick at 1,000 deployments. The anthropic capability probes (12 probes per request) ran fresh on every call.

The maintainer-reported benchmark table (SDK, in-process, mock upstream, fat LTO, 2 interleaved rounds; baseline 85bc5808 is the last commit with both cores) is the honest record:

Case (CPU per call)Python core (last dual-core commit)Main before PRThis PR
select, simple-shuffle, 100 deployments460 µs829 µs354 µs
select, simple-shuffle, 1,0003.8 ms17.5 ms5.9 ms
acompletion, simple-shuffle, 1,00011.6 ms30.9 ms16.9 ms
select, latency-based, 1,00036.2 ms65.5 ms52.4 ms
/v1/messages direct (no router)4.0 ms5.1 ms3.7 ms
/v1/messages, router, 1,000 deployments29.5 ms28.4 ms14.6 ms

Read the middle column against the first: before these fixes, the Rust-only router spent 17.5 ms of CPU per select at 1,000 deployments where the Python core it replaced spent 3.8 ms — roughly 4.6× slower on the hot path. The two host-path fixes (return candidate IDs without per-deployment reads; frozenset membership; lru_cache(256) on capability probes keyed to the model-map state) brought the callback_filters step from 12.2 ms to 0.12 ms at 1,000 deployments — about a 100× drop — and the probe path from 1.7 ms to 0.36 ms on direct /v1/messages. The PR's own caveat keeps it honest: even after the fix, the Rust core is still ~1.5× the old Python core's select cost at 1,000 deployments, with the remainder in per-select usage reads, Python-parsed cooldown keys, and the crate's 2.3 ms final pick. The rewrite is not a performance story yet; it is a correctness-and-velocity story with a performance debt line item.

One operational nuance from the capability cache: it keys on the model-map state — the map object, the mutation generation, its size, and the identity of every entry the probes read — so register_model and litellm.model_cost[key] = ... assignments are still seen on the next request. An in-place edit of an existing entry dict (litellm.model_cost[key]["supports_reasoning"] = True) without register_model is not. If you mutate pricing dicts in place from hooks, this cache will serve you stale capability answers.

total_timeout: One Wall-Clock Budget for the Whole Request (#50496)

The feature release of this cycle addresses the loudest agent-infrastructure complaint about retrying routers. Issue #49751 (filed by an outside user, 2026-11-03) documents it precisely: x-litellm-timeout bounds only one upstream attempt, so Router.async_function_with_retries keeps retrying and sleeping Retry-After intervals with no check on how much of the caller's budget remains. Their repro: fast 503s with Retry-After: 6, num_retries: 3, client timeout 10 seconds — the proxy spends ~20 seconds retrying a caller who already hung up. The consequence is worse than wasted latency: "the client has already timed out and sent its own retry, so upstream load doubles exactly when the provider is struggling." A retry-amplification feedback loop during provider brownouts, driven by the gateway. (The issue also notes the OpenAI SDK sends its timeout as x-stainless-timeout on every request and the proxy never read it — meaning default SDK clients got no deadline at all.)

PR #50496 answers with a new opt-in wall-clock budget. The module docstring in litellm/litellm_core_utils/total_deadline.py states the semantics in three sentences worth quoting whole: "One wall-clock budget for a whole non-streaming proxy request (total_timeout): the proxy opens it, and the router reads it to refuse a retry backoff that cannot finish in time. A request that turns out to stream or run in the background is not bounded: a body total_timeout refuses it, a header one (often set globally on a client) lets it go unbounded."

client                                     proxy internals
------                                     --------------
POST /v1/chat/completions
  body:   "total_timeout": 10       <-- strict: refuse if request streams
  header: x-litellm-total-timeout: 10    <-- lenient: ignore on streams
       |
       v
[preparation]-----+
[routing]---------+--> ONE wall-clock budget (10 s)
[attempt 1]-------+      - covers EVERYTHING:
  503 retry-after: 6        preparation, retries, backoffs,
[backoff sleep 6s]-+        moderation, post-call hooks
[attempt 2]-------+      - a backoff that CANNOT fit inside the
  503 retry-after: 6        remaining budget is not started:
[refuse: next backoff       the last upstream error returns
 needs 6s, only 3.7s left]  immediately
       |
       v
HTTP 408 "the request did not finish within its
total_timeout of 2.0s" + upstream call cancelled
       |
       v
billing: like a client disconnect, at estimated prompt cost

streaming / background requests: NOT bounded
  - body total_timeout   -> 400 (refuses the request)
  - header total_timeout  -> silently ignored (global client
    headers stay safe)

The before/after timing from the PR's verification run tells the operational story: 503s with retry-after: 6, num_retries: 3, total_timeout: 10 — before, the caller waits 25.5 seconds for the final 503; after, one retry fits, the next 6-second backoff does not, and the 503 returns at 6.4 seconds with no further attempts. Against a hard-hung upstream with total_timeout: 2, before you wait for your own client timeout; after you get the 408 at 2.0 seconds with the upstream call closed. It builds on PR #50486, which makes a cancelled attempt release the resources it holds.

Agent-platform operators should note what this is not: a streaming budget. Long agentic completions are streaming completions, and the header form is explicitly allowed to pass through unbounded because clients set global headers. The feature protects the non-streaming blast radius of retry storms; it does not bound your agents' 90-second streams. Budget those client-side, as before.

Two Regressions This Cycle Shipped, and This Cycle Repaired

Both of the following are v1.110.0-dev regressions — introduced in the same native-Rust push that deleted the Python core, then fixed inside the v1.110.0 line before its stable cut. That is the right way round, but it means anyone who rode the dev builds lived through both.

The Redis Batch Scope Leak: 40 KiB Per Request, Forever (#50751, backport #50949)

PR #50751 (by an outside contributor) documents a leak with an unusually clean causal chain: with Redis enabled for routing, coordination, and the spend transaction buffer, every proxy worker grew about 40 KiB of RSS per request served, permanently. Every request's native Redis batch scope stayed in memory for good; requests slowed as the leaked heap grew; Kubernetes eventually evicted the pods. The PR's soak evidence is specific: 1,000 warmup requests, then five rounds of 1,000 requests at 20 concurrency, reading VmRSS from /proc/<pid>/status after every round. The root cause is named without sentiment: the native Redis batch (added in PR #50169 as the Rust Redis foundation, served on every request since PR #50170, regression since v1.110.0-dev.1) was built inside the request's own context, so a finished request's scope was never collectable. The fix builds the batch outside the request context and adds a regression test asserting that a closed request scope is freed.

Two details deserve an operator's respect. First, the PR includes a live A/B that ruled out the suspected cause: the SpendLogs ledger named in the tracking issue leaked at the same rate with the ledger on and off. Second, the leak sat on a path every request exercises — auth, budget checks, routing, spend tracking — not on an optional feature. If you run Redis-backed LiteLLM on 1.110.0-dev builds, your OOMKills were this, and the fix is only in this RC and forward.

Accounting Flushes Scanned the Entire Delivery Table (#50565)

The second dev-line regression is a quieter database killer. The default spend ledger's flush cycle ran a delivery-metrics query that scanned every record in SpendLogsDelivery — retained receipts included — on every flush. Query cost grew with retained data even at zero backlog, and once the delivery table crossed the storage-alert threshold, a High alert landed in Slack on every flush, roughly every 10 seconds, forever. PR #50565 reshapes the path: the backlog query reads only pending and parked rows through the existing index, the batch count and table size run once per interval per worker process, and storage/delay alerts repeat at most once per interval. /metrics still reports pending, parked, oldest-pending-age, retained batches, and table bytes.

The interval is env-driven and hot-read on each flush, so config.yaml environment_variables apply without a restart:

environment_variables:
  # How often storage/delivery health is checked per worker.
  # Default: 900 (15 minutes), read on every flush.
  LITELLM_ACCOUNTING_DELIVERY_CHECK_INTERVAL_SECONDS: "900"

  # Storage alert thresholds (defaults):
  #   batch count >= 1,000,000 batches, OR
  #   table bytes  >= 1,073,741,824 (1 GiB)
  LITELLM_ACCOUNTING_BATCH_DELIVERY_ALERT_COUNT: "1000000"
  LITELLM_ACCOUNTING_DELIVERY_ALERT_BYTES: "1073741824"

If your DB query-insights dashboards show a repeating accounting query whose cost tracks your retention rather than your traffic, you were on the wrong side of this one.

Everything Else That Changes Behavior

Hosted-Tool tool_choice on the Chat Route (#50578)

litellm.completion previously returned a 400 on tool_choice: {"type": "web_search"} before the request ever left the building — while the identical body on /v1/responses worked, because the Responses bridge already passed such dicts through. PR #50578 makes validate_chat_completion_tool_choice pass a dict whose type is a known OpenAI hosted tool through unchanged, so the Chat route can force hosted tools (web search being the case in the tests) and return the live-search answer as a normal chat.completion. The provider guard matters as much as the feature: the same body pointed at an Anthropic deployment now stops with "tool_choice type 'web_search' forces an OpenAI hosted tool and is only supported for azure, litellm_proxy, openai" instead of Anthropic quietly answering without the tool. Agent frameworks that had to maintain two code paths (the PR cites langchain-litellm falling back to tool_choice="required") get one.

Breaking: Wildcard /model/info Rows Show What Calls Bill (#49128)

Marked breaking by its own author, and the ! in the title is earned. Before, a wildcard-expanded /model/info row renamed by an alias or a routing group showed its own listed price while calls actually billed a different model. The PR's repro says it exactly: alias my-openai/gpt-5.5 to my-openai/gpt-5.4-mini, and the row showed gpt-5.5 pricing while the response header billed gpt-5.4-mini's rate. Worse, replaying that row into PATCH /model/{id}/update pinned the row's displayed metadata (key, max_tokens) onto the whole wildcard. After the change, each row follows the caller's aliases and route candidates and shows the model its calls actually bill, with that model's limits; a row goes blank only when its target deployments bill more than one model or declare their own rate — and the PR is careful to note a blank is not a zero price, since a zero reads as free. The save routes now compare against every cost-map entry the row can display, so a replay stores only what the admin actually changed. If you have automation that parses /model/info prices — chargeback scripts, internal model catalogs — re-verify it against this release before you trust its output.

Bedrock: Per-Caller STS Session Policies Instead of One Bedrock-Only Policy (#50592)

A sharp fix for web-identity (federated) auth on AWS: previously every AWS call carried a single Bedrock-only session policy, so most non-Bedrock AWS routes, the AWS loggers, Bedrock realtime, async invoke, Mantle token counting, and the agent-runtime pass-through all received 403s even when the role allowed them — while static keys and IRSA sent no session policy at all and worked. PR #50592 gives each AWS caller class its own session policy listing only the actions its routes make, and adds the policy to the credential cache key (two callers with the same role and token previously shared one cached session, so a logger could reuse a Bedrock-only session and still 403). One role and token can now open up to 19 STS sessions per worker, one per caller policy, each refreshed on expiry. Three scoping decisions in the PR are worth reading as security review: knowledge-base ingestion can now only create, edit, and pass roles named litellm-bedrock-kb-* (previously the session could rewrite the proxy's own role); realtime gets bedrock:InvokeModel + bedrock:ApplyGuardrail with no aws:SecureTransport condition, because a live probe showed Bedrock evaluates a Nova Sonic realtime stream as bedrock:InvokeModel and reports aws:SecureTransport as false (denying the stream outright); and every policy granting bedrock:InvokeModel also grants bedrock:ApplyGuardrail, because an enforced guardrail denies invokes whose caller cannot apply it.

Responses-ID Ownership Becomes a 403 (#50600)

The ownership/scope alignment PR carries one explicitly intentional product change: stored response IDs with no recorded owner are now rejected with 403 Forbidden. This response id has no associated owner, unless general_settings.disable_responses_id_security is set. The hook's own docstring states the goal plainly: "Security hook to prevent user B from seeing response from user A," batch-writing response IDs through DBSpendUpdateWriter rather than a synchronous write per request. Cross-instance flows (create a stored response on one proxy instance, continue it on another) are the QA path in the PR. If your tooling shares Responses API IDs across users or service accounts with no ownership semantics, this is a break you will find in production, not in the changelog.

Smaller Sharp Edges

Migration Toil Assessment

Pick your line by what you need, not by the RC label. The security fix (#51254) is already on stable: v1.109.1 for the 1.109.x line and v1.106.9 for the 1.106.x line shipped it in the same hour as rc.2. The rc line additionally carries everything the stable lines do not have yet: the mid-stream fallback continuation fix, the registry lock, the host-path performance work, total_timeout, the Redis batch scope leak fix, the accounting flush repair, and the full v1.110.0 feature set.

                          security   mid-stream  registry   total_    redis     accounting
                          #51254     fallback    lock       timeout   leak fix  flush fix
v1.106.9  (stable)         YES        --          --        --        --        --
v1.108.6  (lts)            --         --          --        --        --        --
v1.109.1  (stable)         YES        --          --        --        --        --
v1.110.0-rc.2 (rc)         YES        YES         YES       YES       YES       YES

-- = not present on that line per its release notes.
v1.108.6 (lts) is the backport lane for tool_choice fixes (#49043, #50467),
idempotent budget resets (#50298), and a breaking zscaler_ai_guard backport
(#50139) - read its release body before taking the lts bump.

The breaking-change checklist, in order of blast radius:

  1. Credential passthrough callers — the six Entra fields in request bodies now 401. Inventory your SDK wrappers and internal clients for those names before you deploy anything from this train; the fix is on stable lines too, so this is not avoidable by staying behind.
  2. /model/info consumers — wildcard row prices change (some to blank). Chargeback and catalog tooling must be re-verified.
  3. Responses-API ID flows — unowned IDs 403; same-user cross-instance continuation is the supported shape.
  4. Hosted-tool tool_choice on non-OpenAI providers — a clear 400 replaces a silent tool-less answer. Tests that asserted the old behavior will fail, correctly.

Supply chain: all LiteLLM Docker images are cosign-signed, and this release's body recommends verifying against the pinned signing-key commit rather than the tag (tags are mutable pointers protected only by branch rules):

# Strongest: pin the signing key to the immutable commit hash
cosign verify \
  --key https://raw.githubusercontent.com/BerriAI/litellm/e82a3c646908ba0a6ee0beddc4310d7a6cac4065/cosign.pub \
  ghcr.io/berriai/litellm:v1.110.0-rc.2

# Expected output:
# The following checks were performed on each of these signatures:
#   - The cosign claims were validated
#   - The signatures were verified against the specified public key

Who Should Skip This Release

Verdict

LiteLLM v1.110.0-rc.2 is two releases in one artifact: an emergency security patch that is already on your stable line, and the release candidate of the first single-core (Rust-only) minor — which means it is also the repair bill for that transition. The security fix is correctly scoped, long overdue, and honest about its own history (the huntr bounty reference ships in the error message); its blast radius is every caller that leaned on request-body credential passthrough, and that migration is the real cost of this train on every line that took it. The routing repairs fix two of the worst failure classes a gateway can have — silent wrong answers from mid-stream fallback splicing, and cross-model deployment metadata from the registry race — and the perf PR publishes the numbers that show how much debt the core swap still carries. Take the stable security patch today regardless of what else you do; evaluate the RC itself if deployment churn, Redis-backed worker memory, or retry amplification under Retry-After is actively hurting you — those fixes exist nowhere else yet. And when v1.110.0 stable lands, re-read its notes before assuming the remaining Rust-core caveats (the set_model_list window, the two-write upsert, the Responses-API fallback gap, the still-unbounded streaming path) have moved: as of rc.2, they have not.