vLLM 0.34.0 went out at 23:22 UTC on November 16 — 1,061 commits from 412 contributors, 120 of them first-timers, per the release notes. It is the first stable release of the project with published notes in almost a month, and that is not a rhetorical flourish. The 0.33.0 wheels hit PyPI at 10:13 UTC on November 2, three minutes after the v0.33.0rc3 release notes went up — but a stable v0.33.0 release body was never published (the GitHub API returns 404 for it to this day). If you upgraded straight off PyPI in early November, you did it blind. For everyone pinned to v0.32.0, the last stable with notes, this release is a 2,002-commit jump (per the compare API) and the changelog below is the first consolidated documentation of everything in between.

Three architectural changes do the heavy lifting: the extensible KV cache becomes the default and retires the estimate-then-pray memory sizing that made gpu_memory_utilization the most cargo-culted knob in inference operations (PR #56492, njhill, merged November 2); Model Runner V1 is deleted outright — 435 files, +1,693/−40,769 (PR #63373, WoosukKwon, merged November 7); and the default block size moves from 16 to 64 (PR #63231, NickLucche, merged November 5), which quietly changes your prefix-cache behavior. Around them: suffix decoding finally works without Arctic Inference, a CRIU-based checkpoint/restore API lands in the HTTP frontend, and a pile of Blackwell- and MI355X-specific kernel wins post numbers between 1.24× and 1.95×. If you run vLLM behind a gateway as one of several model backends (see our LLM router guide for that topology), the breaking-changes table below is the part to read before you bump the tag.

What Shipped, By Blast Radius

The changes that matter to platform engineers, sorted by who wakes up when they land:

ChangeTypePRWho it hits
Extensible KV cache on by default; gpu_memory_utilization defaults to 1.0Architecture / default flip#56492Every CUDA and ROCm deployment; capacity plans and latency SLOs
Model Runner V1 removedRemoval#63373Anyone with V1 logits processors, custom spec-decode proposer forks, env-pinned runners
Default --block-size 64 (was 16)Default flip#63231Prefix-cache-sensitive workloads (agentic loops, RAG re-prefixing)
ROCm 7.2 wheels and images droppedRemoval#61939Every AMD fleet not already on ROCm 10.0
huggingface_hub 2.0, tokenizers 0.23.3, timm 1.0.30 requiredDependency floor#64779Pinned ML environments, vendored wheels, air-gapped mirrors
Marlin W4A8 and QuTLASS/fp_quant removedRemoval#65542, #65230Legacy W4A8 quantized checkpoints in production
Checkpoint/restore over HTTP (CRIU + cuda-checkpoint)New capability#62570Fleets doing engine migration, drain-and-snapshot, GPU consolidation
Fault-tolerant DP+EP scale-down without restartNew capability#46370Large expert-parallel deployments on flaky nodes
Suffix decoding on MRV2, GPU speculator, no ArcticFeature#60339Agentic and code workloads with high output repetition
Best-effort NUMA binding on by defaultDefault flip#61258Multi-socket hosts with NUMA-unaware orchestrators

Ten New Models, and One Landing With the Full Stack

The model launch that matters architecturally is Mistral Large 4: 1T total parameters, 52B active, FP8 and NVFP4 checkpoints, and it arrives on MRV2 with essentially every large-scale-serving primitive vLLM has shipped this year already wired — sparse MLA plus the DSA indexer, cross-layer KV sharing, DSpark speculative decoding, DCP/PCP context parallelism, NIXL prefill/decode disaggregation, and HiSparse host-tier support (#64534) — on NVIDIA Blackwell/Hopper and AMD MI350. That day-one integration is the tell: new frontier models are now landing directly on the MRV2-only world the runner deletion creates, and V1-era forks will not see them at all.

ModelNotesPR
Mistral Large 41T total / 52B active; FP8 and NVFP4; sparse MLA + DSA indexer, cross-layer KV sharing, DSpark, DCP/PCP, NIXL P/D, HiSparse; Blackwell/Hopper + MI350#64534
MiniMax-M3.1-FlashBlackwell NVFP4 and ROCm kernels; DSpark draft; P/D across different prefill and decode TP (#64475)#64111
NVIDIA Nemotron-3.5-Transcribe30B-A4B speech model: audio chat and /v1/audio/transcriptions, with MTP and DFlash2 (#64944)#64282
Ricursive Ripple-65B PreviewBF16 and NVFP4; ripple reasoning and tool parsers#65745
Zyphra ZAYA1ZAYA1-8B, ZAYA1-74B-preview#65302
Baidu Unlimited-OCR-2DASH sparse attention#64016
Liquid AI LFM3, LFM3-VL, LFM3-AudioLFM3 family, three modalities#65826, #65827, #65828
NVIDIA Canary-1B-v2ASR and X-En translation#64215
Ming3-ASRSpeech recognition#63585
NVIDIA Parakeet TDTShipped through the vllm-parakeet-plugin#65005

The Extensible KV Cache: Your Most-Misconfigured Knob Is Now Obsolete

Since the first versions of vLLM, the KV cache has been sized before CUDA graph capture, because captured graphs bake in the cache's addresses. That forced an awkward sequence: profile activation memory, guess what CUDA graphs and warmup would need, subtract an 8% safety margin, and allocate the cache from what remained. The 8% was not tuning — it was insurance against the guess being wrong. On a 192-GiB H200, the old gpu_memory_utilization=0.92 default idled roughly 15 GiB to absorb estimation error. When the guess was too optimistic, you got an OOM after startup, in the middle of traffic.

PR #56492 (56 files, +4,698/−143, productizing Simen Zhaohan Zhu's demonstration in #47363) inverts the sequence. The engine now reserves the cache's virtual address range up front via the driver's virtual-memory API, runs CUDA graph capture and warmup, measures what is actually free, and only then maps physical memory to the final cache size. The verified CacheConfig source at the v0.34.0 tag says it plainly: gpu_memory_utilization is "1.0 with the extensible KV cache (sized from measured memory) and 0.92 otherwise."

graph TD
  subgraph OLD["v0.33 and earlier - estimate first, hope later"]
    A1["Engine start"] --> B1["Profile run measures activation peak"]
    B1 --> C1["Guess CUDA graph memory plus warmup transients"]
    C1 --> D1["Size cache to 0.92 of device minus profiled peak"]
    D1 --> E1["CUDA graph capture and warmup"]
    E1 --> F1["Runtime: estimate too optimistic means OOM under load, too pessimistic means about 15 GiB idle"]
  end
  subgraph NEW["v0.34.0 - reserve, warm up, measure, map"]
    A2["Engine start"] --> B2["Reserve VA range via driver VM API, no physical pages yet"]
    B2 --> C2["CUDA graph capture"]
    C2 --> D2["Warmup pass"]
    D2 --> E2["Measure memory actually free"]
    E2 --> F2["Map physical pages to the final KV cache size"]
    F2 --> G2["gpu_memory_utilization defaults to 1.0"]
  end

The release's own numbers: MiniMax-M3 on an H200 grows from 433k to 672k cacheable tokens — about 55% more effective capacity from identical hardware, which converts directly into throughput and lower TTFT at high concurrency. The project paid for it with a hardening tail that is worth reading as a risk register: allocator-fragmentation headroom validated on Qwen3.5-397B-A17B-FP8 at TP8+EP on 8×H200 (#67296), startup OOM fixes (#67485, #66523, #66731, #66197), KV connectors that failed to start (#65225), up to 84% undersized caches on DGX Spark (#65413), and --max-model-len auto sizing (#65463).

The trade nobody mentions until the dashboard moves

Capacity is bought with per-token decode latency in some configurations: the release notes state TPOT can rise 6–18% on some H200 models at concurrency 128. That is the signature of a bigger cache winning throughput aggregate while each individual stream waits slightly longer. For fleets on strict per-token latency SLOs — voice, interactive agents — that is a default-on regression, and the opt-outs are explicit:

# Keep the old 0.92 sizing behavior (withholds ~8% as slack):
vllm serve Qwen/Qwen3.5-397B-A17B-FP8 --gpu-memory-utilization 0.9

# Or turn the feature off entirely and pin the old estimate path:
vllm serve Qwen/Qwen3.5-397B-A17B-FP8 --no-enable-extensible-kv-cache

# Pin an exact byte budget instead of a fraction (overrides
# gpu_memory_utilization entirely per CacheConfig):
vllm serve Qwen/Qwen3.5-397B-A17B-FP8 --kv-cache-memory-bytes 120GiB

Three operational consequences to plan for. First, your old tuning advice is now an anti-pattern: an explicit --gpu-memory-utilization in the 0.8–1.0 range that withholds 1 GiB or more now logs a startup warning (#66704) — configs that were "best practice" for two years will start complaining, and that warning is your migration to-do list. Second, the 8% slack is now cache: anything co-resident on the GPU that used to live in that margin — DCGM exporters with GPU-resident buffers, MPS clients, profiling agents that allocate after engine warmup — now competes with the KV cache for what the engine measured as free. The OOM surface moves from startup to runtime, where it is less visible and more expensive. Third, capacity plans must be re-derived per model: "KV cache tokens" on your dashboards will jump after the upgrade, and any bin-packing or quota logic keyed to the old numbers is now wrong. Re-baseline before you re-plan.

Model Runner V1 Is Deleted — 40,769 Lines Gone

PR #63373 is the release's act of hygiene, and it is surgical: 435 files changed, +1,693/−40,769. The V1 GPU runner (gpu_model_runner.py alone was 7.2k lines), the V1 input batch, ubatch wrapper, the CPU model runner, the KV/EC connector runner mixins, the V1 BlockTable, the V1 sampler, and the entire V1 speculative-decode proposer zoo (eagle, dflash, draft_model, ngram, suffix_decoding, custom_class_proposer) with their Triton kernels are gone. Features that only worked on MRV1 were dropped rather than ported — that is the sentence to re-read. The removed-with-it list: the custom_class speculative method, use_heterogeneous_vocab, disable_padded_drafter_batch, parallel_drafting outside DFlash/DSpark, pipeline parallelism with external_launcher, NGramPerReqLogitsProcessor, and x86 CPU deployments without Triton.

The migration surface is blunt. VLLM_USE_V2_MODEL_RUNNER no longer exists; setting it to anything but 1 now fails at startup instead of silently selecting the old runner. If you carried private patches against V1 internals — a common survival pattern for teams that spent 2025 working around V1 scheduler bugs — those patches are dead and the V1 logits-processor API moved to vllm.v1.worker.gpu.sample.logits_processor. Audit your Helm values and deployment env for the flag before upgrading, because the failure mode is a crash loop, not a warning:

# Everything that pins the old runner will hard-fail at startup on 0.34.0:
grep -rn "VLLM_USE_V2_MODEL_RUNNER" charts/ overlays/ .github/workflows/

# V1-era logits processor imports in your forks and plugins:
grep -rn "LogitsProcessor" plugins/ patches/ | grep -v "v1.worker.gpu.sample"

# Deprecated-but-still-alive flags that now only log warnings:
#   parallel_drafting, capture_torch_profiler, enable_layerwise_nvtx_tracing
# Removed outright with V1: custom_class speculative method,
#   use_heterogeneous_vocab, disable_padded_drafter_batch,
#   PP with external_launcher, NGramPerReqLogitsProcessor

What you get in exchange: Dual Batch Overlap now runs on MRV2 by default, including with speculative decoding and context parallelism (#61464) — a combination V1 never supported — plus separate prefill (token) and decode (request) CUDA graph capture sizes (#63406), staged sampler writes that skip idle writes for about 10–11% faster small-batch decode (#62748), and a sampler warmup that stops memory growth under load (#62560). Non-x86 CPU backends without Triton (Arm, IBM Z, PowerPC, RISC-V) got their MRV2 path in #61995, so the runner consolidation does not strand them.

Block Size 64 and the Prefix-Cache Angle Nobody Mentions

The smallest diff in this release — PR #63231 is six files, +26/−4 — is the one most likely to change your steady-state metrics. CacheConfig.DEFAULT_BLOCK_SIZE is now 64 (verified at the tag; it was 16 since the paged-attention era). Kernel-wise this is the right call: fewer blocks means less per-block metadata, smaller block tables, and better gather efficiency on attention kernels tuned for Blackwell's larger tile shapes. The same PR teaches RocmAttentionBackend to keep preferring 16-token blocks so ROCm stays on its native C++ paged-attention kernel instead of silently falling to the slower Triton path, and an explicit --block-size still wins over the default.

But block boundaries are not just a memory-layout detail — they are also where prefix-cache hits land. The CacheConfig docstring at the tag is explicit: cache_hit_alignment_tokens — "token granularity at which prefix-cache hits land: hash_block_size with partial hash hits, otherwise the scheduler block size." If your deployment path lands hits at scheduler-block granularity, that alignment just moved from 16 tokens to 64. For long shared system prompts, irrelevant. For agentic loops and RAG pipelines that share a 300-token prefix and then diverge, a 4× coarser hit boundary can shave real hit-rate and re-prefill cost — exactly the workloads we describe in our RAG architecture guide and agent fleet patterns piece. The upgrade procedure writes itself: record prefix-cache hit rate and re-prefill volume before the bump, and if they regress, pin --block-size 16 and file the regression — the default is a throughput bet, not a law.

Speculative Decoding Grows Up: Suffix Decoding Without Arctic

Since it landed, suffix decoding — the arXiv 2411.04975 method that drafts from suffix matches against a request's own tokens and a corpus of previous responses, frequency-weighted — required Arctic Inference, ran the speculator on CPU, forced Model Runner V1, and disabled async scheduling. On an engine whose future is MRV2-only, that made it a dead feature walking. PR #60339 (20 files, +1,465/−71) rebuilds it on the ngram_gpu foundation: GPU-resident speculator, no host sync on the propose path, async scheduling, no external dependency. Acceptance is on par with the Arctic implementation — 1.937 tokens per step versus Arctic's 1.923 (k=8) and 1.948 (k=24) on a 21k-request replay.

# Agentic / code-editing workloads with high output repetition:
vllm serve Qwen/Qwen3-8B --speculative-config \
  '{"method": "suffix", "num_speculative_tokens": 8}'

# High concurrency: without trimming, verifying every draft slot can make
# suffix decoding SLOWER than no speculation at all. Adaptive verification
# verifies only the slots worth their cost. Requires FULL CUDA graphs and
# an attention backend that replays variable-length decode batches
# (TRITON_ATTN, or FlashAttention 3):
vllm serve Qwen/Qwen3-8B --speculative-config \
  '{"method": "suffix", "num_speculative_tokens": 8, "enable_adaptive_verification": true}'

The defaults, verified in vllm/config/speculative.py at the tag: suffix_decoding_max_tree_depth 24, suffix_decoding_corpus_tokens 2^20 (one million tokens of finished responses, oldest-first), suffix_decoding_max_cached_requests 10,000. The deprecated keys suffix_decoding_max_spec_factor and suffix_decoding_min_token_prob are accepted but do nothing — strip them from configs so their no-op-ness does not mislead the next operator.

The shared corpus is a multi-tenant consideration

The docs ship a warning that deserves amplification: the response corpus is shared across all requests, regardless of cache_salt or LoRA adapter, and any request can draft from it. Drafts never change the output distribution — speculative decoding is lossless by construction — but acceptance rates are observable: a tenant's decode speedup leaks information about what other tenants' outputs look like. For multi-tenant inference platforms with isolation obligations, set suffix_decoding_corpus_tokens to 0 and keep drafting confined to each request's own prompt and output. Two more footguns from the same docs: seeded sampling is not reproducible under suffix decoding (drafts depend on other requests), and the plain cache_salt mechanism you use to partition prefix-cache reuse does not partition the speculator corpus.

The wider speculative-decode story is a platform-ization push: custom speculators get declarative configs plus a plugin API — register_speculative_method in vllm.v1.worker.gpu.spec_decode.speculator_config — so vendors can ship draft methods as vLLM plugins instead of forks (#63205); draft-model speculators get CUDA graphs (#60989), which removes the eager-mode penalty from draft forwards; the draft KV cache now matches the target's dtype (#65198), so an FP8 target gets an FP8 draft cache instead of paying BF16 overhead twice; and adaptive verification works with XPress (#65175) and runs its acceptance budgets on a Triton kernel (#63421).

The Performance Claims, With Their Actual Numbers

The headline table in the release notes is vendor-verified but worth restating with the detail that matters to capacity planning: which gains are default and which are opt-in, and which hardware they actually require. The biggest wins are vendor-tuned paths for the newest silicon — GB200, GB300, MI355X — and most are opt-in. If your fleet is Hopper, the default-on wins for you are the extensible KV cache itself, the sparse-MLA prefill sharding (6.0–7.6% E2E on GLM-5.3 FP8 MoE via sorted-A TMA, #64760), and the block-size change.

ImprovementHardware / verified baselineGainDefault?PR
Step5 NVFP4 MoE on TRTLLM-Gen (SwiGLU enabled)4×GB200, TP4; decode 128/128: batch 1 went 62.71→122.37 out tok/s, batch 8: 392.62→768.39; GSM8K 91.96%→93.18%up to 1.95× decodeDefault for Step5 NVFP4#65152
GLM-5.3-Flash DeepGEMM FP8 MegaMoE4×B200, AgentX replay at concurrency 96, vs DEP4 generic MoE path1.6× output throughput; prefill 1.8× vs DEP4Opt-in via --kernel-config#60895
Token-sharded sparse-MLA prefill (ShardQ)GLM-5.3 NVFP4 TP4 GB200; 16k chunk at 128k ctx: 926→544 msTTFT 1.24×/1.29×/1.40× at 32k/64k/128k promptsDefault at ≥1024 prefill tokens#63956
Muse Spark Open specialized decodeGB300, DSpark verification with 7 draft tokens1.5× single-stream decodeOpt-in#64077
KV-balanced MLA decode for Kimi-K3GB300, agentic workloads1.34–1.59× decode attentionOpt-in via --attention-backend KIMI_K3_KV_BALANCED_MLA#63638
FlyDSL decode monokernel + DSpark verify8×MI355X, concurrency 1–8+35–57% interactivityDefault where applicable#63440
Qwen3.5 on breakable CUDA graphsNVIDIA; replaces torch.compile dependence3.0–8.8% lower decode ITLDefault for Qwen3.5#51012

The opt-in flag for the GLM-5.3-Flash path is worth keeping in your notes because it is the template for the MegaMoE family — one fused kernel dispatches and combines routed plus shared experts across the EP group, which is why it beats the generic path by 60% at high concurrency:

# MegaMoE wants data-parallel + expert-parallel, not plain TP:
vllm serve zai-org/GLM-5.3-Flash \
  --kernel-config '{"moe_backend": "deep_gemm_mega_moe"}' \
  --tensor-parallel-size 1 --data-parallel-size 8 --enable-expert-parallel

A honesty note on the largest single PR: #64077 (Muse Spark Open specialized decode, 42 files, +19,555 lines) is effectively a vendor kernel suite landing in-tree. That is how the project's performance story now works — NVIDIA and AMD ship tuned paths for specific model-silicon pairs, and the portable core improves more slowly. Read the model-support matrix before you plan capacity around any headline number.

Serving and Operations: CRIU Checkpoints, Fault Tolerance, NUMA

The sleeper feature for fleet operators is PR #62570: process checkpoint/restore built on CRIU and cuda-checkpoint, exposed through the HTTP API in both the Python and Rust frontends. This is the primitive that makes engine-level drain-and-snapshot workflows possible without inventing per-deployment hacks — a natural companion to the container-level CRIU story we covered in our Kubernetes checkpoint/restore guide, but at the inference-engine layer where the hard part has always been GPU state.

The wire contract, read directly from vllm/entrypoints/serve/checkpoint/api_router.py at the tag, is a small state machine with operator-friendly semantics: POST /checkpoint/prepare drains with a pause mode and releases everything a checkpoint must not contain (KV cache, CUDA graphs, activation workspace), returning as soon as the prepare starts — poll for completion. GET /checkpoint/status returns 200 once the engine is PREPARED (safe to capture), 503 while preparing or restoring, 500 after a failure. POST /checkpoint/restore is idempotent for orchestrator redelivery: a restore already in progress or already done returns 200. Repeating prepare while running is a no-op rather than an error.

graph TD
  R["RUNNING"] -->|"POST /checkpoint/prepare"| P["PREPARING"]
  P -->|"drain, release KV + graphs, repeat prepare is a no-op"| D["PREPARED"]
  D -->|"GET /checkpoint/status returns 200: external CRIU capture is safe"| C["external snapshot taken"]
  C -->|"POST /checkpoint/restore, redelivery-safe"| T["RESTORING"]
  T -->|"restore weights and GPU state"| R
  P -->|"failure"| F["FAILED"]
  F -->|"GET /checkpoint/status returns 500 with error field"| F
  F -->|"POST /checkpoint/restore rejected with 400"| F
  R -->|"GET /checkpoint/status returns 503 until PREPARED"| R
# 1. Ask the engine to reach a checkpointable state (returns immediately):
curl -s -X POST http://vllm:8000/checkpoint/prepare -H "Content-Type: application/json" \
  -d '{"mode": "wait"}'

# 2. Poll until 200 PREPARED (503 means still draining; 500 means failed):
until curl -s -o /dev/null -w "%{http_code}" http://vllm:8000/checkpoint/status | grep -q "^200$"
do sleep 2; done

# 3. Your CRIU / cuda-checkpoint snapshot runs here, outside the engine.

# 4. Hand the node back (safe to redeliver on orchestrator retry):
curl -s -X POST http://vllm:8000/checkpoint/restore

For big expert-parallel deployments, PR #46370 adds fault-tolerant scale-down: dead DP+EP ranks are removed and their experts redistributed without a restart, driven by POST /v1/fault_tolerance/apply. Pair it with --enable-ep-weight-filter (#64187), which cuts EPLB weight migration reads from 385 GiB to 105 GiB per rank on a DeepSeek-R1 NVFP4 DEP8 deployment. Also new: best-effort NUMA binding is on by default (#61258 — if your orchestrator already handles NUMA, this is a collision to audit; --no-numa-bind restores old behavior), opt-in cache-aware admission ordering via --cache-aware-admission-window (#54625, reorders the waiting queue to protect prefix-cache hits), and async KV loads that start while the batch is full via --max-num-inflight-kv-loads (#65319, with cached reservations dropping per-schedule time from 47.2 ms to 5.4 ms in the PR's own numbers, #65285).

Scheduler CPU overhead — the thing that bites at high request rates on fast GPUs — got a dedicated pass: prompt token ids stay in int32 buffers (40× less per-request CPU on a 128k prompt, #62702), per-request id lists stay out of the cyclic garbage collector (#65300), KV block allocation CPU drops 45–50% (#65831), and new requests and aborts are handled while the model is producing output (#65466) instead of between steps. The Rust frontend keeps closing the parity gap and adds a stateless Responses API (#63902), n>1 sampling (#65625), and external DP load balancing (#64451).

Dependencies, Wheels, and the ROCm 10.0 Cliff

PyTorch stays at 2.14.0 for CUDA — no forced framework bump this cycle. The floors that moved: huggingface_hub 2.0.0 (a major version — if your image pins hub 1.x for your own tooling, that conflict is now vLLM's problem too), tokenizers 0.23.3, timm 1.0.30 (#64779), transformers 5.16.1 with a ceiling below 5.23.0 (#64049, #64529), and mistral-common 1.13.0 (#64169). Kernel ecosystem: FlashInfer 0.7.3 (#65930), DeepEP 2.1 (#59844), NIXL 1.5.2 (#64579), mooncake-transfer-engine 0.3.14 (#65576).

AMD takes the sharpest edge: release artifacts are ROCm 10.0 only (#61939) — the 7.2 wheels and -rocm72 images are simply not published anymore, so a postponed ROCm upgrade stops being postponable the day you want vLLM 0.34.0. The install surface, from the release notes:

# CUDA 13.0 (PyPI default):
pip install vllm

# ROCm 10.0 (the only ROCm line now published):
pip install vllm --extra-index-url https://wheels.vllm.ai/rocm/0.34.0/rocm100

# XPU (note: the release notes themselves flag an open question about the
# vllm-xpu-kernels index for 0.1.17 - see #64633, #60609 before relying on it):
uv pip install vllm --extra-index-url https://wheels.vllm.ai/0.34.0/xpu \
  --extra-index-url https://download.pytorch.org/whl/xpu --index-strategy unsafe-best-match

# Containers: vllm/vllm-openai:v0.34.0 (CUDA 13.0, now ships the AWS EFA
# userspace), v0.34.0-efa (adds DeepEP built for EFA), v0.34.0-rocm,
# v0.34.0-cpu, v0.34.0-xpu

Build-side: the x86_64 CUDA wheel is about 82 MB (~25%) smaller with no kernel changes (#65518), source builds fetch triton_kernels from a tarball (#65162) and ship tuned kernel configs in the wheel (#64063), and CUDA arches no longer silently fall back to a lower minor (#66016) — build-time surprises now fail loudly instead of running slow.

Every Breaking Change, In One Place

BreakWhat to doPR
Model Runner V1 removed; VLLM_USE_V2_MODEL_RUNNER=0 fails at startupRemove the env var; port V1 logits processors to vllm.v1.worker.gpu.sample.logits_processor#63373
Default --block-size 64 (was 16)Pass --block-size 16 to keep old behavior; ROCm paged attention keeps 16 either way#63231
Extensible KV cache on; gpu_memory_utilization defaults to 1.0--gpu-memory-utilization 0.9 or --no-enable-extensible-kv-cache to keep old sizing#56492
ROCm 7.2 wheels/images droppedMove to ROCm 10.0 artifacts or stay on 0.33.x#61939
huggingface_hub 2.0.0 required (plus tokenizers 0.23.3, timm 1.0.30)Rebuild pinned environments; audit hub 1.x-only tooling#64779
QuTLASS / fp_quant removedfp_quant checkpoints no longer load — requantize or stay#65230
Marlin W4A8 removedUse --quantization humming with VLLM_HUMMING_INPUT_QUANT_CONFIG instead of VLLM_MARLIN_INPUT_DTYPE#65542
Keye-VL and Keye-VL-1.5 removedNo migration path — these models stop loading#64287
llama3_json no longer streams tool arguments incrementallyClient-visible: streaming tool-call consumers see arguments arrive differently (moved to the ParserEngine)#63569
Canary transcription without language returns 400 (was server default)Always pass language for Canary models#65445
LoRA adapters on modules vLLM does not wrap are refusedAudit adapter target lists before upgrading multi-LoRA deployments#63489
process_weights_after_loading hooks share one signature; loader runs every module's hookCustom quant loaders and forks must adopt the unified signature#57678
Config compute_hash() replaced by compile_factors() / hash_config()Anything keying compile caches off the old API breaks#65652

The Upgrade Runbook

For a fleet serving production traffic, this is the order that avoids the known failure modes. The two gates that will actually stop you are the dependency floor and the ROCm cliff; everything else degrades into warnings you can triage after the canary:

# 1. Environment first - resolve the huggingface_hub 2.0 floor in staging:
pip install --dry-run "vllm==0.34.0"   # surfaces hub/tokenizers/timm conflicts

# 2. Grep for hard-fail triggers before any rollout:
grep -rn "VLLM_USE_V2_MODEL_RUNNER\|fp_quant\|VLLM_MARLIN_INPUT_DTYPE" \
  charts/ overlays/ runbooks/

# 3. ROCm fleets: confirm ROCm 10.0 base image, or freeze at 0.33.x.

# 4. Capture pre-upgrade baselines (these all move):
#    - prefix-cache hit rate (block-size 64 changes hit alignment)
#    - TPOT / ITL at your SLO concurrency (extensible cache can cost 6-18%)
#    - KV cache token capacity per model (it should grow)
#    - startup warnings log

# 5. Canary with old sizing pinned, so step one isolates the other breaks:
vllm serve MODEL --gpu-memory-utilization 0.9 --block-size 16

# 6. Remove the pins one at a time. Watch for the #66704 warning when an
#    explicit utilization withholds 1 GiB or more - that is the signal the
#    old tuning advice has expired.

# 7. Re-baseline capacity math per model; the 8% slack is now KV cache.

Who Should Skip This Release

  • ROCm 7.2 fleets. There is no upgrade path short of a ROCm 10.0 migration. Freeze on the 0.33.x line until the platform move is done — this is the one hard blocker in the release.
  • Legacy W4A8 Marlin and fp_quant deployments. Your checkpoints stop loading. Requantize first (to humming with VLLM_HUMMING_INPUT_QUANT_CONFIG, or to a maintained format), then upgrade.
  • Latency-SLO fleets on Hopper that haven't measured the trade. The extensible KV cache's TPOT regression (+6–18% on some H200 models at concurrency 128) is default-on. Upgrade, but pin --gpu-memory-utilization 0.9 until you've seen your own numbers — or skip until a 0.34.x patch lands if your SLO has no headroom.
  • Teams with private V1-runner patches or custom_class speculator forks. The port is mandatory and non-trivial; budget it as a project, not a tag bump.
  • Keye-VL and Keye-VL-1.5 users. The models are removed outright. There is nothing in 0.34.0 for you.
  • Fleets that upgrade only on releases with published notes. Note the pattern this cycle: 0.33.0 shipped to PyPI with no release body. If you took it, you are already running unverified changes — this release is your chance to re-baseline onto documented behavior, which is an argument for the upgrade, not against it.

Honest Weaknesses and Open Questions

The release shipped with two reviewer TODO comments still visible in the release body itself — refreshing honesty, but worth knowing: the XPU install line may be missing the extra index that serves vllm_xpu_kernels==0.1.17 (#64633, #60609), and the --mamba-cache-mode removal timeline slipped past its own announced version, with the flag now warning "will be removed in v0.34.0" in v0.34.0 (#64838). The LM Format Enforcer backend is still deprecated rather than opt-in, contrary to the v0.33.0 announcement.

Other rough edges to plan around: Kimi K3.1 groundwork landed but K3.1 is not servable in this release (#66030, #66028) — do not read the model list optimistically. Aria moved to the Transformers modeling backend and pays for it temporarily: about 11% less KV cache and 25% lower image throughput, per the notes' own admission. The xpu and cpu wheel variants still lag the CUDA line in kernel coverage. And the governance gap this cycle — a stable release that shipped to PyPI without published notes — is a process risk that falls on operators to detect; if you automate upgrades off PyPI, add a check that the matching GitHub release exists before your pipeline proceeds.

Verdict

v0.34.0 is the release where vLLM's memory model stops being folklore. The extensible KV cache replaces the industry's most-copied cargo-cult setting with measurement, Model Runner V1's deletion pays down two years of parallel-runner complexity, and the checkpoint/restore API plus fault-tolerant scale-down finally give fleet operators the lifecycle primitives that disaggregated serving assumed existed. The costs are real and front-loaded: two mandatory migrations (MRV1 code, Marlin/fp_quant checkpoints), one conditional one (ROCm 10.0), and a default flip that trades per-token latency for capacity without asking. Upgrade in a window where you can re-baseline — and treat the new startup warnings as the retirement letter for every --gpu-memory-utilization 0.9 you copied from a blog post in 2024.