Every LLM platform hits the same wall, usually around the third week: the GPUs are no longer the constraint — the queue in front of them is. Once more teams want capacity than the fleet can serve, your scheduling layer stops being plumbing and becomes the product. Who waits, who gets preempted, who silently hoards quota while a nightly batch job holds eight H100s at 4% utilization — those decisions are made by whatever admits workloads, and in 2026 that layer on Kubernetes is Kueue.

This is the deep dive into the queue problem half of GPU scheduling: admission ordering, quota charging, preemption, and fairness. The bin-packing half — MIG slicing, time-slicing, DRA prioritized lists, overcommit ratios — lives in our GPU capacity planning guide, and this piece deliberately does not repeat it. Read that one first if you are still deciding how much GPU each workload gets; read this one when the fight is over who gets it first. The device layer that both halves sit on — DRA, GA since Kubernetes 1.34 — is covered in the NVIDIA DRA driver's own docs; production NVIDIA GPU fleets pair that driver with the GPU Operator's DRA setup.

The timing is not arbitrary. Kueue shipped v0.19.6 on September 24, 2026, a patch release whose entire reason to exist is admission correctness: two separate quota-leak bugs that let a fleet admit workloads beyond quota or — worse — strand quota on workloads that no longer exist, starving the queue. Five days later, v0.20.0-rc.2 landed the biggest fair-sharing semantics change since the feature went beta: usage now anchors at quota reservation, not admission. If you run Kueue in front of GPUs, both releases change what your queue actually does.

TL;DR

  • The kube-scheduler cannot arbitrate scarcity. Pod priority is per-pod, quota-blind, and has no concept of a team, a budget, or a waiting queue. Admission control is a separate layer, and for job-shaped work it is Kueue (stable line v0.19.6, parallel line v0.18.10, both released 2026-09-24; official support starts at Kubernetes 1.34).
  • Priority that matters lives on the Workload, not the Pod. WorkloadPriorityClass orders the admission queue and drives preemption without touching in-cluster pod priority — so a batch job that preempts nothing at runtime can still win the admission race.
  • Fairness has two independent knobs. Admission Fair Sharing orders who gets in by LocalQueue usage history; cohort fair sharing plus preemption strategies decides who gets evicted when a queue over-borrows. v0.20 moves the usage anchor to quota reservation, which changes the first knob's behavior even if you change nothing else.
  • DRA made GPU quota honest, and complicated. Kueue now charges DRA devices four ways — by count, by extended resource, by MIG counter, by shared capacity — and the v0.19.6 patch fixed a double-charge bug in exactly the path an NVIDIA DRA fleet uses. Pick your accounting mode deliberately; it is a one-way door per DeviceClass.
  • Quota leaks are the failure mode nobody dashboards. The two v0.19.6 headline fixes — a StatefulSet scaled to zero that kept its reservation, and a Job with parallelism > completions that handed quota back early — are both silent capacity theft. Watch kueue_pending_workloads against admitted usage, or you will find them by outage.

The Structural Shift: Scarcity Moves the Bottleneck Up the Stack

For fifteen years the Kubernetes scheduling conversation has been about placement: which node, which zone, which topology domain. Kube-scheduler is genuinely excellent at placement. What it has never had is a notion of admission at the fleet level. Its only scarcity signal is pod priority — a number that says "this pod may evict that pod on the same node," which is a node-local argument, not a fleet-level queue discipline.

LLM workloads break the pod-first model in three specific ways:

  1. The unit of demand is not a pod. A fine-tune is 8 GPUs for 6 hours. An eval sweep is 64 GPUs for 40 minutes. An agent fleet is 200 pods that each want a fraction of a GPU. The thing you need to order, charge, and evict is the job, not the pod — evicting one pod of a distributed training run just wastes the other 63.
  2. Demand exceeds supply structurally, not transiently. A fleet at 95% GPU utilization is healthy; a cluster at 95% CPU utilization is an incident. The queue is permanent, so its ordering policy is your capacity allocation policy. FIFO-by-default means the loudest submitter wins.
  3. Cost asymmetry makes wrong admission expensive. Admitting a 6-hour fine-tune ahead of a 40-minute production batch is not an inconvenience; it is four hours of SLA breach. Priority, preemption, and fairness are the levers that make that tradeoff explicit instead of accidental.

Kueue's answer is the Workload: an API object representing the job's total demand (all pod sets, all resources, all DRA claims), which exists in one of two states — pending in a queue, or admitted with a quota reservation. The user's Job (or JobSet, RayJob, MPIJob, plain pod group) is created suspended; Kueue unsuspends it only when the Workload clears the queue. The entire discipline happens before kube-scheduler sees a single pod:

   user submits                 kueue control plane                kube cluster
  ---------------  ---------------------------------------------  ----------------
  Job (suspended) --> Workload created (podSets, requests, DRA)
                          |
                          v
                    LocalQueue (namespace-scoped router)
                          |
                          v
                    ClusterQueue (quota + policy)
                    - queueingStrategy: BestEffortFIFO|StrictFIFO
                    - nominalQuota per flavor, borrowing via Cohort
                    - preemption: withinClusterQueue / cohort policies
                    - admissionScope: fair sharing on/off
                          |
                    [ scheduler cycle: order by priority,
                      then fair-share share value ]
                          |
                          v
                    QuotaReserved=True  <--- AdmissionChecks
                          |                 (ProvisioningRequest,
                          |                  external validators)
                          v
                    Job unsuspended --> kube-scheduler places pods
                          |              (DRA: allocates the ACTUAL device)
                          v
                    PodsReady within timeout?
                      yes --> running until done
                      no  --> evict + requeue with backoff
                                  (quota returns to the pool)