RAG Pipelines for Platform Teams: Vector Stores, Chunking, and Evaluation

1. Why RAG in Late 2026: The Stack Moved, the Debate Did Not

Every few months a blog post announces that RAG is dead — that long-context models, prompt caching, or agentic search have made retrieval pipelines obsolete. The discourse is real: Hacker News spent the spring of 2026 arguing through threads like Is RAG Dead? Long Context, Grep, and the End of the Mandatory Vector DB. The claim is half right, and the half that is right matters for architecture: naive chunk-and-embed RAG is dead. Retrieval engineering is not. The pipeline your platform team runs in production just looks nothing like the 2023 tutorial.

What actually changed in the last quarter is the retrieval stack itself, and all of it is verifiable from primary sources:

• Qdrant v1.20.0 (October 22, 2026) added BM25 scoring directly on the full-text payload index, which means one database now serves the dense and the lexical side of hybrid retrieval without a separate Elasticsearch cluster. The same release shipped TurboQuant (turbo8/turbo16 vector datatypes), fixed HNSW layer assignment to match the original paper (6.25% of points on level 1 at m = 16, down from 25% — smaller upper-layer graphs, less memory), and added hnsw_config.inline_storage to stop storing a second copy of vectors outside the graph file.
• Cohere shipped Embed 5 (September 30, 2026) with a shared embedding space across the Pro and Fast variants — index once with Pro, query with Fast — 128k-token context, Matryoshka output dimensions from 256 to 2048, and int8/binary output types. Rerank 5 followed with a 32,768-token context window and calibrated 0–1 relevance scores.
• Voyage AI's voyage-4 family (January 2026) made 1024 dimensions the default with configurable 256/512/2048 Matryoshka truncation, and released voyage-4-nano as an open-weight model on Hugging Face — the first time a frontier-tier embedding family has a self-hostable sibling.
• RAGAS moved to vibrantlabsai/ragas and shipped v0.4.3 — the evaluation framework your team either already runs or is about to be asked for by an auditor.

Meanwhile, the economics shifted underneath: OpenAI's prompt caching now discounts cached input tokens up to 95%, with cache reads at 0.1× the ordinary input rate for GPT-5.6 and later, and Anthropic sells 5-minute cache entries at 1.25× write and roughly 0.1× read. That changes the small-corpus calculus enough that we will compute the exact crossover in section 9 — and the answer is not "RAG always wins." Spoiler: below roughly 225k tokens of stable, frequently-queried corpus, stuffing the whole thing into a cached prompt can be cheaper than the retrieval stack in front of a cheap generation model.

This guide is the reference architecture we would hand a platform team starting a production RAG pipeline in November 2026: chunking strategy, embedding model selection, vector store selection with exact configuration parameters, hybrid retrieval, reranking, an evaluation harness wired into CI, complete Kubernetes manifests, and a cost model computed from the live rate cards. Everything version-specific was verified against primary sources on November 10, 2026 — release notes, official docs, and pricing pages, not vendor blog posts.

2. Reference Architecture: Five Planes

A production RAG pipeline is five planes with different failure modes, different owners, and different cadences. The diagram below is the shape we will build through the rest of this guide.

                        ┌─────────────────────────────────────────────────┐
                        │              GOVERNANCE PLANE                   │
                        │  git = source of truth · index registry          │
                        │  tenant policy · eval thresholds in CI          │
                        └───────────────┬─────────────────────────────────┘
                                        │ config + gates
        ┌───────────────────┐   ┌───────▼────────┐   ┌──────────────────────┐
        │  INGESTION PLANE  │   │  INDEX PLANE   │   │    EVALUATION PLANE  │
        │───────────────────│   │────────────────│   │──────────────────────│
        │ source connectors │   │ Qdrant 1.20    │   │ golden dataset       │
        │ parse (PDF/MD/    │──▶│ dense (HNSW)   │◀──│ RAGAS 0.4.3 metrics  │
        │  code/wikis)      │   │ sparse (BM25   │   │ recall@k · MRR        │
        │ chunker           │   │  payload index) │   │ CronJob in K8s       │
        │ contextualizer    │   │ payload indexes│   │ CI gate on regress   │
        │ embedder          │   │ (tenant, src)  │   └──────────────────────┘
        │ upsert batches    │   └───────▲────────┘
        └───────────────────┘           │ points + payload
        runs on: Job / CronJob           │
        cadence: hourly/daily   ┌────────┴────────────┐
                              │   RETRIEVAL PLANE    │
                              │──────────────────────│
                              │ query embed          │
                              │ hybrid: dense + BM25 │
                              │ RRF fusion           │
                              │ rerank top-150→20   │
                              │ assemble context     │
                              │ ─────────────────── │
                              │ serves: Deployment  │
                              │ +HPA, p99 budget    │
                              └────────┬─────────────┘
                                       │ top-k chunks
                                       ▼
                              ┌─────────────────────┐
                              │  GENERATION (LLM)   │
                              │ prompt-cached prefix│
                              └─────────────────────┘

Three properties of this shape are deliberate. First, the index plane holds one database: Qdrant 1.20 with dense vectors and BM25 on the same points, so a chunk never has a lexical ID in one system and a vector ID in another. Second, ingestion and retrieval are decoupled — the ingestion plane can re-chunk and re-embed the corpus without touching the serving path, which is the capability you need when an embedding model upgrade forces a full reindex (it always does). Third, evaluation is a plane, not a notebook — it runs as a scheduled job with thresholds that fail the build, because a RAG pipeline without retrieval metrics is a pipeline nobody can prove still works.

3. Chunking: The Decision That Dominates Retrieval Quality

Chunking is where most RAG quality is won or lost, and it is the cheapest place to improve. The reason is structural: embedding models compress a chunk into a single vector, and anything the chunk does not say — which company, which quarter, which service — is invisible to that vector. Anthropic's contextual retrieval write-up frames it exactly right: a chunk reading the company's revenue grew by 3% over the previous quarter is unanswerable on its own, because it does not say which company or which quarter.

3.1 The chunking ladder

Four strategies, in increasing order of quality and cost:

• Fixed-size token chunks (400–1000 tokens, 10–20% overlap). The baseline. Cheap, fast, and blind to document structure — sentences and tables get cut mid-thought. Use it only for homogeneous corpora where boundaries do not carry meaning.
• Structure-aware recursive chunking: split by headers first, then paragraphs, then sentences, then tokens — the default in every serious ingestion pipeline. The chunker below implements this. It preserves semantic boundaries and is what you should ship if you ship nothing else.
• Parent-child (small-to-big) retrieval: index small child chunks (say 200 tokens) for precise vector matches, but return their parent section (1200–2000 tokens) to the generator. You get retrieval precision and generation context without choosing one chunk size for both jobs. This is the single best quality-per-effort move in most pipelines.
• Contextual chunking: prepend a generated 50–100 token context blurb to each chunk before embedding and before BM25 indexing — either the Anthropic contextual-retrieval prompt or late chunking (below). This is the current quality ceiling and has a real price tag.

3.2 Contextual retrieval: verified numbers and the hidden cost

Anthropic's published results remain the canonical benchmark for the contextual family of techniques, and the numbers are worth committing to memory. Measured as top-20 retrieval failure rate (1 minus recall@20) across codebases, fiction, arXiv, and science-paper corpora:

Two production facts from the same write-up that blogs tend to drop: passing 20 chunks to the model outperformed 5 and 10 in their tests, and Voyage and Gemini embeddings were the strongest of the models they evaluated. Also note what the ladder implies: hybrid + rerank delivers more gain than contextualization alone. If you can only afford one improvement this quarter, build hybrid retrieval and reranking before you build contextual chunking.

The cost side is the part that surprises teams. Anthropic computed $1.02 per million document tokens to generate contextual blurbs (800-token chunks, 8k-token documents, 50–100 token contexts, Claude 3 Haiku with prompt caching). Scaled to a real corpus, that is $510 one-time for a 500M-token corpus — fifty times the raw embedding bill on voyage-4. Contextualization is an embedding-model-independent, one-time, re-payable cost: every re-chunk or model upgrade pays it again.

3.3 Late chunking: the alternative that avoids the LLM bill

Late chunking (Günther, Mohr, and Williams, arXiv 2409.04701) takes the opposite route: instead of generating context per chunk with an LLM, it embeds the whole document through a long-context embedding model and applies mean pooling per chunk over the full-document token embeddings. Each chunk vector then carries the document's context because the model saw the entire document before the chunk boundary was applied. The trade: you need an embedding model with real long-context quality (the paper used an 8k-context Jina model; modern candidates are the 32k voyage-4 family or the 128k Cohere Embed 5), and per-chunk vector quality depends on the model's long-range attention rather than an explicit prompt. For corpora where documents fit the embedding model's context window, late chunking is contextualization without the $510 bill.

3.4 A complete ingestion pipeline

The ingestion stage below is complete and runnable: structure-aware recursive chunking, optional contextualization, voyage-4-lite embeddings, and batched upserts into Qdrant with tenant and provenance payload. It runs as a Kubernetes Job (section 8) against a mounted corpus directory.

#!/usr/bin/env python3
"""Ingest a Markdown corpus into Qdrant with contextual chunks.

Env vars:
  VOYAGE_API_KEY   — Voyage AI key
  QDRANT_URL       — http://qdrant.qdrant.svc.cluster.local:6333
  QDRANT_API_KEY   — Qdrant API key (empty for in-cluster no-auth)
  CLAUDE_API_KEY   — optional; enables contextualization
  CORPUS_DIR       — /data/corpus (markdown files)
  TENANT           — tenant id written to every point's payload
  BATCH            — points per upsert (default 64)
"""

import os
import time
import pathlib
import requests

QDRANT_URL = os.environ["QDRANT_URL"]
COLLECTION = "corpus_chunks"
TENANT = os.environ["TENANT"]
BATCH = int(os.environ.get("BATCH", "64"))
VOYAGE_KEY = os.environ["VOYAGE_API_KEY"]
CLAUDE_KEY = os.environ.get("CLAUDE_API_KEY", "")
EMBED_MODEL = "voyage-4-lite"
EMBED_DIMS = 1024

HEADERS_V = {"Authorization": "Bearer " + VOYAGE_KEY}
HEADERS_Q = {}
if os.environ.get("QDRANT_API_KEY"):
    HEADERS_Q["api-key"] = os.environ["QDRANT_API_KEY"]


def split_recursive(text, max_tokens=800, overlap_tokens=120):
    """Structure-aware chunking: paragraphs, then sentences, then tokens.

    Approximates tokens at 4 chars/token — deterministic and fast.
    Yields (position, chunk_text) with sentence overlap between chunks.
    """
    paras = [p.strip() for p in text.split("\n\n") if p.strip()]
    # merge paragraphs into ~max_tokens units, never splitting a paragraph
    # unless it alone exceeds the budget
    units = []
    cur = []
    cur_len = 0
    for p in paras:
        plen = len(p) // 4
        if cur_len + plen > max_tokens and cur:
            units.append("\n\n".join(cur))
            cur, cur_len = [], 0
        if plen > max_tokens:
            units.append(p)
        else:
            cur.append(p)
            cur_len += plen
    if cur:
        units.append("\n\n".join(cur))
    # sentence overlap: re-attach the tail of the previous unit
    out = []
    for i, u in enumerate(units):
        if i > 0:
            tail = units[i - 1].split(". ")
            tail_text = ". ".join(tail[-2:])
            if len(tail_text) // 4 <= overlap_tokens:
                u = tail_text + " " + u
        out.append(u)
    return list(enumerate(out))


def contextualize(doc_text, chunk_text):
    """Prepend a 50-100 token situating context to a chunk (Anthropic method).
    Returns the chunk unchanged when no CLAUDE_API_KEY is configured."""
    if not CLAUDE_KEY:
        return chunk_text
    prompt = (
        "\n" + doc_text[:28000] + "\n\n"
        "Here is the chunk we want to situate within the whole document\n"
        "\n" + chunk_text + "\n\n"
        "Please give a short succinct context to situate this chunk within "
        "the overall document for the purposes of improving search retrieval "
        "of the chunk. Answer only with the succinct context and nothing else."
    )
    r = requests.post(
        "https://api.anthropic.com/v1/messages",
        headers={
            "x-api-key": CLAUDE_KEY,
            "anthropic-version": "2023-06-01",
            "content-type": "application/json",
        },
        json={
            "model": "claude-haiku-4-5",
            "max_tokens": 200,
            "messages": [{"role": "user", "content": prompt}],
        },
        timeout=60,
    )
    r.raise_for_status()
    ctx = r.json()["content"][0]["text"].strip()
    return ctx + "\n\n" + chunk_text


def embed(texts, input_type="document"):
    """Voyage embedding call with client-side retry/backoff."""
    for attempt in range(5):
        r = requests.post(
            "https://api.voyageai.com/v1/embeddings",
            headers=HEADERS_V,
            json={
                "input": texts,
                "model": EMBED_MODEL,
                "input_type": input_type,
                "dimensions": EMBED_DIMS,
            },
            timeout=120,
        )
        if r.status_code == 429:
            time.sleep(2 ** attempt)
            continue
        r.raise_for_status()
        return [d["embedding"] for d in r.json()["data"]]
    raise RuntimeError("embedding API rate-limited after 5 retries")


def main():
    corpus = pathlib.Path(os.environ["CORPUS_DIR"])
    points = []
    pid = 0
    files = sorted(corpus.rglob("*.md"))
    for path in files:
        doc_text = path.read_text(encoding="utf-8")
        for pos, chunk in split_recursive(doc_text):
            final_text = contextualize(doc_text, chunk)
            points.append({
                "id": pid,
                "vector": None,  # filled below
                "payload": {
                    "tenant_id": TENANT,
                    "source_path": str(path.relative_to(corpus)),
                    "chunk_pos": pos,
                    "text": chunk,
                },
            })
            pid += 1
    print("chunks:", len(points))

    # embed in batches, then upsert in batches
    for i in range(0, len(points), BATCH):
        batch = points[i:i + BATCH]
        vectors = embed([p["payload"]["text"] for p in batch])
        for p, v in zip(batch, vectors):
            p["vector"] = v
        r = requests.put(
            QDRANT_URL + "/collections/" + COLLECTION + "/points",
            headers=HEADERS_Q,
            json={"points": batch},
            timeout=120,
        )
        r.raise_for_status()
        print("upserted", i + len(batch), "of", len(points))

    # BM25 side: full-text index lives on the payload 'text' field
    print("ingestion complete")


if __name__ == "__main__":
    main()

Three details in that code are load-bearing for later sections. The tenant_id payload field exists on every point — section 5 turns it into a payload index with is_tenant: true. The source_path and chunk_pos fields make every retrieved chunk traceable to a file and position, which is what your evaluation harness (section 7) needs to compute recall against a golden set. And the raw text is stored in the payload itself — that is what Qdrant 1.20's BM25 scores against, so the lexical index and the vector index never drift apart.

4. Embedding Models: The Verified Late-2026 Field

Embedding model choice sets three hard constraints at once: retrieval quality, the dimensionality of every vector you will ever store, and the size of the reindex you owe the day you change your mind. The table below is the current API-facing field, verified from each vendor's own pricing and docs pages on November 10, 2026.

Read that table as three tiers. The cheap tier ($0.02/1M) is no longer a compromise: voyage-4-lite at 1024 dimensions is a serious retrieval model, and a 500M-token corpus costs $6 to embed after Voyage's 200M free tokens. The frontier tier is voyage-4-large or Cohere's embed-v5.0-pro, and the interesting 2026 development is Cohere's shared embedding space: because Pro and Fast share the space, you can index offline with Pro and serve queries with Fast against the same vectors — quality where it is free (index time), latency where it is expensive (query time). The self-hosted tier finally exists for teams with data-residency constraints: voyage-4-nano is open-weight, and Qwen3-Embedding models are permissively licensed.

4.1 Matryoshka dimensions are now table stakes — use them deliberately

Every current family supports shortening: voyage-4 emits 256/512/1024/2048 from one call, Cohere Embed 5 offers [256, 512, 768, 1024, 1536, 2048], and OpenAI's models accept a dimensions parameter — OpenAI's own guide confirms 1536 for text-embedding-3-small and 3072 for text-embedding-3-large by default, with shortening that preserves "concept-representing properties." The mechanics matter: a shortened embedding must be L2-renormalized (slice the first N components, divide by the vector's new magnitude) or similarity scores silently break. The platform move is to pin your storage schema to one deliberate dimension now — 1024 is the 2026 consensus default — rather than inheriting each model's maximum and paying 3× memory for quality you will not measure.

4.2 The migration rule nobody budgets for

Vectors from different embedding models live in incompatible spaces. There is no vector-space migration path from text-embedding-3-large to voyage-4: changing the model means re-embedding every chunk and rebuilding every index — the full corpus bill again, plus the contextualization bill if you contextualize, plus a retrieval-quality regression window while your golden set revalidates. Treat the embedding model like a database major version: pin it in config, gate changes behind the evaluation harness (section 7), and schedule reindex windows. This is also why the ingestion plane from section 2 must be able to run end-to-end on demand — not a one-time script.

5. Vector Store Selection: pgvector, Qdrant, or Managed

The 2026 answer to "which vector database" is less about benchmarks and more about what your platform already operates. Two viable self-hosted paths cover almost everyone: pgvector when PostgreSQL is already your operating model, and Qdrant when retrieval is a first-class, multi-tenant, high-QPS service. Managed services (Pinecone, Qdrant Cloud) buy you the operations you do not want to own.

5.1 pgvector v0.8.7: the parameters that actually matter

pgvector's HNSW defaults are m = 16 and ef_construction = 64, with query-time hnsw.ef_search defaulting to 40 — all confirmed in the current README. The two mechanics that break production systems on pgvector are filtered search and index memory, so here is the complete working setup with both handled.

-- extension and schema
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id          bigserial PRIMARY KEY,
    tenant_id   text NOT NULL,
    source_path text NOT NULL,
    chunk_pos   int  NOT NULL,
    text        text NOT NULL,
    embedding   vector(1024) NOT NULL
);

-- HNSW index with verified defaults made explicit
CREATE INDEX ON chunks
    USING hnsw (embedding vector_cosine_ops)
    WITH (m = 16, ef_construction = 64);

-- CRITICAL: with filters, HNSW applies the filter AFTER the graph scan.
-- A filter matching 10% of rows with the default ef_search of 40 returns
-- ~4 matching rows on average. Iterative scans (pgvector 0.8.0+) fix this:
SET hnsw.iterative_scan = strict_order;   -- or relaxed_order
-- hnsw.max_scan_tuples caps the scan; hnsw.scan_mem_multiplier (multiple of
-- work_mem, default 1) bounds the memory the iterative scan may use.

-- tenant isolation at the planner level: list partitioning
-- CREATE TABLE chunks (...) PARTITION BY LIST (tenant_id);

-- half-precision: index up to 4000 dims, halve vector bytes on disk/heap
CREATE INDEX ON chunks
    USING hnsw ((embedding::halfvec(1024)) halfvec_cosine_ops);

-- binary quantization + re-scoring: smallest index, exact re-rank after
CREATE INDEX ON chunks
    USING hnsw ((binary_quantize(embedding)::bit(1024)) bit_hamming_ops);
WITH candidates AS MATERIALIZED (
    SELECT * FROM chunks
    ORDER BY binary_quantize(embedding)::bit(1024)
             <~> binary_quantize('[0.1, ...]'::vector(1024))
    LIMIT 20 * 10
)
SELECT * FROM candidates
ORDER BY embedding <=> '[0.1, ...]'::vector(1024)
LIMIT 20;

The memory facts to plan around, straight from the README: a vector(1024) costs 4,104 bytes of row storage (4 bytes per dimension plus 8), a halfvec(1024) 2,056, and a binary-quantized bit vector 136. HNSW graph build spills out of maintenance_work_mem after about 100,000 tuples with a server NOTICE, so raise maintenance_work_mem for the index build and put it back — and remember the FAQ's guidance that indexes perform best when they fit in memory, measurable directly with pg_relation_size(). For a 625k-chunk corpus at 1024 dimensions, budget roughly 2.5 GB of vector storage before the graph, or 1.28 GB on halfvec, or 85 MB binary-quantized.

Where pgvector wins: one backup story, one security story, one set of runbooks, joins between vectors and relational data in a single query, and row-level tenancy with Postgres-native tools. Where it hurts at scale: no native BM25 (you will add PostgreSQL full-text search or an external lexical index and fuse it yourself), index rebuilds take locks and memory you must schedule, and per-tenant isolation is manual partitioning rather than a first-class concept.

5.2 Qdrant v1.20.0: one database for dense and lexical

Qdrant's case in late 2026 rests on v1.20.0's headline feature: BM25 scoring directly on the full-text payload index. The chunk text you already store in the payload becomes the lexical index — no second system, no ID sync. The same release fixed HNSW level assignment to match the original paper (6.25% of points on level 1 at m = 16 instead of 25%, shrinking the upper graph layers) and added hnsw_config.inline_storage so vectors are stored only inside the graph file instead of twice.

Here is the complete collection definition for the ingestion pipeline from section 3 — dense voyage-4 vectors, BM25 over the payload text, tenant-aware payload indexes, int8 scalar quantization, and the strict-mode discipline that keeps filter performance predictable:

{
  "vectors": {
    "size": 1024,
    "distance": "Cosine",
    "on_disk": false
  },
  "sparse_vectors": {},
  "hnsw_config": {
    "m": 16,
    "ef_construct": 100,
    "full_scan_threshold": 10000
  },
  "quantization_config": {
    "scalar": {
      "type": "int8",
      "quantile": 0.99,
      "always_ram": true
    }
  },
  "optimizer_config": {
    "default_segment_number": 4,
    "indexing_threshold_kb": 20000
  }
}

# tenant index: groups each tenant's points for filterable HNSW traversal
curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index?wait=true" \
  -H "api-key: $QDRANT_API_KEY" \
  -H "content-type: application/json" \
  -d '{"field_name": "tenant_id", "field_schema": {"type": "keyword", "is_tenant": true}}'

# provenance indexes for filtering and dedupe
curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index?wait=true" \
  -H "api-key: $QDRANT_API_KEY" \
  -H "content-type: application/json" \
  -d '{"field_name": "source_path", "field_schema": {"type": "keyword"}}'
curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index?wait=true" \
  -H "api-key: $QDRANT_API_KEY" \
  -H "content-type: application/json" \
  -d '{"field_name": "chunk_pos", "field_schema": {"type": "integer"}}'

# the BM25 side of hybrid: full-text index on the chunk text itself (v1.20)
curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index?wait=true" \
  -H "api-key: $QDRANT_API_KEY" \
  -H "content-type: application/json" \
  -d '{"field_name": "text", "field_schema": {"type": "text", "tokenizer": "word"}}'

Config notes, each verified against the v1.20 docs: HNSW defaults are m = 16 and ef_construct = 100 (Qdrant builds a wider graph than pgvector's 64), and full_scan_threshold: 10000 is the segment-size boundary below which the planner skips HNSW entirely and brute-forces — faster and exact for small segments. Search-time ef defaults to ef_construct. The payload index flags matter: is_tenant: true (available since Qdrant 1.11) organizes the graph so same-tenant points link together, which is what keeps filtered vector search fast in a multi-tenant collection. And strict mode — reject queries that filter on unindexed fields, on by default for new Qdrant Cloud collections via unindexed_filtering_retrieve: false — is the difference between a slow query today and a page-during-dinner incident, because an unindexed payload filter degrades to a full scan.

5.3 Memory tiers and quantization: the honest configuration ladder

Qdrant assigns dense vectors, the HNSW graph, quantized vectors, payloads, and payload indexes to separate memory tiers — pinned (in RAM), cached (memory-mapped, warmed by use), and cold (memory-mapped, unwarmed). Their operations guide is unusually blunt about the combinations, and it is worth quoting the shape of the advice: cache a compressed copy alongside full-precision vectors only if RAM fits both; an unquantized cold tier is "the configuration to avoid" for anything latency-sensitive because every first touch on a page pays a disk read, and HNSW traversal touches many pages. The recommended end-state for RAM-constrained, latency-sensitive search is dense vectors on the cold tier with the int8-quantized copy pinned — Qdrant rescoring against full-precision vectors is on by default for binary quantization and low-precision TurboQuant, so recall survives the compression. TurboQuant's new turbo8 and turbo16 datatypes (v1.20) extend that ladder below int8 for very large collections.

5.4 The decision matrix

6. Retrieval Patterns: Dense, Hybrid, Rerank, Agentic

Retrieval quality is a ladder, and the rungs have verified prices. This section walks the ladder with exact request shapes.

6.1 Hybrid retrieval with RRF — the request shape

The baseline you should ship is dense + BM25 fused with Reciprocal Rank Fusion. Qdrant's Query API expresses it as two prefetches and a fusion step. The BM25 leg scores the payload text index from section 5; the dense leg takes the query embedding. RRF needs no score calibration — it works only with ranks, which is why it is the safe default between retrievers with incomparable score distributions:

{
  "prefetch": [
    {
      "query": { "text": "how does the blast radius pattern limit config rollouts" },
      "using": "text",
      "filter": {
        "must": [
          { "key": "tenant_id", "match": { "value": "acme-prod" } }
        ]
      },
      "limit": 150
    },
    {
      "query": [0.013, -0.042, 0.771],
      "using": "dense",
      "filter": {
        "must": [
          { "key": "tenant_id", "match": { "value": "acme-prod" } }
        ]
      },
      "limit": 150
    }
  ],
  "query": { "rrf": {} },
  "limit": 150
}

RRF's formula is score(d) = sum over rankings of 1 / (k + rank) with zero-based ranks — the top result of a single list scores 1/k. The constant k has been configurable since Qdrant 1.16.0, and per-prefetch weights (w_r) let you bias dense versus lexical when your eval set says one leg deserves it. The alternative fusion, DBSF (v1.11.0+), normalizes each retriever's scores with 3-sigma statistics from the returned top-k and sums them; it is stateless, falls back to 0.5 when all scores are identical, and can beat weighted RRF on well-calibrated retrievers — but a single outlier in a small prefetch window skews its normalization, so raise prefetch limits before reaching conclusions. Their docs' advice is the right meta-advice: neither dominates in general; choose with your eval set.

6.2 Rerank: the biggest quality jump per line of code

Retrieve 150, rerank, keep 20. Anthropic's ladder showed reranked contextual hybrid cutting retrieval failures by 67% versus plain embeddings — and rerankers are a bolt-on: Cohere explicitly positions Rerank 5 as improving "search relevance on top of existing keyword, vector, or hybrid search, with no re-indexing." The two production options:

• Voyage rerank-3 / rerank-3-lite — billed per processed token: (query tokens × documents) + sum of document tokens. For our standard query (50-token query, 150 documents of 800 tokens), that is 127,500 processed tokens: $0.0064 per query on rerank-3 ($0.05/1M) or $0.0026 on rerank-3-lite ($0.02/1M). Voyage's own per-request estimate for 100 documents × 500 tokens is ~$0.0025 (rerank-3) — same order.
• Cohere rerank-v5.0-pro / fast — a 32,768-token window (query + document combined, ~2B and ~500M parameters respectively), calibrated relevance scores in [0, 1], 100+ languages, and Context Focus: ask for relevant_spans and each returned document carries character offsets into the passages that actually matched — so you can pass the LLM only the relevant span of each chunk instead of all 800 tokens. A search unit is one query with up to 100 documents.

The calibrated-score property is quietly important for production: a fixed relevance threshold (say 0.3) becomes a decision point — below threshold, return "I don't know" or trigger another retrieval strategy, instead of confidently generating from irrelevant context. Napkin math for the rerank step in Kubernetes: it is a single HTTPS call per query, so the latency budget is one network RTT plus model inference — low double-digit milliseconds for the fast tiers, more for pro models over 150 long documents.

6.3 Agentic retrieval: when one shot is not enough

The 2026 pattern is retrieval as a tool the model calls: the agent sees a question, decides to search, reads results, decides to search again with better terms or a different tenant scope, and stops when it can answer. This beats single-shot retrieval on multi-hop questions ("compare the rollback procedure in v3.6 with the v2.x one") and on questions where the query itself needs decomposition. The costs are honest ones: N× retrieval calls and N× model turns per question (each turn pays your generation model's input token rate on the growing conversation), latency measured in seconds instead of hundreds of milliseconds, and an eval harness that must now score trajectories, not single answers — RAGAS's agent metrics (tool call accuracy, agent goal accuracy, topic adherence) exist for exactly this. Ship agentic retrieval for the internal copilot where a 6-second answer is fine; do not put it in the synchronous search box with a 500 ms p99.

7. Evaluation: The Part Teams Skip and Then Regret

A RAG pipeline is a search engine glued to a language model, and search engines rot silently: the corpus changes, the chunker changes, the embedding model gets upgraded, and nobody notices quality degraded until a customer does. The evaluation plane exists to make that rot loud. Build it in three layers, in this order.

7.1 The golden set: provenance is the label

Start with 50–200 real questions (from logs, support tickets, and the questions your engineers actually ask in Slack), each labeled with the source of truth — the source_path (and ideally chunk_pos) of the chunk that contains the answer. That is why section 3 stored provenance in every point's payload: it turns retrieval evaluation into a set-membership computation instead of a human-judgment problem. Add adversarial cases on purpose: questions whose answer is not in the corpus (measures your "I don't know" behavior), questions with exact identifiers (measures the BM25 leg), and questions phrased in vocabulary the corpus never uses (measures the dense leg).

7.2 Layer one: deterministic retrieval metrics in every CI run

Recall@k and MRR (mean reciprocal rank) over the golden set are deterministic, free, and catch the majority of regressions — a chunker change, a broken payload index, a botched reindex. These belong in CI with hard gates. RAGAS v0.4.3's current API makes the harness short:

#!/usr/bin/env python3
"""Retrieval eval: golden set -> recall@20 + MRR + exact-answer gate.

Env: QDRANT_URL, QDRANT_API_KEY, VOYAGE_API_KEY
Input:  golden.jsonl lines of
        {"query": "...", "tenant_id": "...", "expected_paths": ["docs/x.md"], "expected_answer": "..."}
Exit codes: 0 pass, 1 regression (CI gate).
"""

import json
import os
import sys
import requests

from ragas import Dataset, experiment
from ragas.metrics import MetricResult, discrete_metric

QDRANT_URL = os.environ["QDRANT_URL"]
COLLECTION = "corpus_chunks"
HEADERS = {"api-key": os.environ.get("QDRANT_API_KEY", "")}
VOYAGE_KEY = os.environ["VOYAGE_API_KEY"]

THRESHOLD_RECALL = 0.85   # fraction of golden queries whose expected path is retrieved
THRESHOLD_MRR = 0.60

def hybrid_search(query, tenant_id, k=20, candidate_k=150):
    qvec = requests.post(
        "https://api.voyageai.com/v1/embeddings",
        headers={"Authorization": "Bearer " + VOYAGE_KEY},
        json={"input": [query], "model": "voyage-4-lite",
              "input_type": "query", "dimensions": 1024},
        timeout=60,
    )
    qvec.raise_for_status()
    vec = qvec.json()["data"][0]["embedding"]
    body = {
        "prefetch": [
            {"query": {"text": query}, "using": "text",
             "filter": {"must": [{"key": "tenant_id",
                                  "match": {"value": tenant_id}}]},
             "limit": candidate_k},
            {"query": vec, "using": "dense",
             "filter": {"must": [{"key": "tenant_id",
                                  "match": {"value": tenant_id}}]},
             "limit": candidate_k},
        ],
        "query": {"rrf": {}},
        "limit": k,
        "with_payload": True,
    }
    r = requests.post(
        QDRANT_URL + "/collections/" + COLLECTION + "/points/query",
        headers=HEADERS, json=body, timeout=30,
    )
    r.raise_for_status()
    return r.json()["result"]["points"]

@discrete_metric(name="answer_gate", allowed_values=["pass", "fail"])
def answer_gate(response: str, expected_answer: str):
    ok = expected_answer.strip().lower() in response.lower()
    return MetricResult(value="pass" if ok else "fail",
                       reason="expected answer present: " + str(ok))

@experiment()
async def eval_retrieval(row):
    points = hybrid_search(row["query"], row["tenant_id"])
    paths = [p["payload"]["source_path"] for p in points]
    hits = [i for i, p in enumerate(paths) if p in row["expected_paths"]]
    recall_hit = 1.0 if hits else 0.0
    rr = 1.0 / (hits[0] + 1) if hits else 0.0
    gate = answer_gate.score(response=" ".join(paths),
                             expected_answer=row.get("expected_answer", ""))
    return {
        **row,
        "retrieved_paths": paths[:5],
        "recall_hit": recall_hit,
        "reciprocal_rank": rr,
        "answer_gate": gate.value,
    }

def main():
    dataset = Dataset(name="golden", backend="local/jsonl", root_dir=".")
    rows = [json.loads(l) for l in open("golden.jsonl") if l.strip()]
    recalls, rrs, gates = [], [], []
    exp_rows = []
    for row in rows:
        # run the async experiment function synchronously for CI simplicity
        import asyncio
        result = asyncio.run(eval_retrieval(row))
        exp_rows.append(result)
        recalls.append(result["recall_hit"])
        rrs.append(result["reciprocal_rank"])
        gates.append(1.0 if result["answer_gate"] == "pass" else 0.0)

    recall = sum(recalls) / len(recalls)
    mrr = sum(rrs) / len(rrs)
    gate_rate = sum(gates) / len(gates)
    print("golden set size:", len(rows))
    print("recall@20:      %.3f (threshold %.2f)" % (recall, THRESHOLD_RECALL))
    print("MRR:            %.3f (threshold %.2f)" % (mrr, THRESHOLD_MRR))
    print("answer gate:     %.3f" % gate_rate)

    failed = recall < THRESHOLD_RECALL or mrr < THRESHOLD_MRR
    if failed:
        print("EVAL REGRESSION — failing the build")
        for r in sorted(exp_rows, key=lambda x: x["reciprocal_rank"])[:10]:
            print("  MISS:", r["query"], "->", r["retrieved_paths"])
    sys.exit(1 if failed else 0)

if __name__ == "__main__":
    main()

7.3 Layer two: LLM-judged generation metrics on a schedule

The deterministic layer cannot tell you whether answers are faithful to retrieved context. That is the LLM-judge layer, and RAGAS 0.4.3's metric set is the current standard vocabulary, all documented in its metrics reference:

• Faithfulness — decomposes the generated answer into individual statements, verifies each statement against the retrieved context, and scores supported/total. RAGAS's own worked example: an answer with two statements, one inferable from context, scores 0.5. This is your hallucination canary.
• Context precision / context recall — whether the retrieved chunks that actually mattered were ranked high, and whether all needed context was retrieved. Context recall needs your reference answers.
• Context entities recall and noise sensitivity — entity-level coverage and robustness to irrelevant retrieved chunks.
• Response relevancy — does the answer actually address the question.
• Agent metrics (tool call accuracy, agent goal accuracy, topic adherence) — for the agentic retrieval path from section 6.3.

Run judge metrics nightly, not per-commit: they are nondeterministic (pin the judge model and temperature), they cost a generation call per statement verified, and their trend matters more than any single run. Two judge pitfalls worth engineering around: judge models are biased toward answers phrased like their own training distribution (prefer a judge from a different family than your generator), and a judge evaluating an answer without the retrieved context is grading fiction — always pass the exact context you passed the generator.

7.4 What evaluation actually gates

Wire the thresholds so that every ingestion run recomputes retrieval metrics (corpus changes move recall), every config change to hnsw/quantization reruns them (quality/latency trades are claims until measured), and every embedding-model change re-runs the full suite including judge metrics (section 4.2's migration rule). The CronJob in section 8.5 runs the harness; the gate belongs in CI so a regression is a red build, not a Slack message someone reads next week.

8. The Complete Kubernetes Deployment

Everything below is deployable as-is: a Qdrant StatefulSet on v1.20.0, the retrieval API with autoscaling, the ingestion Job that bootstraps and loads the collection, and the evaluation CronJob that closes the loop.

8.1 Qdrant StatefulSet

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: qdrant
  labels: { app: qdrant }
spec:
  serviceName: qdrant
  replicas: 1            # production: 3 replicas, replication_factor 2, sharding on
  selector:
    matchLabels: { app: qdrant }
  template:
    metadata:
      labels: { app: qdrant }
    spec:
      containers:
        - name: qdrant
          image: qdrant/qdrant:v1.20.0
          ports:
            - containerPort: 6333
              name: http
            - containerPort: 6334
              name: grpc
          env:
            - name: QDRANT__SERVICE__API_KEY
              valueFrom:
                secretKeyRef:
                  name: qdrant-secrets
                  key: api-key
          resources:
            requests: { cpu: "1", memory: 4Gi }
            limits:   { cpu: "4", memory: 16Gi }
          readinessProbe:
            tcpSocket: { port: 6333 }
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            tcpSocket: { port: 6333 }
            initialDelaySeconds: 15
            periodSeconds: 20
          volumeMounts:
            - name: storage
              mountPath: /qdrant/storage
  volumeClaimTemplates:
    - metadata:
        name: storage
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 100Gi
---
apiVersion: v1
kind: Service
metadata:
  name: qdrant
spec:
  clusterIP: None        # headless for StatefulSet identity
  selector: { app: qdrant }
  ports:
    - { name: http, port: 6333, targetPort: 6333 }
    - { name: grpc, port: 6334, targetPort: 6334 }

Configuration note: Qdrant reads environment variables with the QDRANT__ prefix and double-underscore nesting, and env always overrides the config file — that is the documented mechanism used above for the API key. On storage sizing, use the section 9 numbers plus headroom for the HNSW graph and payload indexes; v1.20's inline_storage and quantization are the levers when the PVC fills before the budget does.

8.2 The retrieval API

#!/usr/bin/env python3
"""Retrieval API: hybrid dense+BM25 retrieval with RRF and rerank-3.

Env: QDRANT_URL, QDRANT_API_KEY, VOYAGE_API_KEY
Probes: GET /healthz returns 200 when Qdrant answers.
"""

import os
import requests
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

app = FastAPI(title="retrieval-api")

QDRANT_URL = os.environ["QDRANT_URL"]
COLLECTION = "corpus_chunks"
HEADERS = {"api-key": os.environ.get("QDRANT_API_KEY", "")}
VOYAGE_KEY = os.environ["VOYAGE_API_KEY"]
CANDIDATES = 150
TOP_K = 20
RERANK_THRESHOLD = 0.3


class SearchRequest(BaseModel):
    query: str
    tenant_id: str
    top_k: int = TOP_K


def embed_query(text):
    r = requests.post(
        "https://api.voyageai.com/v1/embeddings",
        headers={"Authorization": "Bearer " + VOYAGE_KEY},
        json={"input": [text], "model": "voyage-4-lite",
              "input_type": "query", "dimensions": 1024},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["data"][0]["embedding"]


def hybrid_candidates(query, vec, tenant_id):
    body = {
        "prefetch": [
            {"query": {"text": query}, "using": "text",
             "filter": {"must": [{"key": "tenant_id",
                                  "match": {"value": tenant_id}}]},
             "limit": CANDIDATES},
            {"query": vec, "using": "dense",
             "filter": {"must": [{"key": "tenant_id",
                                  "match": {"value": tenant_id}}]},
             "limit": CANDIDATES},
        ],
        "query": {"rrf": {}},
        "limit": CANDIDATES,
        "with_payload": True,
    }
    r = requests.post(
        QDRANT_URL + "/collections/" + COLLECTION + "/points/query",
        headers=HEADERS, json=body, timeout=30,
    )
    r.raise_for_status()
    return r.json()["result"]["points"]


def rerank(query, candidates):
    docs = [c["payload"]["text"] for c in candidates]
    r = requests.post(
        "https://api.voyageai.com/v1/rerank",
        headers={"Authorization": "Bearer " + VOYAGE_KEY},
        json={"query": query, "documents": docs, "model": "rerank-3",
              "top_k": len(docs)},
        timeout=60,
    )
    r.raise_for_status()
    return r.json()["results"]


@app.get("/healthz")
def healthz():
    r = requests.get(QDRANT_URL + "/collections", headers=HEADERS, timeout=5)
    return {"status": "ok", "qdrant": r.status_code}


@app.post("/search")
def search(req: SearchRequest):
    vec = embed_query(req.query)
    candidates = hybrid_candidates(req.query, vec, req.tenant_id)
    if not candidates:
        raise HTTPException(404, "no candidates retrieved")
    ranked = rerank(req.query, candidates)
    results = []
    for item in ranked:
        if item["relevance_score"] < RERANK_THRESHOLD:
            continue
        c = candidates[item["index"]]
        results.append({
            "text": c["payload"]["text"],
            "source_path": c["payload"]["source_path"],
            "chunk_pos": c["payload"]["chunk_pos"],
            "score": item["relevance_score"],
        })
        if len(results) >= req.top_k:
            break
    return {"results": results,
            "answerable": bool(results)}

apiVersion: apps/v1
kind: Deployment
metadata:
  name: retrieval-api
spec:
  replicas: 2
  selector:
    matchLabels: { app: retrieval-api }
  template:
    metadata:
      labels: { app: retrieval-api }
    spec:
      containers:
        - name: api
          image: registry.internal/platform/retrieval-api:1.0.0
          ports: [{ containerPort: 8000 }]
          env:
            - name: QDRANT_URL
              value: "http://qdrant.default.svc.cluster.local:6333"
            - name: QDRANT_API_KEY
              valueFrom:
                secretKeyRef: { name: qdrant-secrets, key: api-key }
            - name: VOYAGE_API_KEY
              valueFrom:
                secretKeyRef: { name: voyage-secrets, key: api-key }
          readinessProbe:
            httpGet: { path: /healthz, port: 8000 }
            initialDelaySeconds: 3
            periodSeconds: 10
          resources:
            requests: { cpu: 250m, memory: 256Mi }
            limits:   { cpu: "1", memory: 512Mi }
---
apiVersion: v1
kind: Service
metadata:
  name: retrieval-api
spec:
  selector: { app: retrieval-api }
  ports: [{ port: 80, targetPort: 8000 }]
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: retrieval-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: retrieval-api
  minReplicas: 2
  maxReplicas: 12
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 65 }

8.3 Ingestion Job — bootstrap and load

apiVersion: batch/v1
kind: Job
metadata:
  name: corpus-ingest
spec:
  backoffLimit: 2
  ttlSecondsAfterFinished: 86400
  template:
    spec:
      restartPolicy: Never
      initContainers:
        - name: bootstrap-collection
          image: curlimages/curl:8.11.0
          env:
            - name: QDRANT_URL
              value: "http://qdrant.default.svc.cluster.local:6333"
            - name: QDRANT_API_KEY
              valueFrom:
                secretKeyRef: { name: qdrant-secrets, key: api-key }
          command: ["sh", "-c"]
          args:
            - |
              set -e
              # collection (idempotent: tolerate 409 already-exists)
              curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks" \
                -H "api-key: $QDRANT_API_KEY" -H "content-type: application/json" \
                -d @/cfg/collection.json || true
              # payload indexes: tenant, provenance, BM25 full-text
              curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index" \
                -H "api-key: $QDRANT_API_KEY" -H "content-type: application/json" \
                -d '{"field_name": "tenant_id",
                     "field_schema": {"type": "keyword", "is_tenant": true}}'
              curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index" \
                -H "api-key: $QDRANT_API_KEY" -H "content-type: application/json" \
                -d '{"field_name": "source_path", "field_schema": {"type": "keyword"}}'
              curl -s -X PUT "$QDRANT_URL/collections/corpus_chunks/index" \
                -H "api-key: $QDRANT_API_KEY" -H "content-type: application/json" \
                -d '{"field_name": "text", "field_schema": {"type": "text", "tokenizer": "word"}}'
              echo "collection ready"
          volumeMounts:
            - { name: cfg, mountPath: /cfg }
      containers:
        - name: loader
          image: registry.internal/platform/retrieval-ingest:1.0.0
          env:
            - name: QDRANT_URL
              value: "http://qdrant.default.svc.cluster.local:6333"
            - name: QDRANT_API_KEY
              valueFrom:
                secretKeyRef: { name: qdrant-secrets, key: api-key }
            - name: VOYAGE_API_KEY
              valueFrom:
                secretKeyRef: { name: voyage-secrets, key: api-key }
            - name: CORPUS_DIR
              value: /data/corpus
            - name: TENANT
              value: "acme-prod"
          volumeMounts:
            - { name: corpus, mountPath: /data/corpus }
      volumes:
        - name: corpus
          persistentVolumeClaim: { claimName: corpus-pvc }
        - name: cfg
          configMap: { name: qdrant-collection-config }

The collection.json ConfigMap carries the exact JSON from section 5.2. The bootstrap is idempotent by design — Qdrant returns 409 on an existing collection name, which the || true tolerates on re-runs, while the payload-index PUTs are safe to repeat. The strict-mode note from section 5 applies to the search path, not to index creation.

8.4 The evaluation CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: rag-eval
spec:
  schedule: "0 6 * * *"
  concurrencyPolicy: Forbid
  jobTemplate:
    spec:
      backoffLimit: 0        # a regression should page, not retry
      template:
        spec:
          restartPolicy: Never
          containers:
            - name: eval
              image: registry.internal/platform/retrieval-eval:1.0.0
              env:
                - name: QDRANT_URL
                  value: "http://qdrant.default.svc.cluster.local:6333"
                - name: QDRANT_API_KEY
                  valueFrom:
                    secretKeyRef: { name: qdrant-secrets, key: api-key }
                - name: VOYAGE_API_KEY
                  valueFrom:
                    secretKeyRef: { name: voyage-secrets, key: api-key }
              volumeMounts:
                - { name: golden, mountPath: /work/golden }
              command: ["python", "/work/eval_retrieval.py"]
              workingDir: /work/golden
          volumes:
            - name: golden
              configMap: { name: rag-golden-set }

With backoffLimit: 0, a threshold breach turns the CronJob red immediately and your alerting picks it up from the job's failed status — the point of the evaluation plane is to make regressions impossible to miss, not to retry them at 6:05 a.m.

9. The Cost Model, With Real Numbers

Every figure below was computed with exact decimal arithmetic from the vendors' published rate cards on November 10, 2026. The scenario: a 500M-token corpus, 800-token chunks (625,000 chunks), retrieving 150 candidates, reranking, and passing 20 chunks (16,350 input tokens) to the generator with a 400-token answer.

9.1 One-time indexing costs

The embedding bill is noise. The contextualization bill is 8–85× the embedding bill depending on model, and it recurs on every re-chunk. That asymmetry — not the per-token price of embeddings — is the real indexing economics of 2026.

9.2 Vector storage (625k chunks, pgvector formulas)

Add HNSW graph overhead on top of raw vector bytes (measure with pg_relation_size on pgvector, or Qdrant's collection info API) and payload storage for the chunk text. The dimension decision from section 4.1 is what this table is telling you: choosing a 3072-dimension default triples your memory footprint versus the 1024-dimension 2026 consensus, and binary quantization plus re-scoring gets you two orders of magnitude smaller at some recall cost that only your eval set can quantify.

9.3 Per-query costs

Storage and vector search are infrastructure costs, not per-token costs — the per-query bill above is entirely embedding, rerank, and generation, and generation dominates it. Note what rerank costs relative to what it buys: $0.0064 per query for the rerank-3 leg that (per Anthropic's ladder) takes a 49% failure rate down to under 2% when combined with hybrid retrieval.

9.4 The honest crossover: when stuffing beats RAG on cost

Here is the part most RAG guides skip. With 2026 prompt caching, the "just put the whole corpus in the context window" strategy has a real cost case for small corpora. OpenAI's caching charges 1.25× the ordinary input rate to write a 30-minute cache entry (2.5× for 12 hours) and 0.1× for subsequent reads on most current models; Anthropic mirrors it with 1.25× for 5-minute entries, 2× for 1-hour, and reads at roughly a tenth of base input (Claude Haiku 4.5: $0.10/MTok reads against $1.00/MTok base input).

Run the numbers on a 250k-token corpus with gpt-5.4 (input $2.50/1M, cached reads $0.25/1M):

The crossover arithmetic: at gpt-5.4's cached read rate, pure cached-read stuffing matches the full RAG stack at roughly 225k tokens of corpus. Below that, with a stable corpus, repeated identical prefixes, and enough query volume to amortize cache writes, stuffing is genuinely cheaper — on gemini-3.8-flash it wins outright even against the cheapest RAG configuration. Above it, or the moment your corpus changes faster than the cache TTL, RAG pulls ahead fast: uncached stuffing at 250k tokens costs $0.625 per query, 15× the RAG stack.

So the honest 2026 decision rule: tiny stable corpus and cache-friendly traffic — try cached stuffing first and measure; everything else — multi-tenant corpora, fresh content, cost sensitivity at scale, sub-second latency — RAG still wins, and the margin widens with every token of corpus growth. What caching actually killed was not RAG; it was the claim that RAG is always the cheaper option.

10. Production Realities: Where This Pattern Bites

Everything below is a failure mode with a verified source, not folklore. Budget for these before your first production tenant.

• Tenancy is not access control. Qdrant's docs could not be more explicit: the is_tenant index helps storage and search, "it does not authorize requests," and a client with unrestricted credentials can simply omit the filter. Your retrieval API must derive tenant_id from the authenticated identity server-side (as the retrieval API in 8.2 requires it per request but your gateway must inject it), never from the client. For defense in depth, Qdrant's granular access keys can scope credentials per collection.
• Adding a payload index does not retro-fit the HNSW graph. The graph was built without filter-aware links. Qdrant's documented remedy: after creating the payload index, force a rebuild by bumping ef_construct by 1 — the config-mismatch optimizer re-indexes all segments — and do not immediately revert the value. Teams that skip this step wonder why their shiny is_tenant index did not speed anything up.
• BM25 relevance statistics are per-collection by default. A tenant whose corpus is all Kubernetes YAML skews IDF for everyone else sharing the collection. Qdrant 1.20 supports scoring statistics per tenant with the idf search parameter — enable it for multi-tenant collections where lexical relevance matters.
• Embedding model changes are reindex events. Section 4.2 bears repeating in the failure list because it is the most commonly misplanned: there is no vector migration, only re-embedding. With contextualization the bill recurs ($510 per 500M tokens, at Anthropic's verified rate), and there is a quality-regression window until the golden set re-validates.
• pgvector index builds are scheduled events. The HNSW graph stops fitting in maintenance_work_mem after roughly 100k tuples (with a server NOTICE), builds are faster after the initial data load, and the README's guidance is to leave headroom rather than max out server memory. A 625k-chunk index build is a change-window task with a memory plan, not something you run on a busy primary.
• Cold-tier latency lives in the tail. Qdrant's own operations guide flags the unquantized cold tier as "the configuration to avoid" for latency-sensitive workloads — most queries are fine until a rare HNSW hop lands on an uncached page and pays the full disk read. If you must go to disk, go with a pinned quantized copy (int8/TurboQuant), which is exactly what the pinned-quantized configuration in 5.3 exists for.
• Prompt-cache wins are prefix-fragile. Anthropic's cache matching is a cumulative prefix hash: change any block at or before the breakpoint and the entry misses. A corpus that updates hourly under a 5-minute or 30-minute TTL means most reads are cache misses at full input price — the 9.4 crossover table flips back toward RAG immediately. Cached stuffing is a stable-corpus strategy, not a fresh-content strategy.
• Golden sets rot. The corpus drifts, the queries your users ask drift, and a fixed golden set slowly becomes a test of yesterday's product. Recruit new golden examples from production queries monthly, and treat a recall improvement you cannot explain as a bug in the harness before celebrating it.

11. Who Should Skip This Pattern

Integrity section, as always. This pipeline is overkill or wrong for several real situations:

• Your corpus is small, stable, and heavily re-queried (< ~225k tokens). Section 9.4's math is not a fluke: with 2026 prompt-cache pricing, cached stuffing is competitive or cheaper, and it is one API call with zero stateful infrastructure. Measure both for a week before committing to a vector store, a reranker, and an eval plane.
• You cannot staff the evaluation plane. If nobody will own a golden set and CI thresholds, you are not building a retrieval pipeline — you are deploying a search engine that decays silently. Buy a managed RAG product with its own evals instead, and put the on-call you saved elsewhere.
• You have no on-call for stateful services. Self-hosted Qdrant is a distributed Rust service with shards, snapshots, memory tiers, and optimizer knobs. If your platform does not already run stateful workloads competently, use Qdrant Cloud or Pinecone and skip the StatefulSet entirely.
• Your p99 requirement is under ~100 ms end-to-end. This pattern's latency floor is embedding-API RTT plus fusion plus rerank inference — realistically hundreds of milliseconds. Synchronous semantic search at strict SLAs needs the rerank step removed and in-cluster embeddings, which is a different architecture than the one described here.
• You expect RAG to fix a bad corpus. Duplicated runbooks, five versions of every document, no ownership: retrieval amplifies corpus chaos, it does not resolve it. Fix the corpus first; otherwise you will spend the quarter tuning ef_construction to compensate for three conflicting copies of the runbook.

12. References & Further Reading

• Qdrant v1.20.0 release notes — native BM25 on the payload index, TurboQuant, HNSW layer-assignment fix, inline storage (October 22, 2026)
• Qdrant indexing documentation — HNSW defaults (m 16, ef_construct 100), payload index types, strict mode, the ef_construct-bump rebuild procedure
• Qdrant hybrid queries — Query API prefetch shape, RRF formula and k, DBSF normalization behavior
• Qdrant multitenancy — is_tenant payloads, per-tenant HNSW (m 0 + payload_m), tiered sharding, per-tenant scoring statistics
• Qdrant memory tiers — pinned/cached/cold trade-offs and the pinned-quantized recommendation
• Qdrant configuration — the QDRANT__ environment-variable convention used in the StatefulSet
• pgvector README (v0.8.x) — HNSW parameters, iterative scans, halfvec/binary quantization, storage byte formulas, build-time guidance
• Anthropic — Contextual Retrieval — the 35%/49%/67% retrieval-failure ladder and the $1.02/1M-token contextualization cost (September 2024)
• Anthropic — Prompt caching — 5-minute/1-hour TTL pricing, prefix-hash matching, backward-walk cache reads
• OpenAI — Prompt caching guide — up to 95% cached-input discount, 1.25×/2.5× write rates, 30-minute/12-hour lifetimes
• OpenAI API pricing — embedding and generation rates used in the cost model (fetched November 10, 2026)
• Voyage AI embeddings documentation — voyage-4 family dimensions, context lengths, the open-weight voyage-4-nano
• Voyage AI pricing — embedding and rerank-3 rates, 200M free-token tiers, per-request rerank estimates
• Cohere — Embed 5 announcement — shared Pro/Fast embedding space, 128k context, Matryoshka dimensions (September 30, 2026)
• Cohere — Rerank 5 documentation — 32k context, calibrated scores, Context Focus relevant_spans
• RAGAS metrics reference — faithfulness, context precision/recall, noise sensitivity, agent metrics (v0.4.x)
• Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models — Günther, Mohr, Williams (arXiv 2409.04701)
• Gemini API pricing — Gemini 3.8 Flash and Gemini 4 Argon rates including context-caching prices