Loki vs. Elasticsearch in 2026: Object-Store Log Economics Measured on Real Disk
Sources
- Loki v3.7.8 release (Sep 17, 2026) — security fixes to grpc, containerd, x/net, x/crypto
- Loki v3.7.0 release — new query engine scheduler capacity changes, dataobj partitions
- Elasticsearch 9.5.4 release (Sep 15, 2026) — current stable
- Elasticsearch 9.5.0 release notes — columnar and logsdb_columnar index modes (Tech Preview)
- Elastic docs — Logs data streams: logsdb enabled by default in Elastic Stack 9.0+, up to 60% storage reduction, 10-20% indexing impact
- Elastic logsdb GA announcement — Dec 2024, up to 65% storage reduction vs non-logsdb
- Loki 3.7 upgrade notes — BREAKING CHANGES: scheduler compute capacity, shared worker threads, parsed labels vs structured metadata
- Grafana Loki Helm chart index — chart 7.x series, appVersion 3.6.x (Sep 2026)
September 19, 2026. Loki 3.7.8 landed on September 17 — a security-patch release (gRPC, containerd, x/crypto CVE bumps, no fanfare). Elasticsearch 9.5.4 shipped two days earlier on September 15. Neither event made headlines, which is exactly why this comparison needed redoing: the last time most teams seriously evaluated Loki against Elasticsearch, Elastic had nothing like Loki’s object-store economics, and Loki had nothing like Elasticsearch’s query engine. In 2026 both halves of that sentence are wrong, and almost nobody has re-run the numbers.
So we did. We generated a realistic 300,000-line Kubernetes-style log corpus (40.9 MB of raw log-line payload: API access logs across 4 namespaces, 9 pods, INFO/WARN/ERROR levels, HTTP routes, status codes, latencies, plus the kind of operational chatter real pods emit), pushed the identical corpus into a single-binary Loki 3.7.8 (TSDB schema, filesystem object store) and a single-node Elasticsearch 9.5.4, and measured what actually hit disk. Then we repeated the Elasticsearch run in its brand-new logsdb_columnar index mode — the Tech Preview Elastic shipped in 9.5.0 that turns the “Loki is cheaper” argument into a live engineering question.
The headline: on identical data, Loki stored the corpus in 10.1 MB (4.1× compression), standard logsdb Elasticsearch took 52.6 MB (129% of raw), and logsdb_columnar took 7.9 MB with keyword-mapped message fields — but only 32.4 MB (79% of raw) once we mapped the message field as text so phrase search actually works. Every number in this guide comes from the bench we ran; the exact methodology is below so you can reproduce it.
TL;DR — The Verdict First
- Log volume is your primary cost, and the storage profiles have converged — but not the query engines. Loki’s TSDB+chunk pipeline compressed our corpus 4.1×. Elasticsearch’s default logsdb mode stored 129% of raw bytes. logsdb_columnar with an all-keyword schema got to 5.2× — nominally smaller than Loki — but the moment you give the message field an inverted index (so full-text search works), it lands at 79% of raw: still 3.2× bigger than Loki.
- Query latency is the mirror image. On a needle-in-haystack phrase search across 300k lines, Elasticsearch answered in 35 ms where Loki took 280 ms — 8× faster on a single-node toy. That gap is structural: Elasticsearch reads an inverted index, Loki greps compressed chunk data. At petabyte scale Loki’s parallelism closes some of the distance, but you should never adopt Loki expecting Elasticsearch-class ad-hoc search latency.
- Memory is the hidden Elasticsearch line item. Our single-node bench Elasticsearch settled at 2.0 GB RSS with a 2 GB heap. Loki settled at 0.12 GB RSS doing the same ingestion work. That 16× ratio is why “just use Elasticsearch for logs” fleets keep growing JVM heaps faster than their log volume grows.
- Both stacks now require explicit tuning for real log volume. Out of the box, Loki rejected our pushes at 4 MB/s (default
ingestion_rate_mb) and we had to raise it to 64. Elasticsearch 9’s built-inlogscomposable template hijacks anylogs-*index name into a data stream — our first index-creation attempt against that pattern failed withuse create data stream api instead. Neither tool’s defaults are honest about production log volume; both make you opt into the real configuration. - Who should skip this entire debate: if your log volume is under a few GB/day, both systems are overkill — run Vector or Promtail to files, or pay for a SaaS, and spend the saved platform-team quarter on something that touches revenue. And if you already run Elasticsearch for security analytics or search, the question isn’t “Loki or Elasticsearch,” it’s “logsdb or logsdb_columnar,” which is a much cheaper migration.
Why This Comparison Keeps Being Done Wrong
Most Loki-vs-Elasticsearch content on the internet compares two things that were never peers:
- Old Elasticsearch defaults. Elastic shipped
logsdbin 8.17 (GA December 2024) and made it the default for new logs data streams from Stack 9.0 onward. It cuts storage “by up to 65%” per Elastic’s benchmarks. Any comparison still modeling 2022-era Elasticsearch storage is comparing against a product that no longer exists. - Loki without its labels discipline. Loki’s compression assumes you feed it a small number of high-volume streams. Hand it unbounded labels (say, a per-request
trace_idlabel) and it shreds into millions of tiny streams, chunks stop compressing, and the TSDB index balloons. The 4.1× number below is only real because we held streams to 36.
This guide pins both sides to current versions — Loki 3.7.8 (the September 17 patch, which fixed a chain of CVEs in the gRPC push path) and Elasticsearch 9.5.4 (the September 15 stable) — and measures what each actually puts on disk, in RAM, and behind an HTTP query endpoint.
The Benchmark, Exactly As We Ran It
Host: 4 vCPU, 14 GB RAM, Ubuntu, no docker. Both systems ran as plain processes, single node, on the same filesystem. That matters: this is a storage-format and query-engine comparison, not a scale-out one. At the end we extrapolate to fleet scale with the same ratios.
Corpus profile (identical bytes into both systems):
300,000 lines / 40.9 MB raw log-line payload
window: 30 minutes of wall-clock-style timestamps
streams: 36 (4 namespaces x 9 pods x 3 levels, approx.)
structure: "2026-09-19T02:03:06.000Z INFO [ns=prod-app pod=api-7d9f4b-9p1zz
instance=api-538] GET /api/v1/users 200 738ms bytes=1643"
extras: ~30% of lines carry one of 10 operational messages
(upstream latency budget exceeded, circuit breaker half-open, ...)
Loki ingested the corpus at 101,731 lines/second (2.9 s wall) via the JSON push API, after we raised its default ingestion limits. Elasticsearch ingested at 20,511 docs/s in logsdb mode (14.6 s) and 26,679 docs/s in logsdb_columnar (11.2 s) via bulk API. Those ingestion numbers are not apples-to-apples (different push paths, different batching), so treat them as “neither system is ingestion-bound on a single node” and move on to what actually differs: what lands on disk.
Result 1: What Hit the Disk
After Loki’s flush and Elasticsearch’s refresh, we measured on-disk storage with du (Loki) and the _stats/store API (Elasticsearch), so each number is what its own system reports:
Measured storage on identical 40.9 MB of raw log lines:
SYSTEM ON-DISK % OF RAW VS RAW ───────────────────────────────────────────────────────────────────── Loki 3.7.8 (tsdb + chunks) 10.1 MB 24.6% 4.06x smaller ES 9.5.4 index.mode: logsdb 52.6 MB 128.7% 1.29x BIGGER ES 9.5.4 logsdb_columnar, all keyword 7.9 MB 19.3% 5.19x smaller ES 9.5.4 logsdb_columnar + message:text 32.4 MB 79.4% 1.26x smaller the trap: logsdb_columnar's headline number collapses the moment you need phrase search on the message body. 7.9 MB -> 32.4 MB (4.1x) just by mapping message as text instead of keyword.
Read that table twice. The first reading is the one that ends up in vendor slide decks: logsdb_columnar beats Loki on raw compression. The second reading is the one that survives a production week: that number is only real if you never full-text search your logs.
Here is the mechanism, and it matters for any decision:
- Standard logsdb mode is genuinely expensive for log-shaped data. 128.7% of raw means Elasticsearch stored more than the original text — the inverted index, doc values, and stored source together outweigh Zstandard compression of the payload. Elastic’s own logsdb GA numbers (“up to 65% smaller”) are measured against older non-logsdb Elasticsearch, not against raw input; our logsdb-vs-raw result is consistent with that: logsdb is dramatically better than standard ES indexes, but it is still far from free.
- logsdb_columnar’s 85% saving comes from deleting the machinery you use. In columnar mode, per the 9.5.0 release notes: fields are stored once as doc values only — no inverted index or BKD tree is created by default; text fields are “no longer mapped by default”; the original document source is not stored, but synthesized from doc values at query time; mappings are auto-flattened. When our un-mapped
messagefield arrived, columnar mode silently mapped it askeyword— andmatch_phraseon it returned 0 hits where standard mode returned 8,981. That is not a bug; it is the entire storage bet. You trade query capability for bytes. - Restoring full-text search restores most of the cost. With
messageexplicitly mapped astext, phrase search returned the same 8,981 hits — and storage jumped from 7.9 MB to 32.4 MB. Even so, 79.4% of raw in a system that can do sub-second phrase search is a serious engineering achievement. It is just not the same product as the 19.3% headline. - Loki’s 24.6% of raw needs no asterisks at query time. LogQL line filters (
|= "latency budget") grep the decompressed chunk data directly — they work identically on any line, with no mapping decisions made at ingest. You pay for that with the linear-scan latency below.
Result 2: Query Latency — The Bill Comes Due
Storage is only half the economics. The same corpus, the same needle (“latency budget”, present in 8,981 lines), four query shapes:
Single-node query latency, 300k lines, same needle search:
QUERY LOKI 3.7.8 ES 9.5.4 (logsdb) ES 9.5.4 (columnar+text) ──────────────────────────────────────────────────────────────────────────────────────── needle phrase, 100 results 280 ms 35 ms 35 ms count lines with level=ERROR 438 ms 34 ms 34 ms both ES numbers are inverted-index lookups: find term -> read postings. both Loki numbers are: find chunks by label+time -> download -> decompress -> line-filter every entry in the window.
The 8× gap is structural and no tuning removes it:
- Elasticsearch pays at write time to be fast at read time. The inverted index that costs 129% of raw in logsdb mode is exactly what answers in 35 ms. You are not choosing between “cheap and fast” and “expensive and slow” — you are choosing where you pay: at write (Elasticsearch) or at read (Loki).
- Loki’s query path is embarrassingly parallel, which is its real answer. A 300k-line bench is the worst case for Loki — one querier, one node, no fan-out. At terabyte scale, the pattern inverts: label-selector queries shard across hundreds of queriers reading object storage in parallel, while a single Elasticsearch cluster fights JVM heap, shard count, and disk IOPS. But if your log platform serves interactive human search (debugging, incident forensics, exploratory hunting), single-query latency is the metric that decides user satisfaction, and Elasticsearch wins it.
- logsdb_columnar keeps the read side fast where it counts. Our columnar+text run answered the phrase query in the same 35 ms — the inverted index for explicitly-mapped text fields still exists; it is the default mappings that changed. Doc-value skippers (skip lists with min/max metadata, on by default for numeric fields in 9.5) let range aggregations skip whole blocks without scanning values — that is Loki-style columnar thinking, inside Elasticsearch.
Result 3: The Memory Nobody Budgets
After ingestion settled, we took RSS from the process table:
Steady-state resident memory, same corpus, single node:
Loki 3.7.8 single binary 0.12 GB RSS (Go, no heap tuning) Elasticsearch 9.5.4 node 2.01 GB RSS (JVM -Xms2g -Xmx2g) ratio: ~16x. And the ES number is the FLOOR, not the ceiling: a 2 GB heap is the smallest heap anyone runs in production.
This is the number that quietly decides fleet costs. A production Elasticsearch logging node typically runs 16–31 GB of heap (crossing the compressed-oops boundary at 32 GB is its own folklore), plus page cache the JVM leans on for segment reads. Heap sizing for logs is an unending argument because log indexing is garbage-generation-heavy. Loki’s Go runtime did the same ingestion work in 120 MB — not because Go is magic, but because Loki’s design (append chunks to object storage, index only labels) does not hold a query engine’s worth of state in memory.
The honest extrapolation for capacity planning: for the same log volume, plan roughly an order of magnitude more memory for Elasticsearch than Loki — and remember you get query latency in exchange. The mistake is paying that memory bill and treating the platform as if it had Loki’s economics.
The Cost Model, Extrapolated
Single-node disk numbers become fleet decisions through retention math. Using our measured storage ratios and a $0.023/GB-month object-storage price (S3 Standard-class ballpark; your tiering will differ), here is 90 days of 1 TB/day raw log retention, per month:
Object-storage cost of 90-day retention at 1 TB/day raw, $0.023/GB-month:
raw payload (never happens): $2,070 /month Loki 3.7.8 (24.6% of raw): $509 /month ES 9.5.4 logsdb (128.7% of raw): $2,664 /month <-- 5.2x Loki ES 9.5.4 columnar+text (79.4%): $1,643 /month <-- 3.2x Loki notes: excludes compute (see the 16x memory ratio), request costs, and cross-AZ replication. Those make ES look worse, not better.
Three caveats we will not hide behind:
- Our corpus is one shape. High-entropy log lines (stack traces, hex dumps, JSON blobs with random IDs) compress worse for everyone; repetitive structured access logs compress better for everyone. We ran one corpus, not a sweep — the ratios will move with your data, and the direction of the gaps will not.
- Elasticsearch’s S3-backed snapshot tiering changes the math. If you snapshot warm indices to searchable-snapshot tiers on S3, the hot-disk multiplier applies to a shrinking recent window, not 90 days. The gap narrows. It does not close, because the search tier still keeps 100% overhead structures on every byte.
- Loki’s number assumes label discipline. The moment someone labels by
trace_idoruser_id, stream count explodes, chunks fragment, and compression falls off a cliff. This is not a hypothetical — unbounded labels are the single most common way Loki deployments fail, and the reason every Loki runbook starts with cardinality budgets.
Architecture: Where the Bytes Actually Come From
The storage gap is not a tuning artifact — it is two different beliefs about what a log store is. Trace one log line through each system:
LOKI 3.7.8 ELASTICSEARCH 9.5.4
────────── ──────────────────
push → distributor → ingester bulk → indexing buffer
label set = stream identity each doc gets:
line → appended to in-memory chunk • inverted index terms
per (stream, time window) • doc values (columnar)
chunk flush → compressed blob → object store • stored _source (JSON)
(logsdb: synthetic)
index: TSDB labels + chunk pointers only • logsdb_columnar:
query: pick streams by label+time → doc values ONLY
fetch chunk objects → decompress → + optional inverted
line-filter (grep) the window index for text fields
+ no stored source
storage = compressed raw lines + tiny label index storage = index + columns
query = parallel grep over object storage query = term lookups +
(fast, heap-resident)
Two consequences fall out of that picture:
- Loki’s index is small because it indexes the streams, not the lines. TSDB stores label-set → chunk-pointer mappings. The bytes of your log lines are never parsed, tokenized, or double-stored. This is why Loki’s disk bill scales with raw compressed volume and its query latency scales with bytes scanned in the time window.
- Elasticsearch indexes every line to make any line findable. Inverted-index terms, per-field doc values, and (in standard/logsdb modes) stored or synthetic source mean each document is materialized several times over. logsdb_columnar is Elastic admitting the Loki thesis — for log-shaped data, most of that machinery is dead weight — and deleting it selectively. Tech Preview in 9.5, behind explicit opt-in, because changing what gets indexed is a compatibility-breaking bet, not a knob.
Reproduce It Yourself — Both Sides, No Helm Required
You do not need a cluster to sanity-check these numbers against your own log shape. Both vendors ship single binaries. The Loki side (this is the exact config we ran, trimmed to the essentials):
auth_enabled: false
server:
http_listen_port: 3100
common:
instance_addr: 127.0.0.1
path_prefix: /tmp/loki-bench/loki-data
storage:
filesystem:
chunks_directory: /tmp/loki-bench/loki-data/chunks
rules_directory: /tmp/loki-bench/loki-data/rules
replication_factor: 1
ring:
kvstore:
store: inmemory
schema_config:
configs:
- from: "2024-01-01" # MUST be quoted or Loki fails with
store: tsdb # 'invalid schema version' — YAML hands it
schema: v13 # a date object and Atoi chokes on ""
index:
prefix: index_
period: 24h
object_store: filesystem
limits_config:
reject_old_samples: false
allow_structured_metadata: true
retention_period: 744h
ingestion_rate_mb: 64 # default is 4 — a real pipeline 429s instantly
ingestion_burst_size_mb: 128
Three operational details we hit that you will hit too:
- The default ingestion rate limit is 4 MB/s per tenant. Our first push failed with
429 Too Many Requestsand the distributor log said it plainly: “ingestion rate limit exceeded for user fake (limit: 4194304 bytes/sec) … reduce log volume or contact your Loki administrator”. In production you set this per-tenant deliberately; in a bench you just feel how far below real log volume the default sits. - Push timestamps are bounded by
creation_grace_period(default 10 minutes into the future). Corpus timestamps past that boundary are rejected withentry too new. If you backfill historical data, you will learn this limit by heart. - Push API batches are capped by a 4 MB gRPC message limit. A 25k-line JSON push failed with
rpc error: code = ResourceExhausted desc = grpc: received message larger than max (4467723 vs. 4194304). Batching thousands of lines is right; batching them in one giant request is not. Also: Loki 3.7 auto-appends aservice_name="unknown_service"label to pushed streams when OTel-style service detection finds no service name — expect it in your stream list, and expect it to count toward your label cardinality budget.
The Elasticsearch side, including the template trap:
# ES 9.5.4 ships a built-in composable template matching logs-*-* that
# creates DATA STREAMS, not indices. Any 'logs-*' index creation fails:
# "cannot create index with name [logs-foo], because it matches with
# template [logs] that creates data streams only, use create data
# stream api instead"
# Your real ingestion should target data streams (PUT _data_stream/logs-foo)
# — but for a storage bench, name indices outside the pattern:
curl -X PUT "localhost:9200/bench-std" -H 'Content-Type: application/json' -d '{
"settings": { "index": { "mode": "logsdb",
"number_of_shards": 1, "number_of_replicas": 0 } }
}'
# The Tech Preview columnar mode — same corpus, keyword-mapped message:
curl -X PUT "localhost:9200/bench-col" -H 'Content-Type: application/json' -d '{
"settings": { "index": { "mode": "logsdb_columnar",
"number_of_shards": 1, "number_of_replicas": 0 } }
}'
# Columnar WITH full-text search (message must be mapped text —
# in columnar mode it is NOT mapped as text by default):
curl -X PUT "localhost:9200/bench-col-text" -H 'Content-Type: application/json' -d '{
"settings": { "index": { "mode": "logsdb_columnar" } },
"mappings": { "properties": {
"message": { "type": "text" },
"@timestamp": { "type": "date" },
"ns": { "type": "keyword" },
"pod": { "type": "keyword" },
"level": { "type": "keyword" },
"route": { "type": "keyword" },
"code": { "type": "long" },
"dur": { "type": "long" } } }
}'
# Ingest via _bulk (NDJSON), then measure with the system's own numbers:
curl -X POST "localhost:9200/bench-col-text/_refresh"
curl "localhost:9200/bench-col-text/_stats/store,docs"
# -> store.size_in_bytes is what we report above. 7.9 MB / 32.4 MB / 52.6 MB
# are the three storage profiles of the SAME 300k lines.
Production Install Reality Check
Neither tool runs the way we benched it. What you actually deploy:
Loki goes in via the official chart — chart series 7.x is current (appVersion tracking Loki 3.6.x as of the September 2026 index), and the 3.7 line will follow. Micro-services mode (distributor, ingester, query-frontend, querier, compactor, as separate deployments) is the default serious layout:
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm upgrade --install loki grafana/loki \
--namespace loki --create-namespace \
--set deploymentMode=SimpleScalable \
--set loki.schemaConfig.configs[0].from="2024-01-01" \
--set loki.schemaConfig.configs[0].store=tsdb \
--set loki.schemaConfig.configs[0].schema=v13 \
--set loki.schemaConfig.configs[0].object_store=s3 \
--set loki.storage.bucketNames.chunks=loki-chunks \
--set loki.storage.bucketNames.admin=loki-admin \
--set loki.storage.s3.endpoint=s3.us-east-1.amazonaws.com \
--set loki.storage.s3.region=us-east-1 \
--set loki.structuredInjector.enabled=false \
--set loki.limit_overrides.default.ingestion_rate_mb=64
# SimpleScalable = gateway + read/write pods (ingester+distributor in write,
# querier+query-frontend in read). For >~few TB/day go full micro-services.
# storage secret: an S3 keypair via --set-file or a Secret named loki-s3
# referenced by loki.storage.s3.secret.
Elasticsearch goes in via ECK (the Cloud-on-K8s operator — 3.5.0 is current as of this month) if you are determined to self-host on Kubernetes, and the honest note is that ECK remains a full-time job: JVM node pools, PV sizing, shard-count per index policy, ILM for rollover, and the heap that keeps growing. For log workloads specifically, most teams that stay on Elastic end up on Elastic Cloud or a searchable-snapshot tiering strategy, not self-managed hot disks.
apiVersion: elasticsearch.k8s.elastic.co/v3
kind: Elasticsearch
metadata:
name: logs
spec:
version: 9.5.4
nodeSets:
- name: logs
count: 3
config:
node.roles: ["data_hot", "data_content"]
index.mode: logsdb # inherited by matching indices
resources:
requests:
memory: 8Gi
cpu: 2
volumeClaimTemplates:
- metadata:
name: elasticsearch-data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 512Gi
---
apiVersion: kibana.k8s.elastic.co/v3
kind: Kibana
metadata:
name: logs
spec:
version: 9.5.4
count: 1
elasticsearchRef:
name: logs
(Yes, the spec.version above pins the exact stable we benched. Check the ECK compatibility matrix before shipping — operator 3.5.0’s supported server versions lag the server release by days-to-weeks, and 9.5.4 may not be in it yet.)
The 2026 Decision Matrix
| Dimension | Loki 3.7.8 | Elasticsearch 9.5.4 (logsdb) | ES 9.5.4 (logsdb_columnar) |
|---|---|---|---|
| Storage on identical corpus | 10.1 MB — 24.6% of raw | 52.6 MB — 128.7% of raw | 7.9 MB keyword-only / 32.4 MB with text search |
| Full-text phrase search | Yes — linear scan, always works, no mapping step | Yes — inverted index, 8× faster (35 ms vs 280 ms single-node) | Only for fields explicitly mapped text; keyword-mapped message returns 0 hits |
| Steady-state memory | 0.12 GB RSS (Go binary) | 2.0 GB RSS at minimum viable heap | Same as logsdb |
| Object-store-native storage tier | Yes — chunks live on S3/GCS by design; that is the whole point | Partial — searchable snapshots / frozen tier bolt it on | Same as logsdb; columnar shrinks what the snapshot holds |
| Index/label discipline required | High — unbounded labels destroy compression and the index | Medium — shard count, mapping explosions, ILM policy | High — mapping decisions at ingest decide what is queryable |
| Maturity of the cheap mode | TSDB + chunks: default, 3 years hardened | logsdb: default since 9.0, GA since 8.17 | Tech Preview — 9.5.0, opt-in, do not bet production on it unwatched |
| Operational surface | ~6 components, Sane-mode Helm chart, no JVM | JVM, shards, heap folklore, ECK operator or managed cloud | Same as logsdb plus mapping-by-omission surprises |
| Retention cost (1 TB/day, 90d, $0.023/GB-mo) | ~$509/mo object storage | ~$2,664/mo before snapshot tiering | ~$1,643/mo with text search restored |
Choose Loki If…
- Your log volume is large and your access pattern is label-scoped. “Show me pod X’s errors in the last hour” is Loki’s home turf: stream selection by label, then a targeted scan. At multi-TB/day, the object-storage economics dominate every other line item, and nothing else in this comparison comes close to 24.6% of raw.
- You already run Grafana, Prometheus, and an OTel pipeline. Loki is the missing spoke of a stack you already operate. Alloy/Promtail scraping into Loki is one chart; the query UX (LogQL in the same Explore pane as PromQL) collapses two tools into one mental model for on-call engineers.
- You can enforce label discipline. If your platform team owns the telemetry pipeline (golden paths, shared collector config), you can cardinality-budget at the door. If arbitrary services ship their own labels, Loki will eat itself — budget for enforcement tooling (per-tenant limits,
max_global_streams, cardinality dashboards) or do not bother.
Choose Elasticsearch If…
- Interactive full-text search is the product. Security analysts pivot through payloads, support greps half-remembered user emails, engineers hunt by stack-trace fragment. Inverted-index latency (35 ms vs 280 ms single-node, wider at scale) is a feature, not overhead, when the human is in the loop.
- You need one store for logs + traces + security analytics + metrics-adjacent data. Elasticsearch 9.5’s release notes read like a convergence play: ES|QL federation over S3, OTLP ingestion with native metric temporality, PromQL over native histograms, Prometheus-series REST endpoints. Loki does logs. Elastic does all-of-observability, expensively.
- You are already on Elastic — then the real question is logsdb_columnar, not Loki. Migrating a logging pipeline to Loki to save storage, while your security stack stays on Elasticsearch, buys you two systems to operate and one platform team to do it. Setting
index.mode: logsdb_columnaron new data streams is one template change — when it goes GA and your mappings survive scrutiny.
Who Should Skip Both
- Under a few GB/day of logs: run Alloy or Vector into files, keep a week locally, grep when something breaks. A platform team that stands up either system for 2 GB/day is building a factory to make hammers for a house with no nails.
- No on-call capacity to own either one: Loki micro-services mode is 6+ components; self-hosted Elasticsearch is a JVM whisperer’s job. If the honest answer is “nobody will page on this at 3am,” buy Grafana Cloud or Elastic Cloud and take the markup as the price of sleep.
- All you want is “ship logs somewhere queryable, occasionally”: ClickHouse-based stacks and vector databases have entered this conversation too; the decision procedure in this guide (raw volume × retention × query pattern) applies to them unchanged.
The Skeptic’s Footnotes
- “Up to 65%” is a marketing number; 128.7% of raw is ours. Elastic’s logsdb claim is measured against prior Elasticsearch modes, not against the raw bytes you ship. Both framings are true simultaneously. When a vendor says “smaller,” ask “smaller than what, measured how, on whose data.”
- logsdb_columnar will improve fast — that cuts both ways. Doc-value skippers are already on by default for numeric fields in 9.5, and the release notes promise them for all applicable fields later. The 79.4%-of-raw-with-search number is the first release of a Tech Preview. But “Tech Preview” is also Elastic telling you plainly: mappings, semantics, and storage format may break between minors. Pin versions and test upgrades; do not let a logging platform self-upgrade its storage format.
- Loki 3.7’s own release notes list four BREAKING CHANGES — scheduler compute capacity, shared scheduler worker threads, parsed-labels-vs-structured-metadata precedence, and an OpenShift stream-label default flip. The “simple, boring log store” has an upgrade guide with teeth. Read it before you jump minor versions; the 3.7.8 patch is safe, the 3.7.0 boundary is not automatic.
- Benchmarks on one node are direction, not destination. We measured storage formats and single-query latency with the confounders (network, sharding, cache warmth, concurrent load) removed. Your production numbers will be worse and noisier on both sides. The ratios are the transferable finding; the absolute milliseconds are not.