Grafana Alloy's OTel Engine Review: We Ran the v1.21 Public Preview — Real Collector Semantics, Curated-Subset Reality

Verdict up front: With Alloy v1.21.0 moving the OTel engine into Public Preview, Grafana Alloy is now a single 542.6 MiB Linux binary that contains two complete telemetry pipeline engines: the original River-configured Default Engine, and a genuine OpenTelemetry Collector distribution you start with alloy otel. We downloaded the real v1.21.3 binary, ran both engines, pushed a live OTLP trace through each, and counted the components. The good news is that the collector semantics are real — upstream CLI flags, validate exit codes your CI can rely on, and standard collector YAML with zero translation. The uncomfortable news: the engine is a curated subset of 78 components (four of which are Stable), it has no UI, no reload endpoint, and no support bundles, it reports your enabled components to Grafana by default, and the docs already lag the binary (they claim collector v0.161.0; the binary we ran is built from v0.162.0). This is the right architecture arriving at a deliberately early stage — and for several categories of teams, the right answer today is still alloy run or a plain collector distribution.

Executive Scorecard

What Actually Shipped: One Binary, Two Engines, Eight Months of Runway

The OTel engine is not a v1.21 bolt-on. The release history tells a longer story: the engine entered the codebase unexposed in v1.13.0 (February 2026), gained the otelcol compatibility entrypoint for the Helm chart in v1.14.0 (March 2026), grew OpAMP-based Fleet Management support in v1.19.0 (August 2026, "Add opamp supervisor" and "Expose experimental OTel supervisor" in that release's notes), and moved to Public Preview in v1.21.0 (October 19, 2026), announced in the blog post on November 3. The v1.21.3 patch we tested shipped October 28; v1.22.0-rc.0 is already cutting.

Two design decisions deserve honest attention because they shape everything else. First, the OTel engine is not hidden behind a stability flag — the docs are explicit that this is deliberate, to "maintain full compatibility of the OTel Engine with the upstream OpenTelemetry Collector and expose a compatible command line interface." In practice this means upstream tooling assumptions hold (flags, config, Helm), but the Public Preview label carries the breaking-change risk instead. Second: the OTel engine is Grafana's distribution of the collector, not a mirror of contrib. It bundles a curated subset selected on "demand, maturity, maintenance cost, dependency footprint, security risk, and license compatibility" — which is why the count matters so much (see the parity section below).

Architecture Mechanics

One executable, two entrypoints, one escape hatch, and a commercial control plane. This is the actual shape of the thing, as shipped in v1.21.3:

┌───────────────────────────────────────────────────────────────────────────────────────┐
│  grafana/alloy executable  (v1.21.3, 542.6 MiB on disk, 161.5 MiB compressed)         │
│                                                                                       │
│  $ alloy run ──▶  DEFAULT ENGINE  (River .alloy configuration)                        │
│       - HTTP UI + live reload on :12345, clustering, --storage.path                   │
│       - support bundles (/-/support), 186 documented components                       │
│       - Prometheus scrape, Loki pipelines, OTLP-in, Grafana integrations              │
│                                                                                       │
│  $ alloy otel ──▶  OTEL ENGINE  (standard collector YAML, no translation)             │
│       - self-metrics on :8888 ONLY: no UI, no reload, no support bundles              │
│       - disk state via filestorage extension (no --storage.path)                      │
│       - 78 bundled collector components (v0.162.0 contrib modules)                     │
│       - PUBLIC PREVIEW: breaking changes allowed, no stability flag                   │
│                                                                                       │
│  optional: alloyengine extension (EXPERIMENTAL)                                      │
│       - runs ONE Default Engine pipeline inside the OTel engine process               │
│       - no direct data path between engines: a crossover goes out over                │
│         the network (usually OTLP) and comes back in a receiver                       │
│       - ready before the Alloy config loads; reload retries capped at 15s             │
│                                                                                       │
│  $ alloy otel-supervisor ──▶ embedded OpAMP supervisor (EXPERIMENTAL)                 │
│       - pulls OTel engine config from Grafana Fleet Management (Cloud)                │
└───────────────────────────────────────────────────────────────────────────────────────┘

The differences between the engines are not cosmetic, and the docs are unusually blunt about them. The most consequential rows:

Hands-On: What We Actually Ran (v1.21.3 on Linux)

We downloaded the real alloy-linux-amd64.zip (169,396,336 bytes) from the v1.21.3 release, unpacked the binary (568,981,832 bytes on disk), and exercised both engines against a three-signal pipeline: OTLP receiver, batch processor, debug exporter. The hands-on findings below are all from our own runs on this host — nothing here is quoted from a landing page.

The CLI contract is genuinely collector-shaped

$ alloy otel --help
Use Alloy with OpenTelemetry Collector Engine

Available Commands:
  components   Outputs available components in this collector distribution
  featuregate  Display feature gates information
  print-config Prints the Collector's configuration in the specified mode
  validate     Validates the config without running the collector

Flags:
      --config ...    Locations to the config file(s)
      --disable-reporting   Disable reporting of enabled components to Grafana.
      --feature-gates flag  Comma-delimited list of feature gate identifiers
      --set value      Set arbitrary component config property
  -v, --version        version for otel

Two details that matter to platform teams. First, validate is CI-grade: our good config exits 0 silently, and a config with a typo'd exporter exits 1 with the exact valid-value list in the error — 'exporters' unknown type: "not_a_real_exporter" ... (valid values: [zipkin faro file otlp_grpc prometheusremotewrite otlp otlp_http otlphttp awss3 kafka loadbalancing prometheus prometheus_remote_write syslog debug nop load_balancing]). That is a better failure message than most collectors give you. Second, that --disable-reporting flag exists because the default is the opposite: the OTel engine reports which components you enabled to Grafana unless you explicitly opt out. Our startup log confirms the reporter runs: level=INFO msg="running usage stats reporter". For air-gapped or egress-hostile environments, bake the flag into your Helm command and systemd overrides on day one — the flag is not part of upstream collector semantics, so nothing in the upstream docs ecosystem will remind you.

A live trace, end to end, through both engines

We started the OTel engine with a minimal three-signal config (the one below is the exact file we ran):

extensions:
  health_check:
    endpoint: 0.0.0.0:13133

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318

processors:
  batch:
    timeout: 5s
    send_batch_size: 512

exporters:
  debug:
    verbosity: detailed

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [debug]

The engine came up in under a second: health check on :13133 answered HTTP 200, and the self-metrics endpoint on :8888 exposed otelcol_process_* telemetry, including the memory figure we use in the cost section. We then POSTed a single OTLP JSON span to http://localhost:4318/v1/traces — HTTP 200 — and the debug exporter printed the complete span, verbatim:

    Name           : hands-on-probe
    Kind           : Internal
    Start time     : 2025-11-07 12:48:00 +0000 UTC
    End time       : 2025-11-07 12:48:01 +0000 UTC
    Status code    : Ok
    DroppedAttributesCount: 0
    DroppedEventsCount: 0
    DroppedLinksCount: 0
Attributes:
     -> probe: Str(alloy-otel-engine)

We repeated the same experiment on the Default Engine with the mirror-image River pipeline (otelcol.receiver.otlp → otelcol.processor.batch → otelcol.exporter.debug). The trace flowed through identically — but with two engine-specific wrinkles worth knowing. The Default Engine rejected the config outright at first attempt, because otelcol.exporter.debug is marked experimental on that side:

$ alloy run /tmp/config.alloy
Error: /tmp/config.alloy:23:1: component "otelcol.exporter.debug" is at
stability level "experimental", which is below the minimum allowed
stability level "generally-available". Use --stability.level command-line
flag to enable "experimental" features

$ alloy otel --config=collector.yaml     # same exporter, OTel engine side
# starts cleanly with no stability flag at all

This is the sharpest example of the split-brain risk in a two-engine binary: identical components live under different stability regimes depending on which engine loads them. The OTel engine inherits upstream maturity wholesale (so the debug exporter is just... an exporter); the Default Engine applies Grafana's own stability gating on its wrappers (so the same component needs --stability.level=experimental, and in v1.21.0 that gate was tightened even further — loki.write WAL configurations now hard-fail without the flag). Teams running both engines in one fleet are maintaining two config languages and two stability policies at once. Plan for that, or pick one engine per environment deliberately.

Idle memory: the engines are twins, and that tells you what you're running

With identical logical pipelines, idle RSS measured 208,900 KB (~204 MiB) for the OTel engine and 207,524 KB (~203 MiB) for the Default Engine — statistically indistinguishable, and consistent with each engine's self-reported otelcol_process_memory_rss. The practical reading: both engines are carrying the same collector-family machinery underneath, so the engine choice is not a resource optimization — it is a config-language and operations-model choice. At DaemonSet scale, ~204 MiB idle RSS per node is the entry price either way; budget accordingly, and don't let anyone sell you the OTel engine as "lighter." It is not. It is the same machine with a different steering wheel.

The Deployment Shapes That Are Already Real

This is where Public Preview undersells itself: the deployment surface is not aspirational. All four installation paths below are documented, verified paths — the Helm and systemd shapes are the ones you would actually run.

Kubernetes: the upstream collector chart, pointed at the Alloy image

The OTel engine deliberately reuses the upstream OpenTelemetry Collector Helm chart — you are not adopting a Grafana-specific chart. Two values swap the engine:

image:
  repository: grafana/alloy
  tag: v1.21.3

command:
  name: "bin/otelcol"

mode: deployment

ports:
  metrics:
    enabled: true

config:
  receivers:
    otlp:
      protocols:
        grpc: {}
        http: {}
  processors:
    batch:
      timeout: 1s
      send_batch_size: 512
  exporters:
    otlp_http/my_backend:
      endpoint: https://OTLP_ENDPOINT
      auth:
        authenticator: basicauth/my_auth
  extensions:
    basicauth/my_auth:
      client_auth:
        username: "${env:GRAFANA_CLOUD_INSTANCE_ID}"
        password: "${env:GRAFANA_CLOUD_API_KEY}"
  service:
    extensions: [basicauth/my_auth, health_check]
    pipelines:
      traces:
        receivers: [otlp]
        processors: [batch]
        exporters: [otlp_http/my_backend]

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update
helm install alloy-otel open-telemetry/opentelemetry-collector --values values.yaml
kubectl get pods   # Running once the OTel engine starts

The subtle load-bearing detail: command.name: "bin/otelcol" makes the chart run /bin/otelcol inside the Alloy image — a compatibility entrypoint that executes alloy otel. Leave it out and the image's default entrypoint silently starts the Default Engine instead, which will then fail to parse your collector YAML. That is a genuinely confusing failure mode the first time you hit it.

OpenTelemetry Operator: one line, as of v0.153.0

Before Operator v0.153.0 (June 2026), the Operator could not set a container command, so the Alloy image failed under it. Now it is a single field on the collector spec documented by the OpenTelemetry Operator reference:

spec:
  image: grafana/alloy:v1.21.3
  command: ["/bin/alloy", "otel"]

VMs: the packaged service, flipped with one env var

# /etc/default/alloy on Debian/Ubuntu, /etc/sysconfig/alloy on RHEL/SUSE
ALLOY_OTEL_MODE=1
OTEL_CONFIG_FILE=/etc/alloy/config.yaml   # package installs a sample here
# OTEL_CUSTOM_ARGS="--feature-gates=..." # extra flags, like CUSTOM_ARGS

alloy otel validate --config=/etc/alloy/config.yaml
sudo systemctl restart alloy
curl -s http://localhost:8888/metrics | head   # the OTel engine's only HTTP surface

Remote config: otel-supervisor and the Fleet Management coupling

The otel-supervisor subcommand embeds an OpAMP supervisor that fetches pipeline configuration from Grafana Fleet Management. Simple mode is four environment variables — note what kind of product this path assumes:

export GCLOUD_FM_URL="https://your-fleet-endpoint"
export GCLOUD_INSTANCE_ID="your-instance-id"
export GCLOUD_RW_API_KEY="your-api-token"
export STORAGE_DIR="/var/lib/alloy/supervisor"
alloy otel-supervisor

Manual mode speaks standard OpAMP to any compliant server, with remote-config acceptance made explicit — this is the self-hosted escape valve:

server:
  endpoint: "${env:OPAMP_URL}/v1/opamp"
  headers:
    Authorization: "Basic ${env:OPAMP_BASIC_AUTH}"
capabilities:
  accepts_remote_config: true
storage:
  directory: ${env:STORAGE_DIR}

Be clear-eyed about the trade: OpAMP remote config is powerful — a misconfigured fleet push changes every collector's pipeline — and the polished, zero-setup path for it is the Grafana Cloud commercial feature. The supervisor itself is still marked Experimental (it requires stability.level=experimental, unlike the OTel engine), so treat fleet-push as preview-grade too.

The Component Parity Gap, Measured

We counted both sides. The OTel engine bundles 78 components in v1.21.3: 27 receivers, 17 processors, 13 exporters, 6 connectors, 15 extensions — counted directly from alloy otel components on the unpacked binary. The Default Engine's reference documentation carries 186 component pages (72 of them otelcol.* wrappers). Grafana's own engine-choice guidance frames the OTel engine for "push-based, OpenTelemetry-native signals through OTLP" and the Default Engine for everything Grafana-specific — and the numbers back the guidance.

The stability mix inside the bundle is the more important number. By upstream's own per-signal stability model, only four entries are Stable: the otlp receiver, the k8s_attributes processor, and the otlp_grpc/otlp_http exporters. Everything else — the entire Kafka surface, filelog, hostmetrics, the spanmetrics and servicegraph connectors — is Beta, Alpha, Undefined, or Development. That is not a defect: it accurately mirrors contrib's real maturity distribution, where most components never reach Stable. But it means "the OTel engine is collector-native" and "everything in it is production-stable" are very different sentences. Check the stability of the specific components your pipeline needs before standardizing on it.

What is absent matters just as much as what is present. The bundle has no Google Cloud receiver and no googlecloud exporter — the GCP story in the launch blog is about a custom OCB build with the experimental Prometheus-branded collector distro (which embeds Prometheus exporters like stackdriver and blackbox as receivers), not something the shipping binary does today. The lifecycle policy softens drop-risk: deprecated components stay for at least two more minor releases, unmaintained ones for three months, and the docs publish an OCB recipe for adding dropped components back — including the alloyengine extension itself, via its Go import path. The message is coherent: escape hatches are documented, not accidental.

One documentation drift worth flagging, because it is the kind of thing that erodes trust in a preview: the otel command reference page claims "Alloy v1.21.3 bundles OpenTelemetry Collector components from version v0.161.0," while the binary itself reports contrib modules at v0.162.0 — and the same release's changelog records the collector upgrade to v0.162.0 across the otelcol surface. The docs page also undercounts the bundled extensions (13 listed, 15 in the binary). Minor, fixable, and symptomatic of a fast-moving preview: trust alloy otel components output over the docs page for component inventory.

Critical Failure Modes

Where this breaks, in the order you are likely to meet it:

• No cross-engine data path. The two engines share a process (via alloyengine) but not a pipeline bus. Anything that crosses from the Default Engine side to the OTel Engine side goes out over the network, usually OTLP, and comes back in a receiver. Every crossover doubles your serialization cost and failure surface. The docs' own guidance for large workloads is to run separate processes for OTLP gateways and Prometheus scrapers anyway — the extension is explicitly for "small deployments."
• Readiness masking inside the extension. The alloyengine extension reports ready immediately — it does not wait for the embedded Alloy config to load, and it retries failed loads "at most every 15 seconds." The collector's health check can be green while the Default Engine pipeline inside it is broken or absent. If you use the extension, probe the embedded engine's own /-/ready or /-/healthy endpoint as your readiness gate, not the collector's.
• One extension per process, and a bad one bricks startup. The embedded Default Engine holds process-global state (Prometheus registry, controller ID, storage path), so a second alloyengine instance fails — and because collector extension startup failures fail the whole collector, one misconfigured block prevents the process from starting at all. That is fail-closed (good) but unforgiving (plan your config CI with alloy otel validate).
• No reload endpoint means config changes are restarts. The Default Engine gives you live reload and a UI; the OTel engine gives you a rolling restart — or an OpAMP push if you accept remote config. On a 500-node DaemonSet, "config change = restart" is a scheduling decision, not a convenience complaint.
• Forensics gap. The Default Engine's /-/support bundles config, logs, internal metrics, runtime flags, and pprof profiles into one artifact. The OTel engine has no equivalent today — an OTel-native version is "being built" per the announcement. Until it lands, your incident runbook for OTel-engine collectors is manual assembly from :8888 metrics, pprof, and zpages. That is a real gap during a 3 a.m. cardinality incident.
• The anonymous reporting default. Enabled components are reported to Grafana until --disable-reporting is passed. Air-gapped shops: this flag belongs in your base Helm values, systemd unit, and OCB wrapper from day one.
• Public Preview means exactly what it says. "Subject to breaking changes, and may be replaced with equivalent functionality that covers the same use case" is the documented contract. Pin exact versions, read release notes, and keep your collector configs in a form that OCB-rebuilds against plain contrib if Grafana's curation diverges from your needs.

Cost and Trade-Offs

That is the two-engine tax: you ship both engines in every image regardless of which one you run. Runtime memory, as measured above, is ~204 MiB idle per process either way. What the premium buys is optionality — one artifact per host that can serve collector-native pipelines, River pipelines, or both in one process — plus Grafana's curation (a smaller, security-reviewed surface than contrib's long tail). The honest counterpoint: if you are purely collector-native and have no intention of using River components, the plain otelcol-contrib distribution or your own OCB build is a smaller artifact with the same core machinery, and nothing stops you from pointing it at Grafana backends — the Remote Write 2.0 path that Grafana is investing in works from any collector. The commercial center of gravity for the OTel engine is Grafana Cloud: Fleet Management for fleet config, Application Observability as the OTLP-native consumer. For the self-hosted fleet, the OSS price is right; just budget the bytes and the RSS.

Who Should Skip This

• Existing Alloy shops with working River pipelines. There is no migration win today: the Default Engine keeps the stability guarantees, the UI, clustering, support bundles, live reload, and the Grafana-specific components (Database Observability, eBPF auto-instrumentation, the Loki pipeline). The OTel engine is for teams arriving from the collector world, not for teams being pushed out of River — note that alloy convert still translates other formats into Alloy syntax, and nothing translates River into collector YAML.
• Contrib-maximalists. If your pipeline needs the components Grafana's curation excludes — googlecloud exporter, the long tail of vendor receivers — you are OCB-building anyway. Build a plain distribution and skip the 1.5× image premium.
• Air-gapped and strict-egress environments. Usage reporting defaults on, the flagship remote-config path is a cloud product, and the forensics story (support bundles) is still being built. Every one of those adds a workaround to maintain.
• Teams that need reload-without-restart today. Until OpAMP remote config is out of Experimental (or the engine grows a reload endpoint), config changes are process restarts. If your change cadence is high, that is a tax on every rollout.
• Anyone hoping for a quiet GA. This is a Public Preview with documented breaking-change rights, a docs page that already lags its binary by one collector version, and an escape-hatch extension that is Experimental "and we expect it to stay that way" (their words). Adopt it as what it is — a bet on Grafana's OTel convergence — not as settled infrastructure.

Verdict

Grafana has handled an genuinely awkward position better than most vendors would: rather than pretending Alloy's River syntax and the industry's OTel convergence trend don't conflict, it shipped both and let you pick per deployment — with real upstream CLI semantics, a curated component surface, and documented escape hatches rather than accidental ones. Our measurements say the engineering is honest: the collector inside is real (end-to-end trace verified), the validator is CI-grade, the memory cost is the same as the engine you already run, and the component inventory prints itself. The gaps are equally honest — no UI, no reload, no support bundles, a mostly-Beta component maturity mix, and a reporting default you must actively opt out of.

For collector-native teams standardizing on OTLP who want Grafana backends, this is the first Alloy release actually worth running, and the upstream Helm chart + Operator path makes it a low-friction pilot. For everyone already invested in River, the right move is to watch, not switch — the Default Engine remains where the production-hardened Grafana features live, and the interesting remote-write economics (the Remote Write 2.0 work that cut Grafana's own egress "by more than half") land there first. Our long-term storage comparisons live in the Mimir long-term storage guide and VictoriaMetrics vs. Prometheus; the log-side pipeline rebuild that v1.21's changelog hints at is the one we will re-examine in our Loki coverage when the WAL reaches GA. Re-evaluate the OTel engine when support bundles, reload, and a Stable-grade bundle share arrive — or at GA, whichever comes first.

References & Further Reading

• Grafana Labs: Bridging Alloy and OpenTelemetry — the OTel engine is now in Public Preview (announcement, engine roadmap, Prometheus Distro direction)
• Alloy documentation: OpenTelemetry in Alloy (engine comparison, public preview terms, port/storage differences)
• Alloy documentation: the otel command (bundled component list, lifecycle/removal policy, --disable-reporting)
• Alloy documentation: the otel-supervisor command (OpAMP supervisor, simple and manual modes)
• Alloy documentation: set up the OTel engine (CLI, Helm chart, service installations, OCB builds)
• grafana/alloy v1.21.0 release notes (OTel engine Public Preview; loki.write WAL stability gate; collector v0.162.0 upgrade)
• alloyengine extension README (embedded Default Engine lifecycle, single-instance limit, OCB recipe)
• Grafana Fleet Management (the OpAMP-based commercial remote-config path)
• OpenTelemetry Operator v0.153.0 (spec.command support that makes the Alloy image Operator-compatible)
• OpenTelemetry Collector releases, v0.162.0 (the upstream baseline the OTel engine bundles)
• prometheus/prometheus-opentelemetry-collector (the experimental Prometheus-branded distro embedding exporters as receivers)
• Prometheus Remote Write 2.0 specification (the egress-reduction path underpinning the default engine's remote_write roadmap)
• OpenTelemetry Collector documentation (stability levels, OCB builder, upstream configuration reference)