The OCI GitOps Shift: When the Registry Becomes the Source of Truth

On November 4, 2026, at 09:49 UTC, GitHub started investigating degraded performance for Git Operations, Issues and Pull Requests. Two hours later — 11:50 UTC — the incident was resolved. In between, Git clones intermittently failed, SSH pushes and pulls returned permission errors that looked to many engineers exactly like a compromised key, Actions degraded, and a Hacker News thread titled Incident with Pull Requests, Git Operations and Issues climbed to 265 points. Near the top of that thread, a comment that should be printed out and taped to every platform team's wall: "Our deploys are tied to GitHub actions and it is becoming a liability at this point."

That comment is the whole story of this article. A GitOps control loop is a distributed system that treats a third-party SaaS control plane — GitHub's Git operations stack — as a hard runtime dependency. Every reconciliation interval, every cluster in your fleet reaches back out to github.com to ask one question: what is the desired state? When GitHub cannot answer, your fleet is blind. It cannot deploy, it cannot roll back, and it cannot even drift-correct, because the source of truth is unreachable.

Over the past year the Flux and Argo CD ecosystems have quietly built out every primitive needed to relocate that source of truth: Flux's OCIRepository API as a first-class pull source, flux push artifact as the publication path, a plugin system (Flux v2.9, June 2026) whose flagship plugin is a declarative registry-to-registry mirror, Argo CD's OCI application sources and the new v3.6.0 ApplicationSet OCI generator (November 3, 2026), and HelmRelease chart sources that resolve straight out of a registry. The Flux project has a name for the end state: Gitless GitOps — Git stays where humans collaborate, and the registry becomes the delivery interface.

This is not a tutorial about replacing Git wholesale. It is the production mechanics of artifact-based delivery: exactly what changes on the wire, in the API objects, in your auth model, in your audit story, and in your failure modes — including the failure modes that only appear once your registry becomes the source of truth.

The Outage That Made the Argument

The official incident record is worth reading closely, because it names precisely the surface that GitOps pull loops depend on. GitHub Status classifies Git Operations as "Performance of git clones, pulls, pushes, and associated operations." That is the exact endpoint every GitRepository and every Argo CD Application reconciles against.

09:49  We are investigating reports of degraded performance for
       Git Operations, Issues and Pull Requests
09:50  Actions is experiencing degraded performance
09:53  Pull Requests is experiencing degraded availability
10:38  Investigating elevated errors and intermittent failures affecting
       GitHub including Pull Requests, Git Operations, and Issues
11:11  API Requests is experiencing degraded performance
11:17  Working on mitigating elevated errors and intermittent failures
11:30  Degradation affecting API Requests, Git Operations, Issues and
       Pull Requests has been mitigated
11:43  Degradation affecting Actions has been mitigated
11:50  This incident has been resolved

Impact:  major
Components degraded: Git Operations, Pull Requests, Issues,
                      Actions, API Requests

Two structural facts stand out. First, the incident degraded both halves of a GitHub-anchored delivery pipeline: the CI system (Actions) that produces the desired state, and the Git transport that carries it to clusters. Teams whose deploy pipeline was GitHub end-to-end had no working plane at all. Second, the component that stores and serves OCI packages — GitHub Container Registry — was not among the degraded components that day. A cluster that pulls pre-rendered manifests from a registry was still able to reconcile, roll back, and drift-correct during a two-hour Git outage, because its source of truth answered with a different control plane.

The Hacker News thread captured what that means operationally. Users reported "git clone works sometimes, sometimes not", SSH pulls failing with permission errors that mimicked credential theft, and pushes that completed server-side while erroring client-side. One commenter reported standing up a Gitea instance and switching his deployment script to it in 30 minutes. Another runs a file-cache service around GitHub specifically because customers kept using it to survive outages. These are ad-hoc, per-team workarounds for a systemic architectural problem: the source of truth lives inside someone else's blast radius.

Git as a Runtime Dependency: The Mechanics

To understand why teams are moving, you need to be precise about how Git-host dependency actually bites. There are three distinct failure surfaces, and conflating them leads to bad architecture decisions.

Failure surface 1: API rate limits on the integration plane

Everything around your manifests — CI, PR status reporting, image automation, release tooling — rides GitHub's REST and GraphQL APIs, and those have hard, documented quotas:

The 1,000/hour per-repository Actions budget is the one that surprises teams first. A busy repository with several jobs doing API-heavy work — releases, status checks, artifact listing — can genuinely exhaust it. On top of the primary limits sit secondary rate limits: no more than 100 concurrent requests, a 900 points/minute budget for REST endpoints, roughly 80 content-generating requests per minute, and 2,000 OAuth token requests per hour. Unlike primary limits, GitHub states plainly that secondary limits "are subject to change without notice" and can trigger "for undisclosed reasons." When you exceed one, you get a 403 or 429 — the same status code as a permission problem, which is why rate-limit exhaustion so often masquerades as an auth incident in the middle of the night.

Failure surface 2: Git operation throttling

The manifest pull itself — git clone and git fetch over HTTPS or SSH — does not consume the REST API quota, but it is subject to abuse detection, secondary throttling, and (as November 4 demonstrated) availability incidents. A fleet of N clusters each running a GitOps controller with a default interval of M minutes means the Git host sees N × (60/M) clone-or-fetch operations per hour from your automation alone, on top of every human's git pull. During degradation, this traffic does not fail cleanly — it fails intermittently, which is the worst possible mode for a reconciler: some clusters see the new commit, some do not, and your fleet state silently diverges for the duration.

Failure surface 3: correlated control-plane outages

The Git host is one company's operations. When its control plane degrades, everything anchored to it — the repo, the Actions runners, the registry under the same vendor, the web UI your on-call uses to read the incident — tends to wobble together. An architecture that puts authoring (Git), CI (Actions), and delivery (the manifest transport) on one vendor's platform has one shared fate, no matter how many clusters or regions you fan out to.

                    +--------------------------------+
                    |   GitHub (single vendor)       |
                    |                                |
   engineer --PR--> |  repo / PRs / Actions / API    |
                    |  git operations endpoint      |
                    +--+--------------------------+--+
                       ^                         ^
        clone/fetch    |    clone/fetch          |    clone/fetch
       (throttled,     |   (throttled,           |   (throttled,
        intermittent)  |    intermittent)         |    intermittent)
                       |                         |
              +--------+-------+        +---------+------+
              |  cluster A    |        |  cluster B    |
              |  GitRepository|        |  GitRepository|
              |  reconciler   |        |  reconciler  |
              +---------------+        +---------------+

  During a Git-operations incident: no deploys, no rollbacks,
  no drift correction anywhere in the fleet.

What GitOps Actually Requires: Principles vs Tool

The most common objection to moving the source of truth out of Git is that it "breaks GitOps." It does not — because GitOps, as specified, never required Git. The CNCF OpenGitOps working group's GitOps Principles v1.0.0 define four requirements for the desired state of a managed system:

• Declarative — desired state is expressed declaratively.
• Versioned and Immutable — desired state "is stored in a way that enforces immutability, versioning and retains a complete version history."
• Pulled Automatically — software agents automatically pull the desired state declarations from the source.
• Continuously Reconciled — software agents continuously observe actual state and attempt to apply the desired state.

Notably, the glossary that anchors the specification defines the State Store as "a system for storing immutable versions of desired state declarations," and states: "Git, from which GitOps derives its name, is the canonical example used as this state store but any other system that meets these criteria may be used." The specification is transport-agnostic by design. Git is the reference implementation, not the contract.

Now run an OCI registry against the four principles. Declarative: the artifact is a tarball of exactly the YAML your reconciler applies. Pulled and continuously reconciled: that is what OCIRepository plus Kustomization — or an Argo CD Application with an oci:// source — do, on the same reconciliation model as Git sources. Version history: tags, plus a retained set of digests, and the Flux CLI pushes every artifact with its origin metadata attached.

It is the second principle — versioned and immutable — where the registry is not merely equal to Git but strictly stronger. Git's immutability is procedural: history can be rewritten, branches force-pushed, tags re-pointed. Everyone who has watched a supply-chain incident unfold (more on the Trivy action compromise below) knows exactly how tags get force-pushed. An OCI artifact's immutability is cryptographic: the digest sha256:… is the content. Tamper with one byte and you have a different, detectable artifact. If your state store pins a digest, no upstream action — credential theft included — can silently change what your clusters pull.

Repo-Based vs Artifact-Based Delivery

Here is the honest comparison, dimension by dimension, assuming Git remains the authoring plane in both models. The difference is what the cluster reconciles against.

The last two rows are where honest engineering lives. Artifact-based delivery trades a redundant, human-paced dependency (a Git host, which also serves code review) for a load-bearing, machine-paced one (a registry). That trade is excellent when the registry is more reliable, better authenticated, and closer to your clusters than the Git host — and it is a regression when you stand up a registry nobody operates and expect it to have GitHub's operational maturity. The rest of this article assumes you are making that trade deliberately.

The Anatomy of a Flux OCI Artifact

A Flux OCI artifact is a standard OCI image manifest whose layers carry configuration instead of container filesystems. This matters more than it sounds: because it is a standard manifest, every registry, mirror, pull-through cache, and security scanner that knows OCI handles it with zero special-casing — you get the same storage, replication, retention, and auth tooling your images already use. The media types:

The content layer is a tar+gzip of your rendered Kubernetes manifests. The config blob is small JSON metadata — and this is the detail that keeps your audit story intact: flux push artifact stamps every artifact with the source Git URL and the exact revision (branch and commit SHA) that produced it. As the Flux team puts it, this is how flux trace can walk any object on the cluster back to the commit that produced it. The Git linkage is not lost — it moves into the artifact's own metadata, where it travels with the bytes.

Publishing from CI

The publication side is deliberately boring — a CLI push, the same shape as a docker push:

# Inside the CI job. GHCR credentials arrive as environment
# variables from the CI secret store (in GitHub Actions, map the
# built-in token into the job env — see the Flux GitHub Action
# docs for the exact workflow syntax).

# flux push artifact reads registry credentials from the standard
# Docker config; alternatives are --creds or --provider aws|azure|gcp.
echo "$GH_TOKEN" | docker login ghcr.io \
  --username "$GHCR_USER" --password-stdin

# Render final manifests — this is the moment kustomize, helm,
# jsonnet or cue runs. What lands in the artifact is exactly
# what the cluster will apply.
kustomize build ./manifests/staging > ./deploy/app.yaml

# Package ./deploy as an OCI artifact, tagged with the short SHA,
# with provenance metadata pointing back at the Git source.
flux push artifact oci://ghcr.io/my-org/manifests/app:"$GIT_SHA_SHORT" \
  --path="./deploy" \
  --source="https://github.com/my-org/app.git" \
  --revision="main@sha1:$GIT_SHA"

# Move a mutable environment tag to point at the new digest.
flux tag artifact oci://ghcr.io/my-org/manifests/app:"$GIT_SHA_SHORT" \
  --tag staging

On success the CLI prints the digest — this is the value your prod pin will reference later:

artifact successfully pushed to
ghcr.io/my-org/manifests/app@sha256:4f90664660b3a567287e6957fa0481f347541b5908f6f797ec665255a399aed6

Three properties of this flow deserve emphasis. First, the renderer moved to CI: the cluster no longer runs kustomize at reconcile time, so a bad overlay or a chart bug fails the pipeline, not the fleet. Second, the artifact is immutable: flux tag artifact moves a tag; it never mutates a digest. Third, the same command shape works against ECR, GCR, ACR, GHCR, Harbor, or zot — the transport is a registry protocol, not a vendor feature.

OCIRepository: The Full Spec, Field by Field

OCIRepository is the pull-side API — a Source CRD in the same family as GitRepository, today at source.toolkit.fluxcd.io/v1. Everything a GitRepository does — artifact production, in-cluster storage, notifications, webhook triggering — works identically for it. The production-grade shape:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 5m
  timeout: 1m30s
  url: oci://ghcr.io/my-org/manifests/app
  ref:
    semver: ">=6.9.0"
  layerSelector:
    mediaType: "application/vnd.cncf.flux.content.v1.tar+gzip"
    operation: extract
  provider: aws
  serviceAccountName: source-controller
  verify:
    provider: cosign
    secretRef:
      name: cosign-pubkeys
  ignore: |
    tests/
    *.md

URL and reference resolution

spec.url is oci://<host>:<port>/<org>/<repo> — tags and digests are not allowed in the URL; they live in spec.ref, which resolves with strict precedence: digest over semver over tag. If ref is omitted entirely, it defaults to latest — fine for a staging pointer, a landmine for anything else.

The semver field is a Masterminds constraint, which means one footgun you must know: constraints without a prerelease comparator skip prerelease tags. >=6.9.0 will never resolve 7.0.0-rc.1. Append -0 to include them, and combine with semverFilter (a regex over the fetched tag list) when you publish RCs and GA tags into one repository:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app-rc
  namespace: flux-system
spec:
  interval: 5m
  url: oci://ghcr.io/my-org/manifests/app
  ref:
    semver: ">=6.9.0-0"
    semverFilter: ".*-rc.*"

For locked production environments, pin the digest and delete every other resolution strategy from your mind:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app-prod
  namespace: flux-system
spec:
  interval: 10m
  url: oci://ecr-account.dkr.ecr.eu-west-1.amazonaws.com/my-org/app
  ref:
    digest: "sha256:4f90664660b3a567287e6957fa0481f347541b5908f6f797ec665255a399aed6"
  provider: aws

A digest-pinned OCIRepository reconciles idempotently forever: the controller verifies the manifest digest on every poll, and a mismatch is impossible without a registry compromise. Promotion between environments becomes an explicit, reviewable diff of one YAML field — and because the digest is the content, reviewing the digest change is reviewing the deployment.

Interval, timeout, and jitter

interval is the requeue period (Go duration, e.g. 10m0s); timeout bounds each pull, defaulting to 60s. Two operational notes from the controller docs: spec changes are handled instantly outside the interval window, and the controller can be configured to apply jitter across OCIRepository objects sharing an interval — with hundreds of sources on one interval, jitter is the difference between a flat reconciliation profile and a thundering herd against the registry every five minutes.

Layer selection and file exclusion

layerSelector tells the controller which layer to unpack — the content layer above — and how: extract unpacks the tarball, copy stores it as-is. The copy mode matters for Helm chart artifacts, covered below. ignore applies .gitignore-style exclusion rules while archiving; note that specifying it overrides the default exclusion list rather than extending it.

Verification: signatures checked at reconcile time

The field that has no real equivalent in Git-based sources is spec.verify. The controller verifies Cosign or Notation signatures on every reconciliation — not at admission, not in a scanner, but at the moment the source is pulled:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 5m
  url: oci://ghcr.io/my-org/manifests/app
  ref:
    tag: staging
  verify:
    provider: cosign
    secretRef:
      name: cosign-pubkeys

Two supported providers today: cosign and notation. The referenced Secret holds the trusted authors' public keys. Keyless verification is also supported — and since Flux v2.9, organizations running their own Sigstore infrastructure (self-hosted Rekor and Fulcio, air-gapped) can verify keyless signatures against a custom trusted root rather than the public Sigstore infrastructure. The result is a hard guarantee your Git-based pipeline cannot easily match: an unsigned or wrongly-signed artifact fails reconciliation, and the cluster keeps running its last-known-good state instead of applying it.

Authentication: provider, serviceAccountName, secretRef, certSecretRef

OCI auth is richer than Git auth because registries are, and the spec reflects it:

• provider — generic (default), aws, azure, or gcp. The cloud providers enable Workload Identity so the controller pulls from ECR/ACR/GAR with no static credential anywhere — the deep dive is below.
• serviceAccountName — depending on provider: the ServiceAccount whose image pull secrets are used (generic), or whose object identity is used (aws/azure/gcp).
• secretRef — a Secret in imagePullSecrets format (kubectl create secret docker-registry ...) for static credentials.
• certSecretRef — mutual TLS: a Secret with tls.crt and tls.key (client certificate and key, required together), plus ca.crt if the registry serves a self-signed certificate. flux create secret tls builds it.
• proxySecretRef — per-object egress proxy settings (address, username, password keys). One warning from the docs that will cost you a full debugging day if you miss it: Cosign keyless verification is not supported through proxySecretRef. If you verify keyless, proxy the controller via the standard HTTPS_PROXY/ALL_PROXY environment variables instead, or keyless checks will silently bypass the object-level proxy.
• insecure — permits plain-HTTP registries; default false, and keep it that way outside throwaway clusters.

The artifact contract on the cluster

Once reconciled, the produced Artifact is stored in-cluster (source-controller's persistent storage) and exposed over the controllers' internal HTTP endpoints. The status is where your debugging lives — note how the revision carries both the resolved tag and the digest, and how the digest differs from the tag as identity:

$ kubectl get ocirepository app
NAME   URL                                        AGE   READY   STATUS
app    oci://ghcr.io/my-org/manifests/app         5s    True    stored artifact with revision
       'staging@sha256:4f90664660b3a567287e6957fa0481f347541b5908f6f797ec665255a399aed6'

Status:
  Artifact:
    Digest:     sha256:4f90664660b3a567287e6957fa0481f347541b5908f6f797ec665255a399aed6
    Revision:   staging@sha256:4f90664660b3a567287e6957fa0481f347541b5908f6f797ec665255a399aed6
    Size:       1105
  Conditions:
    Reason:     Succeeded
    Status:     True
    Type:       Ready
    Type:       ArtifactInStorage

Downstream, a Kustomization consumes it exactly as it would a Git source — the API is interchangeable:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: app
  namespace: flux-system
spec:
  interval: 10m
  retryInterval: 5m
  path: ./
  prune: true
  wait: true
  timeout: 2m
  targetNamespace: default
  sourceRef:
    kind: OCIRepository
    name: app

Cloud Registry Auth Without Static Secrets

This is where artifact-based delivery pulls decisively ahead of deploy-key GitOps. ECR, GCR/Artifact Registry, and ACR all support Workload Identity, so the pull-side credential can be an identity assertion rather than a stored secret — and the ECR token model makes anything else a maintenance burden anyway: ECR authorization tokens minted by GetAuthorizationToken are valid for 12 hours. A static pull secret for ECR is a secret that silently rots twice a day unless you build rotation jobs for it.

# Patch for the flux-system kustomization: bind the
# source-controller ServiceAccount to an IAM role.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
patches:
  - patch: |
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: source-controller
        annotations:
          eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/flux-source-controller
    target:
      kind: ServiceAccount
      name: source-controller

The IAM role needs only the AWS-managed policy AmazonEC2ContainerRegistryReadOnly. On the source side, provider: aws then makes the controller assume the role, mint 12-hour ECR tokens as needed, and refresh them on its own schedule — rotation is the controller's job, not a cron job you own. The equivalents: on AKS, provider: azure with Workload Identity (the controller's ServiceAccount annotated with azure.workload.identity/client-id and labeled azure.workload.identity/use: "true"; if the cluster has multiple identities, set AZURE_CLIENT_ID on the controller deployment); on GKE, provider: gcp with Workload Identity (ServiceAccount annotated iam.gke.io/gcp-service-account, needing artifactregistry.repositories.downloadArtifacts from the Artifact Registry Reader role).

For self-hosted registries there is an even newer option: since Flux v2.10, object identity is generally available — the controller can authenticate with a short-lived Kubernetes ServiceAccount token. A ServiceAccount opts in with the annotations credential.auth.fluxcd.io/provider: kubernetes and credential.auth.fluxcd.io/type: jwt, optionally pinning audiences and a basic-auth username; the registry side verifies the token's signature (or a SPIFFE identity, v2.10's headline feature) — again, no static credential in the cluster.

Helm Charts: Killing the index.yaml Problem

Helm OCI is not a side note — for many platform teams it is the first artifact-based migration they make, because classic HTTP Helm repositories have a structural flaw the Flux Mirror blog quantified bluntly: every client must download and parse one big index.yaml to resolve a single chart version, and those indexes range from tens of kilobytes to several megabytes — the frozen Bitnami legacy catalog's index is 27 MB. Your GitOps controllers re-download that index on every reconcile of every chart. OCI charts have no index: each chart version resolves by name and tag like any other artifact, which is why the Flux docs state plainly that OCIRepository sources perform significantly better than HTTP HelmRepository sources.

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: podinfo-chart
  namespace: default
spec:
  interval: 10m
  url: oci://ghcr.io/stefanprodan/charts/podinfo
  layerSelector:
    mediaType: "application/vnd.cncf.helm.chart.content.v1.tar+gzip"
    operation: copy
  ref:
    semver: ">=6.9.0"
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: podinfo
  namespace: default
spec:
  interval: 10m
  releaseName: podinfo
  chartRef:
    kind: OCIRepository
    name: podinfo-chart
  values:
    replicaCount: 2

The layerSelector with operation: copy is required for chart artifacts — the Helm chart layer must be stored intact for helm-controller to install from it. Since Flux v2.10, the HelmRelease can also declare the OCIRepository inline under spec.chart (the new spec.chart.kind field), letting helm-controller create and manage the source object in the release namespace for you.

Major community charts — prometheus-community, grafana, cert-manager — already publish OCI chart versions for exactly these reasons, but many upstreams have not. The Flux Mirror answer to "upstream still serves HTTP only" is to stop waiting: mirror the chart into your own registry as a deterministic OCI artifact (Vault's HTTP-published chart mirrored into GHCR in the Mirror docs' own example), point the HelmRelease at your copy, and, in the blog's words, retire your HTTP HelmRepository objects entirely so that clusters only ever watch registries you own.

The Argo CD Side: OCI Sources and the 3.6 Generator Wave

Argo CD's artifact support arrived on a slower track than Flux's — OCI sources landed as a beta in June 2025 (PR #18646), got private-registry documentation in December 2025, and hardened through the 3.x line. But the November 3, 2026 release of Argo CD v3.6.0 is the milestone: it shipped the ApplicationSet OCI generator (#26121), added GHCR webhook support for that generator, and fixed the OCI tag-list caching behavior — a release that treats registries as first-class fan-out sources, not just an exotic alternative to Git.

OCI as an Application source

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: platform-config
  namespace: argocd
spec:
  project: default
  source:
    path: .
    repoURL: oci://registry-1.docker.io/my-org/platform-config
    targetRevision: 1.16.1
  destination:
    server: "https://kubernetes.default.svc"
    namespace: platform

The mechanics mirror Git sources closely: repoURL uses the oci:// scheme (registry plus repository name, no tag), targetRevision is the tag or digest, and path selects a subdirectory inside the expanded artifact — . for the whole thing. For Helm charts packaged as OCI artifacts, the same oci:// source works with a helm stanza, and Argo CD detects chart artifacts by their media type (application/vnd.cncf.helm.chart.content.v1.tar+gzip), in which case the path must be ..

Credentials are repository-server-side, like Argo CD's Git creds:

# A private OCI repository used for manifest artifacts:
argocd repo add oci://registry-1.docker.io/my-org/platform-config \
  --type oci --name platform --username "$REG_USER" --password "$REG_PASS"

# A registry served over plain HTTP must be registered even with
# no credentials, or Argo CD attempts TLS and the connection fails:
argocd repo add registry.internal.lan:5000/my-org/platform-config \
  --type oci --insecure-oci-force-http

# Helm OCI repositories take the Helm type with --enable-oci,
# and the URL carries NO oci:// prefix:
argocd repo add registry-1.docker.io/bitnamicharts/nginx \
  --type helm --enable-oci --name stable \
  --username "$REG_USER" --password "$REG_PASS"

The declarative equivalent of insecure-oci-force-http is insecureOCIForceHttp: "true" in the repository Secret. On the Helm path, remember the two conventions the docs call out: for --type helm --enable-oci the repository URL must not contain the oci:// prefix, and the artifact path moves into the Application's path attribute.

The ApplicationSet OCI generator: registry-driven fan-out

The generator (v3.6.0) is the structural twin of the Git directory generator: "it works similarly to the Git generator but uses OCI artifacts as the source," in the docs' own words — purpose-built for distributing pre-rendered or hydrated Kubernetes manifests as versioned, immutable artifacts. Push one artifact containing a directory of apps with oras push ghcr.io/my-org/platform:v1.2.3 ., and a directory-pattern generator fans out one Application per directory:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: platform-apps
  namespace: argocd
spec:
  generators:
    - oci:
        repoURL: oci://ghcr.io/my-org/platform
        revision: v1.2.3
        directories:
          - path: "platform/*"
  # The template stanza maps each discovered directory to an
  # Application via Go template variables (the docs example
  # derives each Application name from the directory basename).

The generator also has a file variant for parameter-driven fan-out (elements read from a file inside the artifact), and v3.6 can trigger on registry webhooks — the webhook controller matches Helm OCI apps by repoURL and chart against registry events, with GHCR support documented for the OCI generator. That closes the loop: a CI push to the registry creates Applications without any Git transaction, with the artifact digest as the promotion unit.

Flux Mirror: Relocation as a Declarative Pipeline

So far this is a story about your artifacts. The other half of the shift is dependencies — the charts and images your platform consumes from registries you do not operate. The Flux project's framing in the Introducing Flux Mirror post is worth quoting directly: "When you put an image from a registry you don't operate into your Deployment, you make that registry's uptime, rate limits, and retention policy part of your production architecture." The industry's recent history makes the point concrete: Docker Hub's rate-limit changes in 2020 and 2025, Broadcom's freeze of the free Bitnami catalog in August 2025, and Bitnami's own charts-syncer relicensing in 2026 to a Broadcom license permitting use only with Broadcom products.

Flux Mirror is the v2.9 plugin-system answer: a declarative, Apache-2.0-licensed relocation layer that copies container images, Helm charts, and OCI artifacts between registries. Install it through the CLI plugin system (RFC-0013) — pinned to an immutable digest for CI:

flux plugin install mirror

# For CI pipelines and production, pin the plugin to a digest
# taken from the audited release:
flux plugin install mirror@sha256:<digest-of-audited-release>

The configuration is a statement of what your registry should contain — the same declarative principle that makes GitOps work, applied to relocation. It is reviewable in a pull request, and because the sync is idempotent, it is safe to run on a schedule:

apiVersion: mirror.plugin.fluxcd.io/v1beta1
kind: Config
artifacts:
  - source: docker.io/stefanprodan/podinfo
    destination: quay.io/my-org/podinfo
    selector:
      semver: "6.x"
      limit: 2
    includeReferrers: true
    verify:
      provider: cosign
      minAge: 48h
      matchOIDCIdentity:
        - issuer: https://token.actions.githubusercontent.com
          subject: "^https://github.com/stefanprodan/.*$"
charts:
  - name: vault
    source: https://helm.releases.hashicorp.com
    destination: oci://ghcr.io/my-org/charts
    version: "0.34.x"
    limit: 3
hosts:
  - host: ghcr.io
    username: my-org
    credential:
      value: "$GH_TOKEN"  # substituted from the environment at sync time

Read that verify block carefully, because it is the most consequential policy surface in the whole pattern. A tag is copied only if its keyless signature resolves to a certificate from the configured issuer whose subject matches your pattern — the artifact must be provably built by your workflow, not merely signed by someone. And minAge is a supply-chain diode: nothing younger than 48 hours crosses into your registry, with the age measured against the Rekor transparency log's countersigned integration timestamp — not the publisher-controlled image config timestamp, which an attacker can set and reproducible builds make meaningless anyway.

$ flux mirror sync flux-mirror.yaml
(ghcr.io/stefanprodan/podinfo:6.14.0)  copied    -> localhost:5000/apps/podinfo:6.14.0
(ghcr.io/stefanprodan/podinfo:6.14.1)  copied    -> localhost:5000/apps/podinfo:6.14.1
(ghcr.io/stefanprodan/podinfo:6.15.0)  skipped   -> signature-too-new
Summary: 2 copied, 1 skipped in 8.443s

Exit codes:  0 = clean sync
            1 = any tag failed
            2 = a tag drifted upstream and overwrite is disabled

JSON report:  flux mirror sync flux-mirror.yaml -o json
              -> .report.results[].tags  (with "age" vs "minAge" reasons)

Re-running a sync reports already-current tags as skipped, and a tag that changed upstream reports as drifted — it is not silently overwritten, and the exit code fails your CI by default (overridable with --drift-exit-code). Two companion commands remove the usual shell-scripting pain: flux mirror login writes registry credentials to the Docker config without needing the Docker client in CI, and flux mirror secret resolves the same hosts credentials — including short-lived Workload Identity tokens — and upserts them into a dockerconfigjson Secret that serves both Pod imagePullSecrets and OCIRepository secretRef, scheduled next to the sync so pull credentials stay rotated.

Why the minimum-age diode matters is not hypothetical. The Mirror post assembles the 2025–2026 supply-chain record: the Shai-Hulud npm worm compromised 500+ packages in September 2025; a May 2026 successor wave pushed 639 malicious versions across 323 packages in a single hour; in February 2026 attackers stole an Aqua Security CI token and used it to publish a malicious Trivy release in March, force-pushing 76 of the 77 trivy-action tags used in CI pipelines everywhere (CVE-2026-33634) — the malicious artifacts were live for days before remediation completed. One analysis found 8 of 10 major attacks had exploit windows under a week, and the ecosystem is converging on cooldowns as a result (pnpm ships a 24-hour minimumReleaseAge by default; npm and Renovate added equivalents). A registry-side minAge gives your GitOps source of truth the same protection, enforced at the moment of mirroring rather than at scan time.

Production Architecture: The Full Topology

Here is the complete shape, end to end. Git remains the authoring plane — humans still write YAML, review it in PRs, and merge it. What changes is that clusters no longer fetch from Git: CI renders and publishes, and the registry is the only thing the fleet watches.

flowchart LR
    DEV["Engineers + PR review"] --> GIT["Git repository - authoring plane"]
    GIT --> CI["CI: render, sign, flux push artifact"]
    CI --> REG["OCI registry - delivery plane"]
    UP["Upstream registries and HTTP chart repos"] --> MIR["Flux Mirror sync - verify, minAge, referrers"]
    MIR --> REG
    REG -->|"single GET, digest-verified pull"| SC["source-controller"]
    SC --> KS["Kustomization / HelmRelease reconcilers"]
    KS --> C1["staging cluster - ref.tag: staging"]
    KS --> C2["production cluster - digest-pinned"]
    REG -.->|"registry event webhook"| REC["Flux Receiver - HMAC or OIDC"]
    REC --> SC

Staging tracks the tip of development:

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app-staging
  namespace: flux-system
spec:
  interval: 5m
  url: oci://ghcr.io/my-org/manifests/app
  ref:
    tag: staging
  verify:
    provider: cosign
    secretRef:
      name: cosign-pubkeys

Production releases are cut by semver, exactly as the Flux docs' workflow prescribes: CI pushes production manifests tagged with the Git release tag, and the production source resolves the newest matching version:

# In CI, on a Git tag push: build production manifests and
# push them with the release tag as the artifact tag.
flux push artifact oci://ghcr.io/my-org/manifests/app:"$GIT_TAG" \
  --path="./rendered/production" \
  --source="https://github.com/my-org/app.git" \
  --revision="$GIT_TAG@sha1:$GIT_SHA"

apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: app-prod
  namespace: flux-system
spec:
  interval: 10m
  url: oci://ecr-account.dkr.ecr.eu-west-1.amazonaws.com/my-org/app
  ref:
    semver: "6.x"
  provider: aws
  verify:
    provider: cosign
    secretRef:
      name: cosign-pubkeys

For high-change environments, replace interval polling with a webhook: registries emit push events, and Flux Receivers turn them into immediate reconciliations. The Receiver API has native types for Harbor, Docker Hub, Quay, Nexus, Azure Container Registry, and Google Container Registry, plus generic types — and since v2.9, generic Receivers can be secured with OIDC instead of a shared HMAC secret:

apiVersion: notification.toolkit.fluxcd.io/v1
kind: Receiver
metadata:
  name: app-registry-events
  namespace: flux-system
spec:
  type: generic-hmac
  events:
    - "push"
  secretRef:
    name: registry-webhook-token
  resources:
    - apiVersion: source.toolkit.fluxcd.io/v1
      kind: OCIRepository
      name: app-staging
      namespace: flux-system

Test the trigger path from the CLI without crafting HTTP by hand — flux trigger receiver (also new in v2.9) sends the signed request for you. And when your fleet spans trust boundaries, the Mirror post describes the standard topology: CI builds, signs, and pushes to a central registry; mirror pipelines relocate artifacts — together with their signatures and attestations — into each environment's own registry; and clusters only ever pull from inside their own trust boundary. The same shape serves edge fleets and air-gapped estates.

Production Realities: Where Artifact-Based Delivery Breaks

This model is a net operational improvement for a specific kind of fleet — but it breaks in its own characteristic ways, and pretending otherwise is how architectures rot.

• The registry is now your control plane. You did not eliminate the single point of failure; you moved it from a vendor with a status page to a component you now own end-to-end. If you deploy from GHCR, you have kept GitHub's blast radius — Packages was not among the degraded components on November 4, but it is one company's operations. ECR de-correlates you from GitHub and inherits AWS's SLAs; a self-hosted zot or Harbor instance inherits your on-call rotation, which for most teams is worse than either vendor. Choose this deliberately.
• Token churn will outlive your enthusiasm. ECR authorization tokens are valid for 12 hours; a static pull secret for ECR rots twice daily. provider: aws (and the Azure/GCP equivalents) makes refresh the controller's job — but half-migrated estates that keep static dockerconfigjson Secrets while adopting OCI sources quietly replace one credential-rotation burden with a worse one. flux mirror secret exists precisely to automate this; use it or don't adopt the pattern halfway.
• Retention is a silent cost multiplier. Every CI run that pushes an artifact creates a manifest, config blob, and layer. Individually tiny; thousands of tags later, storage is billed per GB-month, and ECR's cold tier (ECR Archive) only helps for genuinely dormant artifacts — it carries a 90-day minimum storage duration plus a per-GB retrieval charge, and you cannot pull directly from it. The Mirror limit selector is not just hygiene; it is your cost control.
• Mutable tags are still mutable. The staging tag is a pointer; repointing it redeploys whatever it references on the next reconcile. That is the feature — and the risk when the push credential is compromised. Digest pinning for production plus verify policies is the mitigation; the semver prerelease skip footgun (-0 suffix to include RCs) is the config error you will make exactly once.
• You moved rendering to CI; own that. GitOps-from-Git could patch per-cluster overlays at reconcile time. Artifact delivery applies finished output, so per-environment differences must exist in the pipeline — the ArtifactGenerator API (Flux v2.9) is the monorepo bridge: a path pattern like @monorepo/apps/{app}/envs/{env} auto-discovers matching directories and generates one artifact source per match, with the named captures available as template variables.
• The review gap is real. Engineers review the pull request; the cluster applies the CI output. Closing that gap is exactly what the signature policy stack is for — verification at mirror time (who is allowed into your registry) and at reconcile time (what is allowed onto the cluster). If you skip verify and matchOIDCIdentity, you have made the review gap wider than Git ever was.
• Flux upgrades now touch your source of truth. v2.10 made the Flux artifact storage backend the default for the ImageRepository tag database — and switching it wipes the local tag database, forcing a rescan of every ImageRepository on first reconcile. Operational coupling like this is the price of the storage model; read upgrade notes as release blockers, not FYIs.
• Your compliance story needs re-education. "Where is the history?" is the first audit question. The honest answer: the review trail lives in the authoring repo's PRs; the deployment trail lives in digest pins, Rekor-countersigned signature timestamps, artifact source/revision metadata, and registry event logs. That is a stronger trail than commit history — but only after you have walked your auditors through it once.
• Git stays load-bearing anyway. The authoring repo, the PR reviews, and the CI that renders all still live in Git — Gitless GitOps means Git stops being the delivery transport, not that it disappears. Teams that expect to delete their Git provider will be disappointed; teams that want deploys to survive their Git provider's bad Tuesday get exactly that.

Who Should Skip This

• Small fleets with low change volume. Three clusters and a handful of deploys a day are already better-served by GitHub's uptime than by a registry you now have to operate. The added moving parts are pure negative ROI.
• Teams with no registry ownership. If nobody is on-call for the registry, you are trading a monitored dependency (a major Git host) for an unmonitored one (your own storage). This is a downgrade, not an upgrade.
• Heavy per-cluster overlay users without CI discipline. If cluster teams patch overlays at their own edge via in-cluster kustomize, artifact delivery centralizes rendering in a pipeline they don't control. That is an org-chart problem wearing a technical costume — fix the ownership first, or keep Git sources.
• Compliance regimes that hard-mandate Git history as the audit record. The signature-plus-registry-event trail is equivalent or stronger, but if your regulator's checklist says "commit history," you are buying yourself an audit exception, not an architecture.
• Self-hosted, healthy Git providers. If your GitLab CE instance on-prem is stable, the November-4 argument does not apply to you. The registry-side benefits (index.yaml relief, mirror policies) may still make Helm OCI worth it — that is the charts-first path below, and it is a legitimate endpoint, not a stopgap.

Migration Path: Strangler, Not Big Bang

Every step below is independently shippable and independently revertible. Do not skip the ordering — the steps are sequenced by blast radius.

• Step 1 — Mirror your Helm charts. Stand up Flux Mirror on a schedule against one or two HTTP chart repositories, republish them as OCI artifacts in your registry, and switch the matching HelmRelease to a chartRef. You have just eliminated the index.yaml downloads for those charts with near-zero risk — nothing app-facing changed.
• Step 2 — Publish, don't consume. Add flux push artifact to one app's CI as a shadow step. Compare the rendered artifact against what the Git-based flow deploys. This is where you discover your in-cluster patches, kustomize ordering quirks, and CI/Git version drift — cheaply.
• Step 3 — Consume for staging. Add an OCIRepository plus Kustomization for that app in the staging cluster while the Git source keeps running. The two sources produce identical artifacts or you have found a bug — run this parallel period long enough to cover your release cadence.
• Step 4 — Add the trust layer. Sign artifacts in CI with cosign, add the verify block, then add the Mirror-side matchOIDCIdentity policy for everything you relocate. An unsigned artifact should fail a staging reconcile before you ever cut prod over.
• Step 5 — Cut production over by digest. Pin the production source to a digest and let promotion become a one-field YAML diff in the authoring repo — which is now a deliberately thin GitOps-of-the-registry-objects layer, bootstrapped however you already bootstrap Flux.
• Step 6 — Segment trust boundaries. When the pattern holds, mirror into per-environment registries (the Mirror CronJob examples with EKS IRSA, AKS Workload Identity, and GKE Workload Identity are in the plugin docs) and retire the Git-based sources for migrated apps. Your fleet now reconciles from registries inside each environment's boundary — and a Git host outage is, at last, someone else's incident.

References & Further Reading

• GitHub Status — Incident with Pull Requests, Git Operations and Issues (Nov 4, 2026)
• Hacker News — incident discussion thread (265 points)
• Flux — OCIRepository API reference
• Flux — OCI Artifacts cheatsheet
• Flux — flux push artifact command reference
• Flux — Introducing Flux Mirror (Aug 2026)
• Flux — Announcing Flux v2.9 GA (plugin system, Mirror, Schema)
• Flux — Announcing Flux v2.10 GA (object identity GA, OCI chart sources)
• Flux — GitHub Action for pushing OCI artifacts in CI
• Argo CD — OCI application sources
• Argo CD — ApplicationSet OCI Generator
• OpenGitOps — GitOps Principles v1.0.0 (CNCF App Delivery TAG)
• GitHub Docs — Rate limits for the REST API
• AWS — ECR GetAuthorizationToken (12-hour token validity)
• AWS — Amazon ECR pricing (Standard vs Archive storage classes)
• fluxcd/flux-mirror — declarative mirroring plugin
• Flux RFC-0013 — CLI Plugin System
• zot — open-source OCI-native registry (OCI 1.1 referrers)