Koreo Review: The Meta-Controller Language That Froze at v0.1.17
Sources
- Koreo documentation (overview, Workflow, ResourceFunction, FunctionTest, CLI)
- Show HN: Koreo launch thread (April 2025, author comments)
- koreo-dev/koreo-core (engine source, v0.1.17, Apache-2.0)
- koreo-dev/koreo-controller (controller source, v0.1.18)
- Koreo scheduler defaults (controller source, this review's hands-on run)
- koreo and koreo-core on PyPI (release timestamps verified 2026-11-10)
- Crossplane v2: What's New (namespaced XRs/MRs, claims removed)
- Crossplane v2 upgrade guide (v2.4.3, released 2026-10-22)
- kro (Kube Resource Orchestrator), kubernetes-sigs
- Konfigurate, the commercial platform powered by Koreo
- Koreo vs Crossplane comparison page (vendor claims audited in this review)
The Verdict, Up Front
Koreo is the most interesting abandoned project in the Kubernetes platform-engineering space. That is not a throwaway line — it is the review. The engine Real Kinetic built here is genuinely clever: a meta-controller programming language that treats Kubernetes control loops as composable primitives, with a testing construct (FunctionTest) that no competing tool has matched even now. We downloaded it, installed it, read every line of its reconcile path, and executed its real engine offline against its own shipped test suites. The mechanics hold up.
The project does not. The last commit to koreo-core landed on 2025-08-09 — fifteen months before this review — and the last PyPI release of the koreo CLI predates that by two months, pinning an even older engine underneath it. The CLI crashes on startup on a clean Python 3.14 install today (we reproduced it; the fix is a one-liner, but nobody has shipped it in fifteen months). Meanwhile Crossplane v2 — whose v1 pain points are the explicit justification for Koreo's existence — shipped namespaced resources and arbitrary-resource composition as core features, and kro, the closest architectural cousin, moved to kubernetes-sigs and is merging commits daily.
So this review does two jobs at once. It documents the engineering — exact loop mechanics, state handling, retry semantics, failure modes, all verified from source and from real execution — because the ideas deserve it, and because a fork-and-own adoption is the only sane way to run Koreo in 2026. And it documents the procurement decision, which is straightforward: do not adopt this as a dependency for a production platform unless you are prepared to become its maintainer.
| Dimension | Assessment | Evidence |
|---|---|---|
| Core concept | Excellent — control loops as first-class programmable units | Workflow/Function model, source-verified |
| Testing story | Best-in-class idea — sequential control-loop simulation with variant isolation | FunctionTest ran offline, real output below |
| Drift handling | Correct convergent patch loop, three-way-merge against last-applied | Reproduced: exact diff message and patch payload |
| Production hardening | Weak — unbounded user retries, PermFail parking, no managed-resource watches | Scheduler source + measured hot loop |
| Project health | Dormant 15 months, 99 stars, 3 contributors, one commercial consumer | GitHub/PyPI timestamps, 2026-11-10 |
| Competitive position | Outflanked — its two headline differentiators are Crossplane v2 core features | Crossplane v2.4 docs, author's own HN comments |
| Bottom line | Study it, steal the FunctionTest pattern, fork if you must — do not adopt as-is | — |
What Koreo Actually Is
Koreo bills itself as a "platform engineering toolkit for Kubernetes," which undersells and misdirects at the same time. Strip the positioning and you get a precise, unusual thing: a data-structure orchestration engine that programs other controllers. You define Workflows that trigger on any CRD; each Workflow's steps call Functions; each ResourceFunction owns one resource's full CRUD lifecycle. Built-in controllers (Deployment, StatefulSet), off-the-shelf operators (AWS ACK, GCP Config Connector), and your own operators are all just things a ResourceFunction manages. There is no provider ecosystem, no function marketplace, no mandatory packaging — if a controller exists in the cluster, Koreo can orchestrate it.
Three resource kinds carry the whole system, plus one that is the actual differentiator:
| Kind | Role | Purity |
|---|---|---|
| Workflow | Declares a trigger (any CRD via crdRef) and a DAG of steps; execution order is derived from which steps reference which | Orchestrator — the controller itself |
| ValueFunction | Pure computation: validate, reshape, compute. CEL in, CEL out, no side effects | Pure function |
| ResourceFunction | Owns one managed resource: read it, diff it, patch/recreate it, wait on its status, return values from it | Side-effect boundary — the only place the API server is touched |
| FunctionTest | Unit tests for any Function: sequential test cases model successive iterations of the control loop, with an in-memory fake API | The killer feature |
The configuration-management layer — ResourceTemplates (base definitions loaded by name), atomic overlays, skipIf conditionals, forEach loops, and a refSwitch construct that dispatches to different Logic per case — is Kustomize's grammar with computation injected at every seam. Where Kustomize overlays static patches, Koreo overlays evaluate CEL expressions with access to inputs, locals, and the resource being built. The expression surface is real CEL (via cel-python — the engine is 100% Python, ~378 KB of it) extended with self_ref(), to_ref(), overlay(), to_json/from_json, split/strip/replace, b64encode/b64decode, and a config_connect_ready() helper that checks another controller's Ready condition. Two compare directives — x-koreo-compare-as-set and x-koreo-compare-as-map — fix the classic "controller reordered my list so now I fight it forever" problem by changing diff semantics for named arrays, and are stripped before anything reaches the API server.
Everything is namespaced. Functions, Workflows, templates, and the resources they manage all live in namespaces — this was one of the two founding grievances against Crossplane v1, whose managed resources were cluster-scoped. Hold that thought; we will come back to it with a knife.
The Programming Model: Five Outcomes and a Wall Clock
Every Function evaluation ends in exactly one outcome, and the outcome lattice is the deepest design decision in the codebase (koreo/result.py):
severity (result.combine picks the rightmost that occurred):
DepSkip ──► Skip ──► Ok ──► Retry ──► PermFail
(lowest) (highest)
DepSkip this step never ran because a dependency wasn't ready
Skip deliberately disabled (config said so) — not an error
Ok succeeded; carries a return value consumed by later steps
Retry not converged yet; re-run this Workflow after N seconds
PermFail unrecoverable; stop retrying entirely until something changesThe semantics this buys are worth spelling out, because they are the difference between Koreo and a shell script with retries:
- Retry is a value, not a crash. A ResourceFunction that just created or patched a resource returns
Retry(delay: 30)— the next reconcile pass re-reads the resource and checks readiness. Convergence is the loop, not an event handler. - Failure propagates as silence, not noise. If a step fails, every step that depends on it is skipped with
DepSkip— not errored. The workflow's overall outcome collapses to the worst surviving result, and dependent steps retry on the next pass without side effects from half-finished work. - PermFail is a hand brake. Validation failures — bad inputs, forbidden values — return PermFail and the controller parks the trigger. No retry storm, no backoff needed: the workflow stays failed until the trigger's spec or the Workflow definition actually changes. (This is also a loaded footgun; see the failure-mode section.)
Workflow steps declare inputs as CEL expressions over =parent (the trigger resource), =steps.<label>.* (prior results), and =inputs.*. Dependency order is inferred from those references — you must list referenced steps first, but listing order does not guarantee execution order; independent steps run concurrently in an asyncio.TaskGroup. Sub-Workflows compose the same way, and Workflows can even generate other Workflows (the HN thread calls this "meta" Workflows — "you could implement something like kro itself in Koreo," per the author). State surfaces two ways: step state: blocks merge into the trigger's status.state, and step condition: blocks write typed conditions into status.conditions — with a warning in the docs that type collisions are intentionally not enforced.
There is one constant you must internalize before writing a single Workflow: the entire step DAG gets a hard-coded 10-second wall-clock budget per execution (workflow/reconcile.py: STEP_TIMEOUT = 10, next to the comment # TODO: What is reasonable here? Perhaps 10 seconds?). Steps that don't finish are cancelled and converted to Retry(delay: 30) — for the whole workflow. It is not configurable at any level: not per Workflow, not per controller, not per step. For fast, healthy API servers this never matters. On a throttling or overloaded API server, it matters a great deal.
The Reconcile Engine, Read from Source
The controller is a single-replica Python process (uvloop, kr8s async client) with four moving parts. Everything below is from the v0.1.18 controller and v0.1.17 engine sources, cross-checked against live behavior where we could drive it offline.
┌───────────────────────────────────────────────────────────┐
│ koreo-controller (Python, uvloop) │
│ │
trigger CRs │ ┌────────────────────┐ sha256 dedup: spec + │
(any crdRef ───┼──► chief_of_the_watch │ labels + annotations │
kind, watch) │ │ one watchstander │ (minus koreo.dev/* and │
│ │ per kind │ kubectl.kubernetes.io/*) │
│ │ 600s forced restart│ │ │
│ │ abandon after 10 │ ▼ │
│ │ consecutive errors │ event_handler ──► scheduler │
│ └────────────────────┘ PriorityQueue │
│ 5 workers, 30s cap │
Functions, │ ┌────────────────────┐ │ │
Workflows, ───┼──► cache maintainers │ ▼ │
Templates │ │ list + watch, │ reconcile_resource │
(one ns per │ │ 900s restart, │ (per-trigger asyncio.Lock, │
kind, env │ │ crash controller │ PermFail parking, │
configured) │ │ after 10 errors) │ stuck-lock release 60s) │
│ └────────────────────┘ │ │
│ ▼ │
│ reconcile_workflow │
│ TaskGroup over steps, 10s wall clock total │
│ │ │ │ │
│ ValueFunction ResourceFunction sub-Workflow │
│ (pure CEL) (GET/diff/CRUD) (recursion) │
│ │ │
│ ▼ │
│ patch trigger status.conditions + state │
│ + koreo.dev/managed-resources annotation │
└───────────────────────────────────────────────────────────┘Scheduler: the exact numbers
The scheduler (controller/scheduler.py, Configuration defaults) is small enough to fully characterize, and we verified each number by running it:
| Parameter | Default | Behavior |
|---|---|---|
| Worker concurrency | concurrency: 5 | Five workflow executions in flight at once — for the entire controller |
| Steady-state frequency | frequency_seconds: 1200 | Every trigger re-reconciles every 20 minutes (the documented "every 20 minutes" cadence) |
| Per-execution timeout | timeout_seconds: 30 | Hard cap on one workflow execution; exceeded counts as a system error |
| System-error retries | retry_max_retries: 10 | Exponential backoff: min(2^n × 10s + jitter(0–30s), 300s), then the code hits a literal # TODO: Exhausted retries... what now? and drops the work item |
| User-requested retries | uncapped | A Function's retry: {delay: N} re-queues at exactly N seconds — no cap, no backoff, no jitter (open issue koreo-controller#23) |
| PermFail | drop | Logged, removed from the queue, and the trigger is refused re-reconcile until its spec or the Workflow changes |
| Step budget | STEP_TIMEOUT = 10 | Wall clock for the whole step DAG per execution (engine constant, not configurable); exceeded steps cancel into Retry(30s) |
Two of those rows are the headline production risks, and we measured both (hands-on section below): an uncapped user-retry loop, and the PermFail parking behavior. Read them together and you get the shape of a bad day: a workflow that returns Retry(delay: 1) hammers the API server every second forever, while consuming one of five total execution slots — five misbehaving triggers can starve every other workflow in the cluster. And because a non-409 ServerError during resource creation is classified as PermFail (more on that below), a single API-server blip can park a trigger indefinitely.
ResourceFunction: the CRUD state machine
Each ResourceFunction execution runs the same pipeline (koreo/resource_function/reconcile/__init__.py):
preconditions (CEL on inputs) ──fail──► Skip / Retry / PermFail / DepSkip
│ ok
▼
resolve apiConfig.name + namespace (CEL, always overlaid — you cannot
escape the declared apiVersion/kind, ever)
▼
GET managed resource
│
┌────┴─────────┐
absent present
│ │
create.enabled? readonly? ──yes──► return raw resource, no mutation
│ no ──► Retry("Waiting...", 30s)
▼ ▼ no
materialize Target Resource Specification:
resource (inline) or resourceTemplateRef (dynamic)
+ overlays list (inline or overlayRef ValueFunctions, skipIf-able)
+ forced overlay (apiVersion, kind, metadata.name, namespace)
│
▼
diff = validate_match(target, actual, koreo.dev/last-applied-configuration)
│
matches? ──yes + ownerRef present──► Ok(current resource) [steady state,
│ zero API writes — verified]
│ no / ownerRef missing
▼
update policy:
never ──► leave it alone (existence-only management)
patch ──► PATCH payload = full expected resource
│ ownerReferences appended if owned + same-namespace
└──► Retry(delay) — next pass re-reads and re-diffs
recreate ──► DELETE, then Retry(delay) — next pass re-creates
│
▼
postconditions (CEL on inputs, locals, resource — status.conditions
readiness checks live here) ──fail──► Retry(delay)
│ pass
▼
return value ──► downstream steps consume it; resource_id recorded
in the trigger's koreo.dev/managed-resources annotationThe diff is a three-way comparison against Koreo's own koreo.dev/last-applied-configuration annotation (kubectl's pattern, with its own key), which is how it distinguishes "field I own and someone changed" from "field another controller added." External edits inside the managed envelope get reverted — that is the point of a control loop. The patch payload is the complete expected resource, converted and sent through kr8s; we captured it verbatim in the hands-on section.
Creation races return HTTP 409 and degrade to a Retry — correct. Every other ServerError during create is classified PermFail. A 503 from an overloaded API server, a webhook failure, a transient network blip — all of it becomes "unrecoverable," which parks the trigger until a human edits the spec. Load failures (load_api_resource) take the opposite path and retry on ServerError. The classification is inconsistent by construction, and the source even flags it: # TODO: Review possible error codes and determine which should retry.
Event handling, watches, and where state lives
The controller watches trigger kinds only — one watchstander per CRD referenced by some Workflow's crdRef. Watches are force-restarted every 600 seconds, reconnect with exponential backoff (30s base, 900s cap), and after ten consecutive errors the watch is abandoned with an error log; triggers of that kind that have not yet been seen never get their first reconcile until the controller restarts. Events are deduplicated by a SHA-256 of spec + labels + annotations (excluding koreo.dev/* and kubectl.kubernetes.io/*), so status-only writes never re-trigger work — and neither does a spec change that a koreo annotation happens to shadow.
Managed resources are never watched. Drift on the things Koreo creates is detected only when a workflow runs — on a trigger event, a Retry's delay, or the 20-minute cadence. Open issue koreo-controller#38 ("Investigate watching for managed resource updates instead of polling") has sat untouched since May 2025. Compare that to Crossplane, where every managed resource is a watched, informer-cached object with its own controller.
All runtime state is in-memory: a module-level dict caches prepared Functions/Workflows keyed by name and invalidated by resourceVersion; another caches trigger resources with their reconcile locks and last outcomes. Durable state lives in exactly two places — the cluster (CRs are the source of truth, re-listed at startup, in dependency order: ResourceTemplate → ValueFunction → ResourceFunction → Workflow) and the trigger's annotations (koreo.dev/managed-resources is a recursive JSON map of every resource the workflow owns, which is what the Koreo UI renders as an instance graph). Hot reloading is hard-coded on: edit a Function and the cache maintainer re-prepares it, subscribers (anything that referenced it) are notified, and owning Workflows re-run — no restart, in prod or dev alike.
Hands-On: We Ran the Real Engine
No cluster was harmed (or used): Koreo's FunctionTest machinery runs the genuine reconcile path — prepare, CEL evaluation, diffing, patching — against an in-memory fake API, which is precisely the feature worth having. We installed the current public releases exactly as a user would and drove the engine directly.
Installing it in 2026: the CLI is broken out of the box
$ python3 -m venv /tmp/koreo-venv
$ /tmp/koreo-venv/bin/pip install koreo==0.3.3
$ /tmp/koreo-venv/bin/koreo -h
Traceback (most recent call last):
File "/tmp/koreo-venv/bin/koreo", line 3, in <module>
from cli.__main__ import main
...
File "/tmp/koreo-venv/lib/python3.14/site-packages/kr8s/_portforward.py", line 16, in <module>
import sniffio
ModuleNotFoundError: No module named 'sniffio'The current CLI release (0.3.3, published 2025-06-12) declares kr8s>=0.20.6 unpinned; eighteen months of transitive-dependency drift later, the resolved set no longer imports. pip install sniffio fixes it — a one-line patch that has not shipped because nothing has shipped. Note also what pip actually resolves: the CLI pins koreo-core==0.1.14 (June 2025) while the engine's last release is 0.1.17 (August 2025). The newest public tooling has never run the newest engine.
With sniffio installed, the CLI works and exposes four commands — apply, prune, inspect, reconcile:
$ koreo -h
usage: koreo [-h] {apply,inspect,prune,reconcile} ...
apply Apply updated .koreo/.k files as YAML via kubectl.
inspect Inspect a Koreo Workflow and resource hierarchy.
prune Prune unused koreo resource and value functions.
reconcile Reconcile a Koreo Workflow instance.Three observations from that surface: apply shells out to kubectl and tracks incremental applies with a .last-modified mtime file per directory (workable, but you are back to mtime-based idempotence); reconcile exists in the binary but not in the docs' CLI page; and — the one that matters — there is no koreo test command. FunctionTests run through the language server (IDE experience) or by importing koreo.function_test.run yourself. The project's flagship feature has no first-class CI entry point; wiring test runs into a pipeline means writing your own driver against library internals, exactly as we did.
Running Koreo's own test suites
The engine repository ships regression tests that are themselves documentation of failure modes. examples/patch-loop-testing.koreo contains a ResourceFunction whose create.overlay intentionally disagrees with its steady-state resource spec, under two different update policies. We loaded the file and ran both shipped FunctionTests through the installed engine:
=== shipped FunctionTest 'patch-looper.v1' ===
function_healthy: Ok(data=None)
fatal_error: False
[00] PASS Creation Values
[01] PASS Check return value | user skipped
[02] PASS Unexpected update Values
[03] PASS Stable return value
=== shipped FunctionTest 'recreate-looper.v1' ===
function_healthy: Ok(data=None)
fatal_error: False
[00] PASS Creation Values
[01] PASS Check return value | user skipped
[02] PASS Check the delete call | Delete called
[03] PASS Yet another create... | (retry)
[04] PASS Check the delete call | Delete calledRead case [02] through [04] of the second suite carefully, because it is the most honest artifact in the repository: with update.recreate and a create-time overlay that differs from the steady-state spec, the resource never converges — the function deletes and re-creates it, forever, every retry delay (15 seconds as shipped). The test suite doesn't flag this as a failure because the test suite is showing you the loop; detecting it is the author's job. In production there is no circuit breaker for this churn — no max-recreate counter, no anomaly detection, just a delete/create cycle until someone notices. The FunctionTest framework makes such loops visible before they ship; nothing at runtime stops them after.
Drift correction: the exact message and the exact patch
We built a ResourceFunction managing a Deployment (target: 3 replicas, one label), then handed it a cluster resource that had drifted to 9 replicas plus an extra label, as if edited by hand:
E3 drift outcome -> Retry(delay=30,
message=Patching `Deployment:prod:web` due to 'spec' -> 'replicas'
-> <target:[3], resource:[9]>.)
E3b steady-state outcome -> Ok(replicas: 3)
E3b api called: False (no patch attempted — converged, no write)The diff rendering is genuinely good — the path and both values, in one line, plus the delay until the next check. The patch payload sent to the API server is the complete expected resource (metadata with ownerReferences pointing at the trigger, spec with the target state), with Koreo's last-applied annotation attached and its compare directives stripped. On the second iteration the function detects a match and makes zero API writes — the loop is convergent, not churning. This is the part of Koreo that simply works.
Measuring the retry semantics
The scheduler is a standalone asyncio component, so we ran the real one against stub work processors. First, the uncapped user-retry loop — a function returning Retry(delay: 1):
### REAL scheduler - user-retry hot loop (Retry delay=1, NO cap)
executions in ~8s: 8 | user_retries counter reached: 7
intervals (s): [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0]
### REAL scheduler - PermFail is dropped, never requeued
executions in ~6s after PermFail: 1
### REAL scheduler - Ok requeues at frequency_seconds
executions: 1, ok_frequency_seconds seen: [1200]Three verified facts in one experiment: a stuck Retry(1) is a permanent one-second reconcile loop (86,400 executions/day per trigger, each doing real API reads), with no cap and no backoff; a PermFail is executed exactly once and dropped, parked until the trigger spec changes; and a converged workflow settles to the 20-minute cadence. Five hot-looping triggers consume all five worker slots — the concurrency default means a handful of badly written postconditions can wedge an entire controller.
Error messages and the validation gap
E4a runtime type error (inputs.int + "string") -> PermFail:
Error evaluating `bad-runtime.v1:spec.return` (at inputs.int + "string")
("found no matching overload for 'addition_add' applied to
(<class 'celpy.celtypes.IntType'>, <class 'celpy.celtypes.StringType'>)",
<class 'TypeError'> ("int() argument must be a string, a bytes-like
object or a real number, not 'NotImplementedType'"))
E4b parse error at prepare time -> PermFail:
Parsing error at line 1, column 14. '[inputs.int +=]' in spec.return
E4c missing input key -> PermFail:
Error evaluating `missing-input.v1:spec.return`
(at inputs.does_not_exist) ("no such member in mapping:
'does_not_exist'")Locations are precise and the parse error even draws a caret. But the runtime errors leak the Python implementation — celpy class names, NotImplementedType internals — because the engine is Python. The controller is a Python process holding the loop policy for your platform; that is a real operational constraint (image size, GIL, dependency surface) that the "language" framing hides.
The nastier finding is validation asymmetry. The shipped CRD schema contains CEL rules — for example, step labels must match ^[[:word:]]+$ — but client-side prepare does not enforce them. We fed the engine a Workflow with a step labeled not a valid label!; schema.validate() returned no error and the Workflow prepared with an ErrorStep that merely retries on a missing function reference. The API server will reject what the local tooling accepts: tests can pass on YAML that the cluster will refuse, and you find out at apply time. For a project whose entire thesis is "configuration as testable code," a validation gap between the test environment and the runtime is a credibility problem.
The Pattern in Practice
One worked example conveys the model faster than any architecture prose — a Workflow exposing a "bucket" API, the ResourceFunction doing CRUD, and the FunctionTest proving the loop. Shapes are from the shipped docs and examples; we ran equivalents of all three offline:
apiVersion: koreo.dev/v1beta1
kind: Workflow
metadata:
name: s3-bucket-workflow.v1
namespace: platform
spec:
crdRef: # any resource of this kind now triggers this Workflow
apiGroup: demo.koreo.dev
version: v1beta1
kind: BucketRequest
steps:
- label: config
ref: {kind: ValueFunction, name: bucket-config.v1}
inputs:
metadata: =parent.metadata
- label: bucket
ref: {kind: ResourceFunction, name: s3-bucket-factory.v1}
inputs:
metadata: =steps.config.metadata
condition: {type: BucketReady, name: S3 bucket}
state:
bucket_arn: =value.arn
---
apiVersion: koreo.dev/v1beta1
kind: ResourceFunction
metadata:
name: s3-bucket-factory.v1
namespace: platform
spec:
apiConfig:
apiVersion: s3.services.k8s.aws/v1alpha1 # managed by the ACK operator
kind: Bucket
name: =inputs.metadata.name
namespace: =inputs.metadata.namespace
resource: # steady-state Target Resource Spec
spec:
region: =inputs.metadata.annotations["demo.koreo.dev/region"]
create: {delay: 15} # wait after create before re-check
update: {patch: {delay: 30}} # diff → patch → re-check in 30s
postconditions:
- assert: =has(resource.status.ackResourceMetadata.arn)
retry: {delay: 5, message: Bucket not ready}
return:
arn: =resource.status.ackResourceMetadata.arn
---
apiVersion: koreo.dev/v1beta1
kind: FunctionTest
metadata:
name: s3-bucket-factory.v1
namespace: platform
spec:
functionRef: {kind: ResourceFunction, name: s3-bucket-factory.v1}
inputs:
metadata: {name: test-bucket, namespace: tests}
testCases:
- label: Initial create
expectResource: # asserts the exact patch payload
apiVersion: s3.services.k8s.aws/v1alpha1
kind: Bucket
metadata: {name: test-bucket, namespace: tests}
spec: {region: us-east-1}
- label: Simulate ACK reporting ready
overlayResource: # fake the other controller
status: {ackResourceMetadata: {arn: "arn:aws:s3:::test-bucket"}}
expectReturn: {arn: "arn:aws:s3:::test-bucket"}Everything notable is visible there: the Workflow never talks to the API server itself, the ResourceFunction declares its steady state and lets the loop close the gap, and the test simulates the ACK operator's status writes with an overlay — the two-controller dance that is normally an integration-test nightmare is a unit test here. This is the pattern worth extracting from Koreo regardless of what you run in production.
The Comparison That Aged Badly
Koreo's own comparison page frames Crossplane's fundamental limitations like this: Crossplane "generally requires resources to be cluster-scoped and live within the same cloud project or account as the control plane itself," and pushes you into "needing to implement the Crossplane API or using Crossplane-specific operators." In April 2025 — when the page was written and the HN thread ran — that was a fair reading of Crossplane v1. The Koreo author's own reply in that thread is the timestamp on the blind spot: "I thought Crossplane v2 was only a design proposal at the moment... But I guess there is an actual preview implementation now?"
Crossplane v2 shipped, and as of this review the current release is v2.4.3 (2026-10-22). The four headline changes, straight from the official "What's New in v2" documentation:
- Composite resources are namespaced by default — XRDs gained a
scopefield defaulting toNamespacedin the v2 XRD API; claims, the v1 workaround for exactly the cluster-scoping problem Koreo was built to escape, are gone ("with namespaced XRs there's no longer a need for claims"). - Managed resources are namespaced too — a namespaced XR can compose Crossplane MRs, plain Kubernetes resources, and third-party CRs alike ("compose any resource, not just Crossplane resources").
- Composition supports any Kubernetes resource — the second Koreo founding grievance (provider-locked composition) is also a v2 core feature.
- Operations — Cron and Watch operations give Crossplane workflow-ish triggers on resources and schedules, encroaching on the "operational workflows" territory that was Koreo's remaining differentiator.
The Crossplane upgrade guide even documents a clean, backward-compatible path from v1 (v1.20 → v2, one minor at a time, v1-style resources keep working via a LegacyCluster scope). Meanwhile Koreo's comparison page is unchanged since May 2025. Both of Koreo's founding differentiators against Crossplane are now Crossplane core features, documented better than Koreo documents itself, in a project with 12,232 stars and commits landing the day before this review. What is left of the contrast is real but narrow: Koreo orchestrates any existing operator directly (ACK, Config Connector, your in-house CRDs) with no provider packaging step, composes arbitrary control flow (forEach, refSwitch, sub-workflows), and tests reconciles as sequential simulations. Crossplane v2's composition pipeline is function-shaped and its testing story is nowhere near FunctionTest. Whether that narrow slice justifies a dormant dependency is the entire adoption question.
The kro comparison aged differently — in Koreo's favor on capability, and against it on everything else. kro's ResourceGraphDefinitions are genuinely less expressive (static resource templates, CEL value passing between RGD members, no orchestration outside the RGD, and — the vendor page is right about this — no reading of values from resources outside the graph). But kro now lives at kubernetes-sigs/kro: 3,090 stars, 389 forks, v0.9.4 released September 2026, commits merged the day before this review. The Koreo docs' own "when to choose kro" list ("your use case does not warrant the initial upfront complexity of Koreo") reads, in 2026, like an argument to choose kro for the 80% case and accept its limits.
| Capability | Koreo v0.1.17 | Crossplane v2.4 | kro v0.9.4 |
|---|---|---|---|
| Compose arbitrary K8s resources | Yes — any operator, no packaging | Yes (v2 core) | Yes (within RGD) |
| Read values from resources outside your abstraction | Yes — read-only ResourceFunctions | Limited (function pipeline) | No (documented limit) |
| Control flow (loops, switch, conditions) | forEach / skipIf / refSwitch / sub-workflows | Function pipeline + patches | CEL expressions only |
| Unit-test reconciles offline | FunctionTest (unique) | No equivalent | No equivalent |
| Namespaced resources | Yes (founding design) | Yes (v2 default) | Yes |
| Cloud APIs without writing an operator | Via ACK / Config Connector / any operator | Provider functions marketplace | Via ACK etc. inside RGDs |
| Watches managed resources for drift | No — re-read on next run (≤20 min window) | Yes — informers per MR | Yes |
| Project velocity (verified 2026-11-09) | Last commit 2025-08-09 | Commits daily, v2.4.3 out Oct 2026 | Commits daily, v0.10 in RC |
| Governance | Real Kinetic (3 contributors) | CNCF | kubernetes-sigs |
Where Koreo Breaks: The Degradation Conditions
Every item below was verified in source, and the ones we could drive offline were executed. These are the conditions under which Koreo stops being a clever control-plane toolkit and starts being an incident:
| Condition | Consequence | Where |
|---|---|---|
A Function returns Retry with a small delay forever (e.g., a postcondition that never goes ready) | Permanent hot loop — measured: 8 executions in 8s at delay=1, uncapped, no backoff; each pass does real API reads; five of these saturate the controller's five worker slots | scheduler.py Retry branch; open issue koreo-controller#23 |
| Non-409 ServerError during resource create (API-server blip, webhook failure, 503) | Classified PermFail → trigger parked with no retries until a human edits the spec — while the equivalent error on resource load retries. Inconsistent by design | resource_function/reconcile, # TODO: Review possible error codes |
| Any step exceeds the 10-second wall clock (throttled API server, large CRDs, slow CEL on big overlays) | Entire step DAG cancelled, workflow retried at 30s — potentially forever; budget is a module constant, not configurable | workflow/reconcile.py STEP_TIMEOUT = 10 |
create.overlay disagrees with steady-state resource under update.recreate | Infinite delete/create churn (the repo ships this as a demo); no convergence circuit breaker exists at runtime | examples/patch-loop-testing.koreo; verified E1/E2 |
| Someone edits a managed resource out-of-band | Undetected until the next workflow run — trigger event, Retry delay, or the 20-minute cadence. Managed resources are never watched | open issue koreo-controller#38 (since May 2025) |
| Trigger or Workflow definition deleted | Cleanup is documented (delete: abandon | destroy) but the implementation issues are still open — controller#18, #19, core#25. Same-namespace owned resources are saved by ownerReferences GC; everything else orphans | GitHub issues, CRD schema accepts the field anyway |
| Watch on a trigger kind errors 10 consecutive times | Watch abandoned — new resources of that kind never reconcile until controller restart; a failing Functions-cache watch raises and exits the whole controller | events.py, koreo_cache.py MAX_SYS_ERRORS |
| You query conditions with standard tooling | Workflow-written conditions always carry status: "True" regardless of outcome (severity lives in reason) — kubectl wait --for=condition=Ready succeeds on broken workflows — while controller-side lookup errors use lowercase "false". Two conventions, one status field | workflow/reconcile.py _condition_helper |
| Schema-valid-looking specs that violate CRD CEL rules | Local prepare and FunctionTest accept them; the API server rejects at apply time — your green test suite says nothing | Verified: schema.validate() passes label: "not a valid label!" |
| Second controller replica for HA | No leader election anywhere in the entrypoint; both replicas watch and reconcile the same triggers with in-process locks | koreo_controller.py — replicas: 1 is a requirement, not a default |
| System errors persist past 10 retries | The scheduler hits # TODO: Exhausted retries... what now? and silently drops the work item | scheduler.py |
Deployment Realities
The Helm chart (koreo/koreo, chart 0.1.4, appVersion: "0.1") deploys the controller from us-docker.pkg.dev/koreo-dev/koreo/controller with one replica, a PDB template, service accounts, and Role/ClusterRole pairs for controller and UI. The CRDs install with the chart and can be disabled. Three details deserve a platform engineer's attention:
- RBAC is opt-in per managed kind. The chart ships only "default required permissions"; granting the controller access to whatever resource types your Functions manage is your job via
controller.rbacvalues. There is no kubebuilder-style marker generating per-CRD rules, so the safe-by-default ceiling is low and the escape hatch is a ClusterRole you write by hand. The chart also shipsdevelopment: true— "superuser capabilities on the controller and ui, not for production use" — which is exactly the kind of flag that ends up set in a values file nobody reviews. - The controller's namespace defaults are development leftovers.
KOREO_NAMESPACE,TEMPLATE_NAMESPACE, andRESOURCE_NAMESPACEall default tokoreo-testingin source. Nothing stops a misconfigured deployment from quietly loading Functions from a test namespace, and there is no way to scope which Workflows a controller instance owns (open issue koreo-controller#22) — one controller reconciles every Workflow whose trigger kind it can see. - Availability is single-process. All locks, caches, and the scheduler queue are in-process; the log format is structured JSON (good), but there is no leader election, no metrics endpoint (health/metrics exposure was the last feature branch, merged August 2025), and no documented probe endpoints beyond the chart's defaults. Treat the controller as a singleton with a PDB, and treat its restart as a full cache rebuild.
Project Health Forensics
| Date | Event |
|---|---|
| 2024-04-04 | Real Kinetic announces Konfig (the GitLab + Google Cloud product Koreo would power) |
| 2025-03-19 | koreo-core repository created |
| 2025-04-10 | Show HN launch (120 points); first issue batch filed the same week |
| 2025-04 – 2025-06 | Release cadence ~weekly (core v0.1.8 → v0.1.14; CLI → 0.3.3 on 2025-06-12) |
| 2025-08-01 → 2025-08-09 | Last sprint: core v0.1.15–v0.1.17, controller v0.1.18, health/metrics branch merged |
| 2025-08-09 | Last commit to koreo-core; last issue filed (controller packaging, zero comments since) |
| 2025-08-20 | Last push to koreo-controller |
| 2026-11-10 (this review) | 15 months, 3 days without a commit across all eleven repos in the org |
The bus factor is three people — Tyler Treat, Robert Kluin, and Eric Larssen, all of Real Kinetic, LLC (Boulder, CO), per package metadata and commit history. The FAQ's answer to "is it production-ready?" is honest as far as it goes — CRDs are v1beta1, "tooling is still evolving" — and names exactly one production user: Konfigurate, the company's own commercial platform (currently "early access," discount for the first two years, no public pricing). That is not astroturfing — the docs say it plainly — but it means Koreo's entire production validation is one consultancy's internal dogfooding. Ninety-nine stars and two forks say the market agreed with the concept and declined the dependency; the HN launch thread's most-upvoted technical comment is a request to compare against Crossplane v2 and kro, which is precisely the comparison the project never updated.
Who Should Use Koreo
There is a real team for whom this is the right tool, with one amendment — the word "use" must mean "fork":
- Platform teams sitting on an existing operator estate — ACK, Config Connector, or in-house CRDs you cannot or will not replace — who need to chain those operators' outputs into each other's inputs, with conditional logic, in multi-tenant namespaces, without writing a new Go controller for every composite. That is the exact shape Koreo was built for, and no other tool covers it as directly.
- Teams who will own the fork. Pin the engine, vendor it, apply the one-line dependency fix, budget for the missing hardening (backoff on user retries, watch-based drift, delete propagation, condition-status semantics), and you have a compact (~50 KB of core engine) Python codebase that a strong engineer can genuinely hold in their head.
- Anyone who wants the FunctionTest pattern — sequential control-loop simulation with variant isolation and exact patch-payload assertions. Steal this idea into whatever you run; it is the single most transferable artifact in the project.
Who Should Skip Koreo
Everyone else, and the authors effectively said so themselves. From the launch thread, Robert Kluin — who wrote the core engine: "I like Kustomize and KPT a lot. In my opinion, they should be your starting point... If Kustomize or kpt are sufficient, I would strongly encourage using one of them." And to the engineer who asked what it offers over rolling a custom operator: "You should write your operator. You are correct: it is quite straightforward." Take the maintainer at his word:
- If your configuration problem is templating — charts, overlays, golden paths — Helm/Kustomize/kpt + Argo CD is the maintainer-recommended path, with a support ecosystem measured in careers rather than GitHub issues.
- If you need cloud resources as Kubernetes APIs with a real ecosystem — Crossplane v2 gives you namespaced everything, any-resource composition, provider functions, and Cron/Watch operations, under CNCF governance, on a weekly release cadence. The pain points Koreo cites are v1 history.
- If you want the "one CRD that stands up a stack" experience — kro under kubernetes-sigs is the boring, governable version, and its documented limits (no cross-graph reads, no workflow engine) are limits most teams never hit.
- If you cannot afford a dependency whose last commit predates this year's budget cycle — which is every team running production. A dormant control-plane dependency is not a library you call; it is a controller holding RBAC power over your cluster that nobody patches. The sniffio crash is the harmless version of this. The version that matters is the security fix that never ships.
- If your platform strategy requires vendor viability — the only commercial entity behind Koreo monetizes a batteries-included product on top of it, not the engine. Buying Konfigurate and adopting Koreo are very different decisions; the first comes with the authors on call, and the second does not.
The Bottom Line
Koreo solved a real problem — programmable, testable, namespaced orchestration of arbitrary controllers — with a design that still reads as fresh in 2026: outcomes as a lattice, side effects quarantined to one function type, reconciles as unit-testable sequences. Then the company that built it got busy with its product, the ecosystem moved (Crossplane v2 absorbed the differentiators, kro absorbed the governance story), and the repository went quiet with the honesty of its own TODOs still in the source. Read this engine the way you'd read a good design doc written by strong engineers: take the Outcome lattice, take FunctionTest, take the compare directives, and take the warning baked into a fifteen-month-old # TODO: Exhausted retries... what now? — a control plane you adopt is a control plane you maintain.
References & Further Reading
- Koreo documentation — Overview, Workflow, ResourceFunction, FunctionTest, Expressions (koreo.dev)
- Show HN: Koreo — launch thread with author replies (April 2025)
- koreo-dev/koreo-core — engine source, v0.1.17 (Apache-2.0)
- koreo-dev/koreo-controller — controller source, v0.1.18
- Koreo scheduler source — the defaults table and retry semantics cited in this review
- koreo tooling on PyPI (0.3.3, 2025-06-12; pins koreo-core==0.1.14)
- Koreo's Crossplane comparison page — audited against v2.4 in this review
- Crossplane v2 — What's New (namespaced XRs/MRs, claims removed, Operations)
- Crossplane v2 upgrade guide (v2.4 docs)
- kro — Kube Resource Orchestrator (kubernetes-sigs)
- Konfigurate — the commercial platform powered by Koreo
- FunctionTest documentation — the pattern this review recommends stealing