The Exploded Cluster · The Delivery Arc
The foundations first - what an image is, what a cluster is - then the toolchain that delivers to them: how change travels, how images are named, and how one definition serves a fleet. Scroll, and each machine comes apart.
Purpose of this document
The Exploded Cluster teaches how modern container platforms work by taking them apart - literally. Each course is one machine drawn as a single exploded illustration, sliced into its real components and wired to your scroll, so the architecture moves while the words explain it. Start at Course I with what an image actually is; finish knowing how a change travels from a git commit to a running, secret-fed, digest-pinned workload on a fleet. Every claim comes from running these systems for real, and the library at the end links only to official documentation.
Course I · Docker
Scroll, and the thing you keep calling "a container image" comes apart in your hands. Four layers. Each one only stores what changed from the layer under it - and here they rise one at a time, bottom up.
An image is not a copy of a machine. It is a stack of read-only layers, each recording only what changed from the one beneath. The base OS is pulled once and shared by every image on the host; your dependencies sit on it; your code - usually the smallest layer, always the most volatile - sits above that; and the writable layer is created fresh per container and thrown away when it dies. Nothing you write there survives.
Change a layer and every layer above it must be rebuilt.
The builder caches layer by layer, and a cached layer survives only while everything beneath it is unchanged. Put COPY . . above your dependency install and you have told the builder to throw the dependency cache away every time one line of code changes. Dependencies first, code last - a Dockerfile is a cache policy that happens to build software.
Field note. A rebuild that takes twenty minutes and one that takes twenty seconds are usually the same Dockerfile with the lines swapped.
This stack explains the whole ecosystem above it: sharing explains why pulls are fast, immutability explains why a digest can name the exact bytes (Course VII), and the throwaway top layer explains why state needs volumes. One idea - frozen diffs - all the way down.
Pre-reads: none - start here Further: Docker get-started · Docker build + layers
Course II · Kubernetes
You never tell Kubernetes how to run your app. You describe what you want - declarative intent - and the cluster works, forever, to make it true. Scroll, and the formation splits: the half that decides rises, the halves that run spread below.
The habit underneath everything: the reconciliation loop - compare desired state against actual state, fix the difference, repeat. That kubectl apply didn't launch anything; it filed paperwork. The machine took it from there, and it never stops taking it from there: kill a pod and it returns, not because something noticed the crash but because the loop noticed the difference.
Kubernetes doesn't run your app - it reconciles it.
The split in the scene is the split that makes everything else possible: a control plane that decides - holds the truth, schedules, reconciles - and worker nodes that run pods. The workers are deliberately interchangeable: identical, replaceable, cattle from day one. Authority does not live where the work happens.
Field note. If you are SSHing into nodes to "fix" things, you are arm-wrestling the reconciler - and it does not get tired. Change the desired state instead.
Everything the delivery arc teaches from Course VI onward is this loop wearing bigger clothes - git as the desired state, whole fleets as reconciled objects. Learn the promise once; it repeats all the way up.
Pre-reads: C-I Further: Kubernetes overview · cluster architecture
Course VI · GitOps
The mental model everyone arrives with: someone with credentials pushes manifests at the cluster. In GitOps nothing is pushed. A repository holds the desired state, an agent inside the cluster watches it, and the cluster pulls its own future from git. kubectl is for looking; git is for changing.
C-II taught the reconciliation loop: desired versus actual, fix the difference, repeat. ArgoCD is the same habit applied to delivery. An Application names a repo, a path and a revision - watch this branch of this repository - and the controller renders what it finds there, compares it against the live cluster, and syncs the difference. The deploy button is a git commit; the audit log is git log; code review is change control.
kubectl apply is for archaeology, not for change.
The cluster pulls. No CI system, no laptop, no build pipeline holds a credential that can touch it - the agent inside holds a read-only deploy key, and the trust arrow points out. Compromise the build system and you can propose a change, which is visible; you cannot reach into production. Hand-edit a live object and the controller flags it OutOfSync - selfHeal puts it back, and what leaves git leaves the cluster. Rollback is git revert, which is why commit hygiene is an operational skill.
Field note. Fighting the reconciler over SSH is arm-wrestling a machine that does not get tired - C-II's warning, now with a face. The fix is always upstream, in git.
Run many OpenShift clusters under a hub - RHACM, with edge clusters arriving through zero-touch provisioning - and nobody applies apps to clusters by hand. Each app carries a Placement that selects cluster labels; the hub matches placements against the labels a cluster wears, and the chosen cluster pulls its assignment. Labelling the cluster is the deploy action: attach the label and the app follows, remove it and the app leaves. The same pull model as above, one level bigger - a cluster's labels are its entitlements, reconciled like everything else.
Label the cluster; the app follows.
Git holds everything except secrets - a secret in git is published, forever, to everyone who ever clones. So the pattern splits the reference from the value: git carries an ExternalSecret naming a logical key; a vault - Azure Key Vault in the worked example - holds the value; an operator inside the cluster exchanges one for the other at runtime. Rotation happens in the vault, never as a commit. That is why the vault floats apart in the scene above: it is never absorbed into the pipeline.
Git holds the shape of the secret. The vault holds the secret.
Delivery stops being an event and becomes a property: the cluster is always converging on what the repository says. "Who deployed this?" becomes "who merged this?" - and that question always has an answer.
Pre-reads: C-II · Kubernetes concepts · git + pull requests Further: Argo CD · OpenShift GitOps · External Secrets · Azure Key Vault
Course VII · The image supply chain
myapp:latest feels like a name. It is a sticky note - a mutable pointer anyone with push rights can peel off one image and press onto another, and nothing anywhere records that it moved. The digest is the image's actual name: same bytes, same digest, forever.
Three ways to name an image, in rising order of honesty: :latest (a moving target), :1.4.2 (a promise somebody keeps, until they re-push it), and name:1.4.2@sha256:... - a fact. The tag stays for human eyes; the digest does the pulling. Pin by digest and "what is running?" has exactly one answer.
Field note. :latest is how two nodes run different code from one manifest - the second node pulled an hour later, after a re-push. Nobody changed the YAML.
Between the build and the node sits the registry chain. Upstream, a managed registry - Azure Container Registry in the worked example - holds what CI built. In front of the cluster sits a mirror: a pull-through cache like zot. The fleet asks the mirror, the mirror asks upstream once, everything after is local. Rate limits, egress cost, disconnected sites, control - one place to gate and audit what enters. OpenShift formalises the re-route with image mirror rules, and carries a sharp edge: digest-mirror rules rewrite digest pulls only, so a by-tag pull silently skips the mirror. The pinning habit is what makes the mirror actually catch the traffic.
Every rebuild is a different artefact - different digest, untested by the stages before it. So build once, then promote the same digest through environments by copying, registry to registry - dev proves the exact bytes prod will run. Human tags ride along; the digest is the through-line.
If the digest changed, it is not a promotion - it is a new candidate.
Names that can move are convenient exactly until they move. Address content by what it is, and the supply chain stops being a chain of trust and becomes a chain of arithmetic.
Pre-reads: C-I · Kubernetes images Further: zot · Azure Container Registry · OpenShift image mirroring · skopeo
Course VIII · Helm
The default way to run one app on five clusters is five copies of the YAML, and the default result is five slightly different apps. The inversion: stop copying outputs and ship the function. A chart is a template with holes; each cluster supplies one small values file that fills them.
Helm templates are Go text/template: {{ .Values.device.address }} is a pipeline walking a values object, _helpers.tpl holds the named functions every manifest calls. You are not writing YAML - you are writing a program whose output is YAML. So render locally, read the output, and lint what came out, not what went in.
Review the render, not just the template.
Field note. ArgoCD deploys charts by running helm template - a pure render. Anything needing a live cluster or an install lifecycle (lookup, hooks) is silently dead there. Render the way your deployer renders.
The chart owns everything structural - resources, probes, security, policy. Each cluster owns one values file: names, addresses, sizes, flags. The context is deliberately values-only; the moment it carries its own manifests there are two owners for one object, and they will disagree. One value can feed many rendered artefacts - an address appearing in the app config, the network attachment and two policies renders from one field, so the copies cannot diverge - the lived version is on the blog.
The chart owns the shape. The context owns the numbers.
A template that renders whatever it is given just moves the failure downstream. The grown-up chart carries a values.schema.json: a context missing a required value fails at render time, in the pipeline, with a message naming the field - not months later as enforcement pointed at nothing.
Field note. The failure you want is the render that refuses. It costs a red pipeline. The alternative reports healthy the whole time.
Fleet consistency is not a discipline problem, it is a construction problem. Divergence stops being something you police and becomes something the tooling cannot express.
Pre-reads: C-II · Kubernetes objects Further: Helm docs · chart template guide · Go text/template · Helm on OpenShift
Appendix · The dependency ledger
The arc reads like a closed machine: repo to reconciler to registry to node. It is not closed. Three load-bearing pieces live outside the cluster - and the honest move is to write down what leans on them, and what actually happens when they are down.
Where the desired state lives - the system of record the whole loop watches, through a read-only deploy key.
Leans on it: sync, rollback, change review, the "who merged this" answer.
When it is down: the cluster does not care - the reconciler enforces the last synced state indefinitely. What stops is change. GitOps degrades to read-only, which is the graceful half of the design.
Where the secret values live - git carries the reference, the vault carries the value, an operator keeps them synced.
Leans on it: secret sync, rotation, the first deploy of anything that needs a credential.
When it is down: already-synced Secrets keep working - values are materialised in-cluster. What stops is rotation and new secrets. Survivable - unless you are inside a rotation window.
Where the fleet pulls from - a pull-through mirror between the cluster and the internet, and the control point for what enters.
Leans on it: every image pull on every node - boot, reschedule, scale-up, recovery.
When it is down: the sharpest edge. Upstream down + mirror up = nobody notices. Mirror down on a mirror-only pull path = nothing new schedules, and a rebooting node may not come back whole.
None of these outages stop what is already running - they stop change, rotation and recovery, in that order of pain. Cache what you pull, split references from values, and let the cluster hold its last known truth without asking anyone's permission.
Further: GitHub docs · Azure Key Vault · zot · Kubernetes · Helm · Red Hat OpenShift
The whole arc · end to end
Every course above is one stretch of the same journey. Here is the full run, drawn in the house blueprint style: the change lane, the shape lane, the artefact lane and the secret lane, all converging on one running workload.
The library
Every technology this site teaches, one sentence each, official documentation only.