From 50993bedecd700b74c9da5298ee8e9625af32a5a Mon Sep 17 00:00:00 2001
From: jwright
Date: Mon, 17 Aug 2026 19:51:12 +1000
Subject: [PATCH] pre-review hardening: technical qualifiers across all
courses, git/Argo CD proper nouns, a11y (skip link, main, focus-visible,
reduced-motion net, textContent legends), perf (intrinsic image dims,
decoding async, lazy scenes + will-change lifecycle), nginx ^~ /assets/
priority, OG/canonical/build-revision, preflight gates
---
Dockerfile | 2 +
assets/dist/delivery-flow.svg | 4 +-
nginx/default.conf | 9 +-
pilot/arc.tpl.html | 448 ++++++++++++++++++----------------
pilot/assemble-arc.mjs | 114 ++++++++-
pilot/manifest-gitops.json | 4 +-
pilot/manifest-node.json | 2 +-
pilot/manifest-pod.json | 4 +-
pilot/manifest-supply.json | 4 +-
9 files changed, 358 insertions(+), 233 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 4455c65..e84b0b1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -7,6 +7,8 @@
# verified and cached in the zot pull-through mirror, so this layer costs nothing
# extra. Only bash/sed/find/install are used here — node itself is incidental.
FROM node:22-bookworm-slim@sha256:d9f850096136edbc402debdd8729579a288aac64574ada0ff4db26b6ae58b0b2 AS build
+ARG BUILD_REV=dev
+ENV BUILD_REV=$BUILD_REV
WORKDIR /app
COPY . .
RUN bash build.sh
diff --git a/assets/dist/delivery-flow.svg b/assets/dist/delivery-flow.svg
index 26906ff..358cf02 100644
--- a/assets/dist/delivery-flow.svg
+++ b/assets/dist/delivery-flow.svg
@@ -1,4 +1,4 @@
-
That 1/1 reads as containers-ready over containers-wanted: a pod
can hold more than one, which is Course IIIb.
-
An image got built - of what, exactly? Pushed - to where, and what travelled?
- Applied - which is not the same as launched. Running - according to whom?
- Every course below takes one of those words apart. The armour is already loose.
+
Three of those were delivery actions - build, push, apply - and the
+ last, oc get pods, only checked the result. Built of what, exactly?
+ Pushed to where, and what travelled? Applied, which is not the same as launched. Running,
+ according to whom? Each course below takes one of those words apart.
@@ -124,9 +125,9 @@
What an image is made of
An image is not a copy of a machine. It is a stack of read-only layers, each
recording only what changed from the one beneath. Layers are content-addressed, so an
- identical layer is stored once and reused by every image that references it, so a pull only
- fetches the layers you do not already have. A base sits at the bottom, your dependencies on it,
- your code - usually the smallest layer, always the most volatile - above that.
+ identical layer is stored once and reused by every image that references it, so a pull
+ fetches just the layers you do not already have. A base sits at the bottom, your dependencies on it,
+ your code - usually the smallest layer, and typically the most volatile - above that.
Those layers become one filesystem through a union mount - overlayfs, the same
kernel feature you can mount by hand. The read-only image layers are the lower dirs; the
container gets a fresh upper dir of its own. Writes land in the upper, and editing an
@@ -144,10 +145,9 @@
happens to build software.
Field note. A rebuild that takes twenty minutes and one that
takes twenty seconds are usually the same Containerfile with the lines swapped.
-
This stack explains the whole ecosystem above it: sharing explains why
- pulls are fast, immutability explains why a digest can name the exact bytes (Course VII),
- and the throwaway top layer explains why state needs volumes. One idea - frozen diffs -
- all the way down.
+
This stack explains a lot of what sits above it: shared layers make
+ pulls fast, immutable content lets a digest name exact bytes (Course VII), and the throwaway
+ top layer is why persistent state needs volumes.
You never tell Kubernetes how to run your app. You describe what you
- want - declarative intent - and the cluster works, forever, to make it true. Scroll, and
+ want - declarative intent - and the cluster works continuously to make it true. Scroll, and
the formation splits: the half that decides rises, the halves that run spread below.
@@ -178,8 +178,7 @@
Desired against actual, on a loop
The habit underneath everything: the reconciliation loop - compare desired state
against actual state, fix the difference, repeat. That oc apply
- didn't launch anything; it filed paperwork. The machine took it from there, and it never
- stops taking it from there: kill a pod and it returns, not because something noticed the
+ didn't launch anything; it filed paperwork. The machine took it from there, and it keeps taking it from there: kill a pod and it returns, not because something noticed the
crash but because the loop noticed the difference.
Kubernetes doesn't run your app - it reconciles it.
One half decides, one half runs
@@ -187,28 +186,30 @@
plane that decides - holds the truth, schedules, reconciles - and worker nodes
that run pods. The workers are deliberately interchangeable: identical, replaceable, cattle
from day one. Authority does not live where the work happens.
-
Field note. If you are SSHing into nodes to "fix" things, you
- are arm-wrestling the reconciler - and it does not get tired. Change the desired state
- instead.
+
Field note. If you SSH into a node to "fix" a workload, a
+ controller may reconcile your change away. Change the desired state instead, and let the loop
+ carry it.
What the control plane is made of
-
"Control plane" is four processes and a database, and naming them makes every later error
- message readable. The api-server is the only door: everything authenticates, is
- authorised and is admitted there, and it is the only component allowed to touch etcd -
- the key-value store holding the entire cluster state. Lose etcd and you have lost the
- cluster, which is why backing it up is the homework nobody should skip. The scheduler
- decides which node a new pod belongs on - packing by the resources a pod
- requests, not by what it currently uses - and writes that decision down; it never
- starts anything. The controller-manager runs the reconciliation loops. Nothing talks
- sideways - every component watches the api-server.
+
The core control plane is the API server, scheduler and
+ controller-manager, backed by etcd; cloud deployments may also run a
+ cloud-controller-manager. Naming them makes the later error messages readable. The API server
+ (kube-apiserver) is the only door: requests authenticate, are
+ authorised and are admitted there, and it is the one component that talks to etcd, the
+ key-value store holding Kubernetes API state. Lose etcd and you have lost the cluster, which
+ is why backing it up is the homework to not skip. The scheduler decides which node a new pod
+ belongs on - packing by the resources a pod requests, not by what it currently uses -
+ and records that decision; it does not start containers. The controller-manager runs the
+ reconciliation loops. Control-plane state converges through the API server: the scheduler and
+ built-in controllers watch API objects and write decisions back through the API rather than
+ modifying etcd directly.
One of those loops is the chain you will debug most: a Deployment creates a
ReplicaSet, and the ReplicaSet creates pods. That is why the deploy in the
aperitif printed deployment.apps/api created and you then went
looking for a pod - and why, when a rollout is stuck with no pod at all, the answer is
upstream in that chain rather than on any node.
-
etcd is the truth. Everything else is a cache of it.
+
etcd is the durable store for Kubernetes API state; controllers reconcile from it continually.
Everything the delivery arc teaches from Course VI onward is the same
- loop at a bigger scale - git as the desired state, whole fleets as reconciled objects.
- Learn the promise once; it repeats all the way up.
+ loop at a larger scale: Git as the desired state, whole fleets as reconciled objects.
Every node runs a kubelet - the agent that owns what should be running there. It does
- not create containers itself. It speaks CRI - the Container Runtime Interface - to containerd or CRI-O, and that
- runtime pulls the image (through the mirror of Course VII) and actually creates and starts
- the container, handing the low-level work to runc or crun. A CNI plugin (Container Network Interface) hands the pod
- a real IP, and kube-proxy - or an eBPF datapath (code running
- inside the kernel itself) replacing it - makes Service
- addresses route to real pods. One correction worth carrying: the runtime calls the CNI plugin,
- not the kubelet.
-
The kubelet decides what should run. The runtime is what starts it.
+
The scheduler decides which node a pod belongs on; the kubelet on that node
+ makes the assigned PodSpec real; the runtime creates and starts the containers. The
+ kubelet does not place pods, and does not itself create containers - it speaks CRI (the
+ Container Runtime Interface) to containerd or CRI-O, and that runtime pulls the image
+ (through the mirror of Course VII) and starts it, handing the low-level work to runc or crun.
+ A CNI plugin (Container Network Interface) gives the pod a real IP - called by the
+ runtime, not the kubelet - and kube-proxy, or an eBPF datapath replacing it, makes
+ Service addresses route to real pods.
+
The scheduler places the pod. The kubelet realises it. The runtime runs it.
Field note. Node NotReady? The kubelet is a systemd unit
like any other: journalctl -u kubelet on that node, and check it
can still reach the api-server. A node that cannot phone home is presumed lost.
-
The control plane never touches your workload. It writes intent; the
- kubelet turns intent into instructions, and the runtime turns instructions into processes. Authority and execution meet exactly here, nowhere
- else.
+
The control plane does not start container processes directly. It records
+ and reconciles intent; node-side components execute it - the kubelet turning intent into
+ instructions, the runtime turning instructions into processes.
Everything inside the jacket shares a network namespace: one IP, localhost between
friends. Volumes are declared once on the pod, but each container mounts the ones it needs -
- sharing storage is opt-in, not automatic. initContainers run first, in order, each to
- completion - nothing else in the pod starts until every one of them has exited successfully. Sidecars ride along with their own containers and their
- own jobs: proxy, logs, reload.
+ sharing storage is opt-in, not automatic. regular init containers run in order and
+ complete before the app containers start. Native sidecars are the deliberate exception:
+ restartable init containers that keep running alongside the app, for jobs like proxy, logs and
+ config reload.
Three probes, three different jobs
startup owns warm-up, readiness gates traffic, liveness restarts the
truly hung. Confusing them is how healthy pods get executed - a slow start killed by an
@@ -296,7 +298,7 @@
the number alone: oc describe pod <name> shows both together,
and oc logs --previous shows what the dead container said on its
way out. Worth connecting to what you already know: a memory limit becomes a
- cgroup ceiling, and the kernel's OOM killer enforces it exactly as it would for any
+ cgroup ceiling, and the kernel's OOM killer enforces it just as it would for any
other process on the box.
The pod is the smallest schedulable unit - the jacket, never the
container. Once that distinction lands, half of Kubernetes networking stops being
@@ -341,26 +343,30 @@
everything here finds everything else, from a Service picking pods to a fleet hub picking
whole clusters (Course VI). Wear the label and you are eligible; readiness decides whether
you stay.
-
And the ClusterIP is worth a Linux translation: no interface owns that address.
- Nothing answers ARP for it. It exists as a rule - kube-proxy programs iptables (or
- IPVS) on every node so packets aimed at the virtual IP are DNATed to one of the ready pod
- IPs, and an eBPF dataplane does the same job further down without the rule tables. Cluster
- DNS - CoreDNS - resolves api.myns.svc.cluster.local to that
- VIP. If you have ever written a DNAT rule by hand, you have already built a small Service.
+
A ClusterIP is a virtual Service address implemented by the node dataplane rather
+ than an application process listening on that IP. In iptables or nftables mode,
+ kube-proxy installs rules that steer Service traffic to endpoint IPs; if you have
+ written a DNAT rule by hand, that mode will feel familiar. In IPVS mode, kube-proxy binds
+ Service IPs to the kube-ipvs0 dummy interface and creates IPVS
+ virtual servers. eBPF implementations can replace kube-proxy with their own dataplane. Cluster
+ DNS (CoreDNS) resolves api.myns.svc.cluster.local to the
+ Service address.
Readiness decides membership
-
A pod failing its readiness probe silently leaves the pool. No error, no event at
- the client - traffic simply stops arriving. That is the feature: broken instances remove
- themselves. It is also the first place to look when traffic "disappears".
+
When a configured readiness probe fails, the pod becomes unready and normal
+ Kubernetes Service traffic stops selecting it for new connections. There is no error at the
+ client - the pod drops out of the endpoint set. That is the feature: broken instances take
+ themselves out of rotation. It is also the first place to look when traffic "disappears".
The rule has deliberate exceptions: you address pods directly when debugging a specific
instance, and headless Services exist precisely so StatefulSet members can be reached
individually by stable DNS name. For ordinary application traffic, though, the Service is
the only address worth knowing.
-
For application traffic: never talk to a pod; talk to a Service.
+
For ordinary Service traffic: address the Service, not a pod.
Field note. "The network is broken" after a deploy is usually
readiness telling the truth about your app - not the network lying about your packets.
The dark pod in the scene is not an error state - it is the system
- working. Membership is re-earned every few seconds, by every pod, for as long as it serves.
+ working. For pods with readiness probes, eligibility for Service traffic is continually
+ re-evaluated for the life of the pod.
The SCC - Security Context Constraint - is admission deciding what a pod may BE, checked
- by the api-server when the pod is created and before any node sees it. The default, restricted-v2, runs
- your container as a random non-root UID - your image has to cope. Workloads that
- genuinely need privilege get a dedicated ServiceAccount bound to a minimal custom SCC,
- never the stock one.
+ by the api-server when the pod is created and before any node sees it. Under OpenShift's restricted
+ SCCs, workloads normally run as a non-root UID drawn from the project's allocated range,
+ so an image has to work with an arbitrary permitted UID rather than assuming a fixed user.
+ Workloads that genuinely need privilege get a dedicated ServiceAccount bound to the minimum
+ SCC that grants it - not the stock one.
The platform runs itself
-
Routes predate Ingress and still rule here. An Operator is a controller paired with a custom
- resource: you describe what you want in YAML, and its controller builds it and keeps it true
- - the reconciliation loop of C-II, sold as a product. The split matters: the platform's own operators are driven by the Cluster Version
+
Routes predate Ingress and still rule here. An Operator is a controller paired with a
+ custom resource: you describe what you want in YAML, and its controller builds it and keeps it
+ true - an Operator applies the same reconciliation pattern to an application or platform
+ capability. The split matters: the platform's own operators are driven by the Cluster Version
Operator, while OLM installs and upgrades the add-on Operators you choose from
- OperatorHub. The OS underneath is immutable - changed by MachineConfig, never by SSH.
- And SNO - single-node OpenShift - puts the whole cluster on one box at the edge. At fleet scale the labels
+ OperatorHub. MachineConfig is the supported declarative path for node OS configuration;
+ ad-hoc SSH changes create drift and are not the intended operating model. And SNO -
+ single-node OpenShift - puts a whole cluster on one box at the edge. At fleet scale the labels
from Course VI decide which of these boxes runs what.
On OpenShift, admission is the interview - the SCC is the dress code.
Field note. Deployment stuck at 0/1 with no pod at all? The
refusal happened above scheduling - read the ReplicaSet events. The error lives a level
up.
-
Everything in the ring wraps the same core you already know. What
- OpenShift adds is a house style with teeth - and admission is where it bites first.
+
Everything in the ring wraps the same core you already know; what
+ OpenShift adds is admission, routing and lifecycle opinions on top of it.
The mental model everyone arrives with: someone with credentials pushes manifests
at the cluster. In the architecture this course teaches, nothing is pushed: a repository holds
the desired state, an agent inside the cluster watches it, and the cluster pulls its own future
- from git.
+ from Git.
@@ -436,51 +445,52 @@
The loop you already know, one level up
C-II taught the reconciliation loop: desired versus actual, fix the difference, repeat.
- ArgoCD is the same habit applied to delivery. An Application names a repo, a path and a
- revision - watch this branch of this repository - and the controller renders what it finds
- there, compares it against the live cluster, and syncs the difference. The deploy button is a
- git commit; the change history is git log; code review is change control. Git is the
- record of intent - the cluster's own audit log and the reconciler's sync history still
- record what actually happened, including everything git never sees.
-
oc is for archaeology. git is for change.
+ Argo CD applies the same habit to delivery. An Application names a repo, a path
+ and a revision - watch this branch of this repository - and the controller renders what it
+ finds there, compares it against the live cluster, and syncs the difference. The deploy button
+ is a Git commit; the change history is git log; code review is
+ change control. Git records intent - the cluster's own audit log and Argo CD's sync
+ history record what actually happened, including what Git does not see.
+
oc reads the running system; Git records the change.
Pull, not push - the security inversion
-
Here the cluster pulls. Push-based delivery exists and is still GitOps to many - this
- is the stronger variant, and worth choosing deliberately: no CI system, no laptop, no build
- pipeline holds a credential that can touch the cluster, because the agent inside holds a
- read-only deploy key and the trust arrow points out. Compromise the build system and you can
- propose a change, which is visible; you cannot reach into production through git. It
- can still push images - which is the other half of why a manifest should name the digest, not
- the tag. Hand-edit a live object
- and the controller flags it OutOfSync - with selfHeal enabled it puts the object back,
- and with prune enabled what leaves git leaves the cluster. Both are opt-in: without
- them the reconciler reports the drift and waits for a human. Rollback is git revert,
- which is why commit hygiene is an operational skill.
-
Field note. A hand-patch on a live object survives exactly until
- the next sync. If you must touch production directly to stop the bleeding, open the pull
- request in the same hour - otherwise the reconciler quietly undoes your fix, and the outage
- returns with nobody able to say why.
+
Here the cluster pulls. Push-based delivery exists and is still GitOps to many; the
+ pull variant is worth choosing deliberately, because no CI system, laptop or build pipeline
+ holds a credential that can touch the cluster - the agent inside holds a read-only deploy key
+ and the trust arrow points out. Compromise the build system and you can propose a change,
+ which is visible; you cannot reach into production through Git. It can still push
+ images, which is the other half of why a manifest should name the digest, not the tag.
+ Hand-edit a live object and the controller flags it OutOfSync. With self-heal enabled,
+ Argo CD can restore the desired state; with prune enabled, objects removed from Git can
+ be deleted. Both are opt-in: without them the reconciler reports the drift and waits for a
+ sync. Rollback is git revert, which is why commit hygiene is an
+ operational skill.
+
Field note. With self-heal enabled, a live hand-patch may be
+ reverted on the next reconciliation. Without self-heal, Argo CD reports the drift until someone
+ syncs it. Either way, land the change in Git in the same hour, or the fix and the reason for it
+ go missing.
At fleet scale, the label is the deploy button
Run many OpenShift clusters under a hub - RHACM, Red Hat's fleet manager, with edge clusters
arriving through zero-touch provisioning - and nobody applies apps to clusters by hand. Each app carries a
Placement that selects cluster labels; the hub matches placements against the
labels a cluster wears, and the chosen cluster is given its assignment - push-style from the
- hub by default, or in a pull model where each cluster runs its own reconciler, which is the
- variant this arc teaches. Either way, labelling the cluster is the deploy action: attach the label and the app follows, remove it and the app
- leaves. The same pull model as above, one level bigger - a cluster's labels are its
- entitlements, reconciled like everything else.
+ hub by default, or in a pull model where each cluster runs its own reconciler. Labelling a
+ cluster changes its placement eligibility; the lifecycle and cleanup of what lands still follow
+ the configured propagation and deletion policy, so removing a label does not, on its own,
+ guarantee an app is deleted.
Label the cluster; the app follows.
-
The one thing git never holds
-
Git holds everything except secrets. Commit a plaintext secret and you should treat it as
- compromised from that moment: deleting it later does not guarantee it is gone from
- history, forks, clones, CI caches or backups. So the pattern splits the reference from the value: git carries an ExternalSecret
- naming a logical key; a vault - Azure Key Vault in the worked example - holds the value;
+
Keep plaintext secret values out of Git
+
In this architecture, plaintext secret values do not live in Git: Git stores the
+ ExternalSecret reference, and the vault stores the value. Commit a plaintext secret and you
+ should treat it as compromised from that moment - deleting it later does not guarantee
+ it is gone from history, forks, clones, CI caches or backups. So Git carries an ExternalSecret
+ naming a logical key; a vault (Azure Key Vault in the worked example) holds the value;
an operator inside the cluster exchanges one for the other at runtime. Rotation happens in the
- vault, never as a commit. That is why the vault floats apart in the scene above: it is never
- absorbed into the pipeline.
+ vault, not as a commit - which is why the vault sits apart from the pipeline in the scene
+ above.
Git holds the shape of the secret. The vault holds the secret.
-
Delivery stops being an event and becomes a property: the cluster is always
- converging on what the repository says. "Who deployed this?" becomes "who merged this?" - and
- that question always has an answer.
+
Delivery becomes a property rather than an event: the cluster converges on
+ what the repository says. With protected branches and attributable identities, changes should
+ be traceable to reviewed commits.
myapp:latest feels like a name. It is a sticky note - a
- mutable pointer anyone with push rights can peel off one image and press onto another, and
- nothing anywhere records that it moved. The digest - sha256 of the image manifest, the
- small JSON index listing an image's layers, and no relation to the YAML manifests you apply
- to a cluster - is
- its actual name: same bytes, same digest, forever. (Careful: the sha256 a build prints is the
- local image ID, a different hash from the manifest digest the registry mints on push - the
- pushed one is what you pin.)
+ mutable reference anyone with push rights can move from one image to another. Registries may
+ audit tag updates, but the tag itself carries no immutability guarantee. The digest - the
+ sha256 of the image manifest, the small JSON index listing an image's layers, and no
+ relation to the YAML manifests you apply to a cluster - identifies the exact uploaded content.
+ A local image ID printed by a build is not necessarily the same value, so pin the registry
+ digest that consumers actually pull.
@@ -521,7 +530,7 @@
Three ways to name an image, in rising order of honesty: :latest
(a moving target), :1.4.2 (a label someone maintains, until they
re-push it), and name:1.4.2@sha256:... - a fact. The tag stays for
- human eyes; the digest does the pulling. Pin by digest and "what is running?" has exactly one
+ human eyes; the digest does the pulling. Pin by digest and "what is running?" has a single
answer.
Field note.:latest is how two nodes run
different code from one manifest - the second node pulled an hour later, after a re-push.
@@ -529,33 +538,34 @@
Why a fleet pulls once
Between the build and the node sits the registry chain. Upstream, a managed registry - Azure
Container Registry in the worked example - holds what CI built. In front of the cluster sits a
- mirror: a pull-through cache like zot. The fleet asks the mirror, the mirror asks
- upstream once, everything after is local. Rate limits, egress cost, disconnected sites,
+ mirror: a pull-through cache like zot. On a cache miss the mirror fetches the artefact
+ from upstream; subsequent requests can be served locally while that content stays cached. Rate
+ limits, egress cost, disconnected sites,
control - one place to gate and audit what enters. OpenShift formalises the re-route with
image mirror rules, and carries a sharp edge: digest-mirror rules rewrite digest pulls
only, so a by-tag pull silently skips them - unless you also add an
ImageTagMirrorSet, which is the rule type built for tag pulls. Pinning by digest is
still the habit that makes the digest rules catch everything.
At real fleet scale the mirror itself tiers: a central mirror in the cloud fronts
- upstream once, and every site's mirror pulls from the centre rather than from upstream
- directly. A new image ripples outward in layers - upstream to the centre, centre to each
- site as it asks, site to its nodes over the LAN - instead of every site hammering upstream
- at the same moment. Upstream sees one consumer; each site sees one hop; the nodes never
- leave the building.
-
Mirrors tier: the load fans out in layers, never all at once.
+ upstream, and each site's mirror pulls from the centre rather than from upstream directly. A
+ new image ripples outward in layers - upstream to the centre, centre to each site as it asks,
+ site to its nodes over the LAN - rather than every site hitting upstream at the same moment.
+ With a site configured to consume only its local mirror, its node image traffic stays on the
+ local network.
+
Tiered mirrors fan the load out in layers rather than all at once.
Build once, promote by copy
Every rebuild is a new artefact - in practice a different digest (reproducible builds
are the deliberate exception), untested by the stages before it.
- So build once, then promote the same digest through environments by copying, registry
- to registry - dev proves the exact bytes prod will run. On a multi-arch image that copy
- (skopeo copy) needs --all (and --preserve-digests to fail
- loudly rather than quietly), or you copy one architecture and the digest you promoted is not
- the digest that lands. Human tags ride along; the digest is
- the through-line.
+ So build once, then promote the same digest through environments by copying, registry to
+ registry - dev proves the exact bytes prod will run. For a multi-architecture image,
+ skopeo copy --all copies the complete image list; add
+ --preserve-digests when promotion requires the destination to keep
+ the source digests, so the copy fails if that cannot be done rather than silently landing a
+ different digest. Human tags ride along; the digest is the through-line.
If the digest changed, it is not a promotion - it is a new candidate.
-
Names that can move are convenient exactly until they move. Address content
- by what it is, and the supply chain stops resting on trust: it rests on a hash anyone can
- check.
+
Names that can move are convenient right up until they move. Address
+ content by its digest and the supply chain rests on a hash anyone can check, not on trust that
+ a tag stayed put.
Helm templates are Go text/template: {{ .Values.device.address }}
- is a pipeline walking a values object, _helpers.tpl holds the
- named templates - the partials every manifest includes, not
- functions you can call bare. You are not writing YAML - you are writing a program whose
- output is YAML. So render locally, read the output, and lint what came out, not what went in.
+ is a pipeline walking a values object. _helpers.tpl commonly holds
+ named templates - reusable partials that manifests can include
+ - rather than ordinary functions invoked directly. You are not writing YAML; you are writing a
+ program whose output is YAML, so render locally, read the output, and lint what came out rather
+ than what went in.
Review the render, not just the template.
-
Field note. ArgoCD renders charts with
+
Field note. Argo CD renders charts with
helm template rather than running helm
install, so the lifecycle differs from Helm's own. lookup
comes back empty - there is no live cluster at render time. Hooks are not dead, though: Argo
maps Helm hooks onto its sync phases (pre-install and pre-upgrade become PreSync, post-install
and post-upgrade become PostSync), while a few - rollback and test hooks - have no equivalent
at all. Render the way your deployer renders, and check where your hooks actually land.
-
Contexts: the cluster's whole voice is one small file
+
Contexts: one small file per cluster
The chart owns everything structural - resources, probes, security, policy. Each cluster owns
one values file: names, addresses, sizes, flags. The context is deliberately values-only; the
moment it carries its own manifests there are two owners for one object, and they will
- disagree. One value can feed many rendered artefacts - an address appearing in the app config,
- the network attachment and two policies renders from one field, so the copies cannot diverge - the lived version is on the blog.
+ disagree. One value can feed several rendered artefacts: the app config, network
+ attachment and two policies can all render from the same field, so those copies cannot diverge
+ - the lived version is on the blog.
The chart owns the shape. The context owns the numbers.
Make the template refuse
-
A template that renders whatever it is given just moves the failure downstream. The grown-up
+
A template that renders whatever it is given just moves the failure downstream. A production-grade
chart carries a values.schema.json: a context missing a required
value fails at render time, in the pipeline, with a message naming the field - not months
later as enforcement pointed at nothing.
@@ -633,7 +645,7 @@
Every toolchain stands on services it does not run.
The arc reads like a closed machine: repo to reconciler to registry to node. It
is not closed. Three load-bearing pieces live outside the cluster - and the honest move is to
- write down what leans on them, and what actually happens when they are down.
+ write down what leans on them, and what happens when they are down.
Leans on it: sync, rollback, change review, the "who merged this"
answer.
-
When it is down: Kubernetes keeps running the last applied state
- indefinitely. The reconciler keeps self-healing only while its rendered manifests are
- still cached - hours, not forever, and gone after a restart. What stops is change. GitOps degrades to
- read-only, which is the graceful half of the design.
+
When it is down: existing workloads keep running. Repository-backed
+ refreshes and new changes become unavailable; what the controller can still evaluate depends
+ on state already available to it locally. Do not design around an assumed cache lifetime.
+ What stops is change; the running system is unaffected.
Azure Key Vault
@@ -659,9 +671,9 @@
the value, an operator keeps them synced.
Leans on it: secret sync, rotation, the first deploy of anything that
needs a credential.
-
When it is down: already-synced Secrets keep working - values are
- materialised in-cluster. What stops is rotation and new secrets. Survivable - unless
- you are inside a rotation window.
+
When it is down: already-materialised Kubernetes Secrets typically
+ remain usable while the vault is unavailable. What fails is refresh, rotation and the
+ creation of new secret material - survivable, unless you are inside a rotation window.
zot
@@ -672,15 +684,16 @@
scale-up, recovery.
When it is down: the sharpest edge. Upstream down + mirror up = nobody
notices, provided the image is already cached - a cold entry still needs upstream.
- Mirror down on a mirror-only pull path = nothing new schedules unless the node already
- holds the image, and imagePullPolicy: Always turns a mirror
- outage into a hard stop.
+ Mirror down on a mirror-only pull path: a pod whose image is already in the node's store
+ can still start under IfNotPresent, but
+ imagePullPolicy: Always must resolve the reference against the
+ registry on each launch, so a mirror outage stops it even when the layers are present.
None of these outages stop what is already running - they stop change,
- rotation and recovery, in that order of pain. Cache what you pull, split references from
- values, and let the cluster hold its last known truth without asking anyone's permission.
+ rotation and recovery. Cache what you pull, keep secret values in a vault rather than Git, and
+ the cluster can hold its last applied state while a dependency is away.
Argo CDThe GitOps reconciler that watches a repository and keeps the cluster
- converged on what it finds there - the pull model that means nothing ever pushes to
- production.
+ converged on what it finds there - in the pull model, the cluster fetches its own desired state
+ rather than being pushed to.
argo-cd.readthedocs.io
-
GitHub ActionsCI that builds and tests on every push - in this flow it makes
- artefacts and proposes changes, and never holds a credential that can touch a cluster.
+
GitHub ActionsCI that builds and tests on each push - in this flow it makes
+ artefacts and proposes changes, and does not hold a credential that can touch a cluster.docs.github.com/actions
Azure Key VaultThe managed vault holding secret values, so git carries only
- references and rotation never has to become a commit.
+ references and rotation does not have to become a commit.
learn.microsoft.com/key-vault