technical accuracy pass: kubelet/runtime split, image vs container layer, exit 137 = SIGKILL, ArgoCD maps helm hooks, selfHeal/prune opt-in, scoped absolutes (reviewer findings)
This commit is contained in:
+59
-37
@@ -13,8 +13,8 @@
|
||||
<li><a href="#c0"><b>Course 00</b><span>Aperitif - three commands, and what actually just happened</span></a></li>
|
||||
<li><a href="#c1"><b>Course I</b><span>Podman - an image is a stack of frozen diffs</span></a></li>
|
||||
<li><a href="#c2"><b>Course II</b><span>Kubernetes - a cluster is a promise, not a place</span></a></li>
|
||||
<li><a href="#c3a"><b>Course III</b><span>The node - where intent becomes a process</span></a></li>
|
||||
<li><a href="#c3b"><b>Course III</b><span>The pod - one IP, shared fate</span></a></li>
|
||||
<li><a href="#c3a"><b>Course IIIa</b><span>The node - where intent becomes a process</span></a></li>
|
||||
<li><a href="#c3b"><b>Course IIIb</b><span>The pod - one IP, shared fate</span></a></li>
|
||||
<li><a href="#c4"><b>Course IV</b><span>The traffic - pods are cattle, Services are the brand</span></a></li>
|
||||
<li><a href="#c5"><b>Course V</b><span>OpenShift - Kubernetes with opinions, and a security guard</span></a></li>
|
||||
<li><a href="#c6"><b>Course VI</b><span>GitOps - nobody deploys anything; the cluster syncs itself</span></a></li>
|
||||
@@ -96,11 +96,14 @@
|
||||
</div>
|
||||
<div class="prose">
|
||||
<h3>A stack of diffs wearing a name</h3>
|
||||
<p>An image is not a copy of a machine. It is a stack of read-only layers, each recording only
|
||||
what changed from the one beneath. The base OS is pulled once and shared by every image on
|
||||
the host; your dependencies sit on it; your code - usually the smallest layer, always the
|
||||
most volatile - sits above that; and the writable layer is created fresh per container and
|
||||
thrown away when it dies. Nothing you write there survives.</p>
|
||||
<p>An image is not a copy of a machine. It is a stack of <b>read-only</b> layers, each
|
||||
recording only what changed from the one beneath. Layers are content-addressed, so an
|
||||
identical layer is stored once and reused by every image that references it - which is why
|
||||
the second pull of anything is quick. A base sits at the bottom, your dependencies on it,
|
||||
your code - usually the smallest layer, always the most volatile - above that.</p>
|
||||
<p>The top slab in the scene is the odd one out: the <b>writable layer is not part of the
|
||||
image</b>. The runtime adds it when it creates the <i>container</i>, and discards it when
|
||||
that container dies. Nothing written there ships, and nothing written there survives.</p>
|
||||
<p class="sayit">Change a layer and every layer above it must be rebuilt.</p>
|
||||
<h3>The order is a caching decision</h3>
|
||||
<p>The builder caches layer by layer, and a cached layer survives only while everything
|
||||
@@ -168,7 +171,7 @@
|
||||
<!-- ============ COURSE III - THE NODE ============ -->
|
||||
<section class="course" id="c3a" data-scene="node" aria-labelledby="c3ah">
|
||||
<header class="chead">
|
||||
<p class="eyebrow">Course III · The node</p>
|
||||
<p class="eyebrow">Course IIIa · The node</p>
|
||||
<h2 id="c3ah" class="thesis">Where intent<br><em>becomes a process.</em></h2>
|
||||
<p class="lede">Everything so far was decision. This is the machine where a pod stops being
|
||||
paperwork and starts being a process - and the chain assembles link by link as you scroll.</p>
|
||||
@@ -185,18 +188,19 @@
|
||||
</div>
|
||||
</div>
|
||||
<div class="prose">
|
||||
<h3>One component starts containers. One.</h3>
|
||||
<p>Every node runs a <b>kubelet</b> - and in the entire system it is the only component that
|
||||
actually starts your container. It speaks CRI to <b>containerd</b> or CRI-O, which pulls the
|
||||
image (through the mirror of Course VII) and launches it. A <b>CNI plugin</b> hands the pod
|
||||
<h3>The kubelet decides; the runtime does</h3>
|
||||
<p>Every node runs a <b>kubelet</b> - the agent that owns what should be running there. It does
|
||||
not create containers itself. It speaks <b>CRI</b> to <b>containerd</b> or CRI-O, and that
|
||||
runtime pulls the image (through the mirror of Course VII) and actually creates and starts
|
||||
the container, handing the low-level work to runc or crun. A <b>CNI plugin</b> hands the pod
|
||||
a real IP, and <b>kube-proxy</b> - or an eBPF datapath replacing it - makes Service
|
||||
addresses route to real pods. The kubelet drives, the runtime executes, the CNI wires, the
|
||||
proxy routes.</p>
|
||||
<p class="sayit">The kubelet is the only component that actually starts your container.</p>
|
||||
<p class="sayit">The kubelet decides what should run. The runtime is what starts it.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> Node NotReady? Check kubelet-to-API-server
|
||||
connectivity before anything else - a node that cannot phone home is presumed lost.</p>
|
||||
<p class="closeline">The control plane never touches your workload. It writes intent; the
|
||||
kubelet reads it and makes it flesh. Authority and execution meet exactly here, nowhere
|
||||
kubelet turns intent into instructions, and the runtime turns instructions into processes. Authority and execution meet exactly here, nowhere
|
||||
else.</p>
|
||||
<p class="reads"><b>Pre-reads:</b> C-II
|
||||
 <b>Further:</b> <a href="https://kubernetes.io/docs/concepts/architecture/">cluster architecture</a> ·
|
||||
@@ -207,7 +211,7 @@
|
||||
<!-- ============ COURSE III - THE POD ============ -->
|
||||
<section class="course" id="c3b" data-scene="pod" aria-labelledby="c3bh">
|
||||
<header class="chead">
|
||||
<p class="eyebrow">Course III · The pod</p>
|
||||
<p class="eyebrow">Course IIIb · The pod</p>
|
||||
<h2 id="c3bh" class="thesis">One IP,<br><em>shared fate.</em></h2>
|
||||
<p class="lede">A pod is not a container - it is a jacket around one or more. Scroll, and the
|
||||
capsule opens like a clamshell: shells apart, contents on display.</p>
|
||||
@@ -226,7 +230,8 @@
|
||||
<div class="prose">
|
||||
<h3>The jacket, not the container</h3>
|
||||
<p>Everything inside the jacket shares a <b>network namespace</b>: one IP, localhost between
|
||||
friends, volumes mountable by all. <b>initContainers</b> run first, in order, to completion -
|
||||
friends. Volumes are declared once on the pod, but each container mounts the ones it needs -
|
||||
sharing storage is opt-in, not automatic. <b>initContainers</b> run first, in order, to completion -
|
||||
gatekeepers under everything. <b>Sidecars</b> ride along with their own containers and their
|
||||
own jobs: proxy, logs, reload.</p>
|
||||
<h3>Three probes, three different jobs</h3>
|
||||
@@ -234,8 +239,11 @@
|
||||
truly hung. Confusing them is how healthy pods get executed - a slow start killed by an
|
||||
impatient liveness probe looks exactly like a crash.</p>
|
||||
<p class="sayit">containerPort is documentation - the app still has to bind the port itself.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> Exit 137 with reason Error is the kubelet's probe
|
||||
kill; OOMKilled is memory. Same exit code, different crimes - read the reason.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> Exit 137 is 128 + 9: the process was <b>SIGKILLed</b>.
|
||||
It does not say by whom. The kernel's OOM killer surfaces as reason OOMKilled; a failed
|
||||
liveness probe shows up in the pod's events; eviction and node pressure look different again.
|
||||
One exit code, several possible crimes - read the termination reason and the events, never
|
||||
the number alone.</p>
|
||||
<p class="closeline">The pod is the smallest schedulable unit - the jacket, never the
|
||||
container. Once that distinction lands, half of Kubernetes networking stops being
|
||||
mysterious.</p>
|
||||
@@ -274,7 +282,11 @@
|
||||
<p>A pod failing its readiness probe <b>silently leaves the pool</b>. No error, no event at
|
||||
the client - traffic simply stops arriving. That is the feature: broken instances remove
|
||||
themselves. It is also the first place to look when traffic "disappears".</p>
|
||||
<p class="sayit">Never talk to a pod; talk to a Service.</p>
|
||||
<p>The rule has deliberate exceptions: you address pods directly when debugging a specific
|
||||
instance, and headless Services exist precisely so StatefulSet members can be reached
|
||||
individually by stable DNS name. For ordinary application traffic, though, the Service is
|
||||
the only address worth knowing.</p>
|
||||
<p class="sayit">For application traffic: never talk to a pod; talk to a Service.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> "The network is broken" after a deploy is usually
|
||||
readiness telling the truth about your app - not the network lying about your packets.</p>
|
||||
<p class="closeline">The dark pod in the scene is not an error state - it is the system
|
||||
@@ -332,8 +344,9 @@
|
||||
<p class="eyebrow">Course VI · GitOps</p>
|
||||
<h2 id="c6h" class="thesis">Nobody deploys anything.<br><em>The cluster syncs itself.</em></h2>
|
||||
<p class="lede">The mental model everyone arrives with: someone with credentials pushes manifests
|
||||
at the cluster. In GitOps nothing is pushed. A repository holds the desired state, an agent
|
||||
inside the cluster watches it, and the cluster pulls its own future from git. oc is for
|
||||
at the cluster. In the architecture this course teaches, nothing is pushed: a repository holds
|
||||
the desired state, an agent inside the cluster watches it, and the cluster pulls its own future
|
||||
from git. oc is for
|
||||
looking; git is for changing.</p>
|
||||
</header>
|
||||
<div class="pin">
|
||||
@@ -353,14 +366,19 @@
|
||||
ArgoCD is the same habit applied to <b>delivery</b>. An Application names a repo, a path and a
|
||||
revision - watch this branch of this repository - and the controller renders what it finds
|
||||
there, compares it against the live cluster, and syncs the difference. The deploy button is a
|
||||
git commit; the audit log is git log; code review <i>is</i> change control.</p>
|
||||
git commit; the change history is git log; code review <i>is</i> change control. Git is the
|
||||
record of <i>intent</i> - the cluster's own audit log and the reconciler's sync history still
|
||||
record what actually happened, including everything git never sees.</p>
|
||||
<p class="sayit">oc apply is for archaeology, not for change.</p>
|
||||
<h3>Pull, not push - the security inversion</h3>
|
||||
<p>The cluster <b>pulls</b>. No CI system, no laptop, no build pipeline holds a credential that
|
||||
can touch it - the agent inside holds a read-only deploy key, and the trust arrow points out.
|
||||
Compromise the build system and you can propose a change, which is visible; you cannot reach
|
||||
into production. Hand-edit a live object and the controller flags it OutOfSync - selfHeal puts
|
||||
it back, and what leaves git leaves the cluster. Rollback is <span class="mono">git revert</span>,
|
||||
<p>Here the cluster <b>pulls</b>. Push-based delivery exists and is still GitOps to many - this
|
||||
is the stronger variant, and worth choosing deliberately: no CI system, no laptop, no build
|
||||
pipeline holds a credential that can touch the cluster, because the agent inside holds a
|
||||
read-only deploy key and the trust arrow points out. Compromise the build system and you can
|
||||
propose a change, which is visible; you cannot reach into production. Hand-edit a live object
|
||||
and the controller flags it OutOfSync - <b>with selfHeal enabled</b> it puts the object back,
|
||||
and <b>with prune enabled</b> what leaves git leaves the cluster. Both are opt-in: without
|
||||
them the reconciler reports the drift and waits for a human. Rollback is <span class="mono">git revert</span>,
|
||||
which is why commit hygiene is an operational skill.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> Fighting the reconciler over SSH is arm-wrestling a
|
||||
machine that does not get tired - C-II's warning, now with a face. The fix is always
|
||||
@@ -375,8 +393,9 @@
|
||||
entitlements, reconciled like everything else.</p>
|
||||
<p class="sayit">Label the cluster; the app follows.</p>
|
||||
<h3>The one thing git never holds</h3>
|
||||
<p>Git holds everything except secrets - a secret in git is published, forever, to everyone who
|
||||
ever clones. So the pattern splits the reference from the value: git carries an ExternalSecret
|
||||
<p>Git holds everything except secrets. Commit a plaintext secret and you should treat it as
|
||||
<b>compromised from that moment</b>: deleting it later does not guarantee it is gone from
|
||||
history, forks, clones, CI caches or backups. So the pattern splits the reference from the value: git carries an ExternalSecret
|
||||
naming a <b>logical key</b>; a vault - Azure Key Vault in the worked example - holds the value;
|
||||
an operator inside the cluster exchanges one for the other at runtime. Rotation happens in the
|
||||
vault, never as a commit. That is why the vault floats apart in the scene above: it is never
|
||||
@@ -488,10 +507,13 @@
|
||||
functions every manifest calls. You are not writing YAML - you are writing a program whose
|
||||
output is YAML. So render locally, read the output, and lint what came out, not what went in.</p>
|
||||
<p class="sayit">Review the render, not just the template.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> ArgoCD deploys charts by running
|
||||
<span class="mono">helm template</span> - a pure render. Anything needing a live cluster or an
|
||||
install lifecycle (<span class="mono">lookup</span>, hooks) is silently dead there. Render the
|
||||
way your deployer renders.</p>
|
||||
<p class="fieldnote"><b>Field note.</b> ArgoCD renders charts with
|
||||
<span class="mono">helm template</span> rather than running <span class="mono">helm
|
||||
install</span>, so the lifecycle differs from Helm's own. <span class="mono">lookup</span>
|
||||
comes back empty - there is no live cluster at render time. Hooks are not dead, though: Argo
|
||||
maps Helm hooks onto its sync phases (pre-install and pre-upgrade become PreSync, post-install
|
||||
and post-upgrade become PostSync), while a few - rollback and test hooks - have no equivalent
|
||||
at all. Render the way your deployer renders, and check where your hooks actually land.</p>
|
||||
<h3>Contexts: the cluster's whole voice is one small file</h3>
|
||||
<p>The chart owns everything structural - resources, probes, security, policy. Each cluster owns
|
||||
one values file: names, addresses, sizes, flags. The context is deliberately values-only; the
|
||||
@@ -712,16 +734,16 @@ b{color:var(--ink)} .mono{font-family:var(--mono);font-size:.9em;color:var(--cya
|
||||
.legend .b{font-size:.76rem;line-height:1.5;color:var(--dim);margin:0}
|
||||
|
||||
.viewport.solo{grid-template-columns:1fr;max-width:980px}
|
||||
.crt{position:absolute;inset:22% 26%;display:flex;flex-direction:column;justify-content:center;
|
||||
gap:.35rem;padding:clamp(10px,2.5vw,26px);border:1px solid var(--line);border-radius:10px;
|
||||
.crt{position:absolute;inset:18% 22%;display:flex;flex-direction:column;justify-content:center;
|
||||
gap:.3rem;padding:clamp(10px,2.5vw,26px);border:1px solid var(--line);border-radius:10px;
|
||||
background:rgba(5,8,16,0.88);box-shadow:0 0 40px rgba(63,186,245,0.07) inset}
|
||||
.crt .line{font-family:var(--mono);font-size:clamp(.55rem,1.35vw,.85rem);line-height:1.5;
|
||||
color:var(--ink);margin:0;white-space:nowrap;overflow:hidden}
|
||||
color:var(--ink);margin:0;white-space:nowrap;padding-bottom:.12em}
|
||||
.crt .line.ok{color:var(--dim)}
|
||||
.crt .line.q{color:var(--magenta);margin-top:.5rem}
|
||||
body.fx .crt .line{opacity:0;transform:translateY(4px);transition:opacity .3s ease,transform .3s ease}
|
||||
body.fx .crt .line.on{opacity:1;transform:none}
|
||||
@media (max-width:700px){.crt{inset:16% 12%}}
|
||||
@media (max-width:700px){.crt{inset:10% 8%;gap:.2rem}.crt .line{line-height:1.7}}
|
||||
|
||||
|
||||
.prose{max-width:66ch;margin:0 auto;padding:5vh 24px 9vh}
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
{
|
||||
"name": "base",
|
||||
"title": "Base image",
|
||||
"blurb": "The OS layer everything else is stacked on. Pull it once, share it across every image on the host.",
|
||||
"blurb": "The foundation everything stacks on. Layers are content-addressed - an identical layer is stored once and reused by every image that references it.",
|
||||
"z": 0,
|
||||
"points": [
|
||||
[
|
||||
@@ -144,7 +144,7 @@
|
||||
{
|
||||
"name": "writable",
|
||||
"title": "Writable layer",
|
||||
"blurb": "Created fresh per container, thrown away when it dies. Anything written here is not in the image.",
|
||||
"blurb": "NOT part of the image. The runtime adds it when it creates the container, and discards it when that container dies.",
|
||||
"z": 0,
|
||||
"points": [
|
||||
[
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
{
|
||||
"name": "kubelet",
|
||||
"title": "kubelet",
|
||||
"blurb": "The node agent - the only component in the whole system that actually starts your container.",
|
||||
"blurb": "The node agent. It decides what should be running and tells the runtime over CRI - it does not create containers itself.",
|
||||
"z": 0,
|
||||
"points": [
|
||||
[
|
||||
@@ -147,7 +147,7 @@
|
||||
{
|
||||
"name": "engine",
|
||||
"title": "containerd / CRI-O",
|
||||
"blurb": "Speaks CRI: pulls images, launches containers. The engine the kubelet drives.",
|
||||
"blurb": "containerd or CRI-O: pulls the image and actually creates and starts the container.",
|
||||
"z": 0,
|
||||
"points": [
|
||||
[
|
||||
|
||||
@@ -495,7 +495,7 @@
|
||||
{
|
||||
"name": "discs",
|
||||
"title": "Volumes",
|
||||
"blurb": "Shared mounts - the only place anything survives the writable layer's death.",
|
||||
"blurb": "Declared once on the pod - but each container mounts the ones it needs. Sharing storage is opt-in.",
|
||||
"z": 0,
|
||||
"points": [
|
||||
[
|
||||
|
||||
Reference in New Issue
Block a user