site: directory landing + The Mirror (chapter 02)
The face of learn.bztmon.com is now a chapter index: 01 The Exploded Cluster (moved to /cluster) and 02 The Mirror (/mirror), with a forming-chapter ghost slot. The Mirror ships rev 3 of the dossier - six curated plates fingerprinted under /assets/, its scroll engine as an external fingerprinted script (CSP script-src self holds), diagrams as inline SVG, replay panes from real receipts. Assembler grows a shared document shell + per-page preflight gates; build.sh inline-script check now covers every page.
This commit is contained in:
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 38 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 25 KiB |
@@ -21,7 +21,10 @@ for f in pilot/arc.tpl.html pilot/manifest-docker.json pilot/manifest-cluster.js
|
||||
assets/dist/course-II-cluster.webp assets/dist/course-00-surround.webp \
|
||||
assets/dist/course-III-node.webp assets/dist/course-III-pod.webp \
|
||||
assets/dist/course-IV-traffic.webp assets/dist/course-V-orbit.webp \
|
||||
assets/dist/intro-establishing.webp; do
|
||||
assets/dist/intro-establishing.webp \
|
||||
pilot/mirror.tpl.html pilot/mirror.js pilot/index.tpl.html \
|
||||
assets/mirror/20.jpg assets/mirror/21.jpg assets/mirror/22.jpg \
|
||||
assets/mirror/23.jpg assets/mirror/24.jpg assets/mirror/25.jpg; do
|
||||
[[ -f "$f" ]] || { printf 'missing source: %s\n' "$f" >&2; exit 1; }
|
||||
done
|
||||
|
||||
@@ -29,11 +32,13 @@ dist="$here/dist"
|
||||
rm -rf -- "$dist"
|
||||
node pilot/assemble-arc.mjs "$dist"
|
||||
|
||||
# No inline JS may survive into the served HTML, or the CSP silently kills the page.
|
||||
if grep -oiE '<script[^>]*>' -- "$dist/index.html" | grep -qivE 'src='; then
|
||||
printf 'index.html contains an inline <script> — CSP script-src self would block it\n' >&2
|
||||
# No inline JS may survive into ANY served HTML, or the CSP silently kills the page.
|
||||
for page in "$dist"/*.html; do
|
||||
if grep -oiE '<script[^>]*>' -- "$page" | grep -qivE 'src='; then
|
||||
printf '%s contains an inline <script> — CSP script-src self would block it\n' "$page" >&2
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
printf 'built %s\n' "$dist"
|
||||
find "$dist" -type f -printf ' %-32P %8s bytes\n' | sort
|
||||
|
||||
+65
-31
@@ -115,41 +115,77 @@ const appRef = "/assets/" + appName;
|
||||
const DESC = "Interactive teardown of Kubernetes and OpenShift delivery: containers, Services, GitOps, image supply chains and Helm, explained with scroll-driven exploded diagrams.";
|
||||
const OG_IMG = SITE + heroUrls["__HERO_GITOPS__"];
|
||||
|
||||
const html = `<!doctype html>
|
||||
// shared document shell: every page on the site gets the same head discipline
|
||||
const shell = ({ title, desc, path, ogImg, scripts, bodyHtml }) => `<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>The Exploded Cluster</title>
|
||||
<meta name="description" content="${DESC}">
|
||||
<title>${title}</title>
|
||||
<meta name="description" content="${desc}">
|
||||
<meta name="theme-color" content="#070b14">
|
||||
<meta name="build-revision" content="${BUILD_REV}">
|
||||
<link rel="canonical" href="${SITE}/">
|
||||
<meta property="og:title" content="The Exploded Cluster">
|
||||
<meta property="og:description" content="${DESC}">
|
||||
<link rel="canonical" href="${SITE}${path}">
|
||||
<meta property="og:title" content="${title}">
|
||||
<meta property="og:description" content="${desc}">
|
||||
<meta property="og:type" content="website">
|
||||
<meta property="og:url" content="${SITE}/">
|
||||
<meta property="og:image" content="${OG_IMG}">
|
||||
<meta property="og:url" content="${SITE}${path}">
|
||||
<meta property="og:image" content="${ogImg}">
|
||||
<meta name="twitter:card" content="summary_large_image">
|
||||
<meta name="twitter:title" content="The Exploded Cluster">
|
||||
<meta name="twitter:description" content="${DESC}">
|
||||
<meta name="twitter:image" content="${OG_IMG}">
|
||||
<meta name="twitter:title" content="${title}">
|
||||
<meta name="twitter:description" content="${desc}">
|
||||
<meta name="twitter:image" content="${ogImg}">
|
||||
<link rel="icon" href="${favicon}">
|
||||
</head>
|
||||
<body>
|
||||
<a class="skip" href="#main-content">Skip to content</a>
|
||||
<main id="main-content">
|
||||
${body}
|
||||
${bodyHtml}
|
||||
</main>
|
||||
<script src="${gsapRef}"></script>
|
||||
<script src="${stRef}"></script>
|
||||
<script src="${appRef}"></script>
|
||||
${scripts.map(s => `<script src="${s}"></script>`).join("\n")}
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
|
||||
writeFileSync(join(DIST, "index.html"), html);
|
||||
writeFileSync(join(DIST, "build.json"), JSON.stringify({ revision: BUILD_REV, app: appName }) + "\n");
|
||||
const html = shell({
|
||||
title: "The Exploded Cluster", desc: DESC, path: "/cluster", ogImg: OG_IMG,
|
||||
scripts: [gsapRef, stRef, appRef], bodyHtml: body,
|
||||
});
|
||||
|
||||
// ---- The Mirror (chapter 02): fingerprint its plates + its app script -------
|
||||
const MIRROR_DESC = "A pull-through container registry taken apart: why a lab runs its own mirror, the chicken-and-egg loops inside it, and the scars that taught each rule.";
|
||||
let mirror = readFileSync(join(ROOT, "pilot/mirror.tpl.html"), "utf8");
|
||||
const mirrorPlates = {};
|
||||
for (const n of ["20", "21", "22", "23", "24", "25"]) {
|
||||
const { url } = emit(join(ROOT, "assets/mirror", `${n}.jpg`), `mirror-${n}`, "jpg");
|
||||
mirrorPlates[n] = url;
|
||||
mirror = mirror.replace(`__M${n}__`, url);
|
||||
}
|
||||
const mirrorJs = readFileSync(join(ROOT, "pilot/mirror.js"));
|
||||
const mirrorJsName = `mirror.${fp(mirrorJs)}.js`;
|
||||
writeFileSync(join(DIST, "assets", mirrorJsName), mirrorJs);
|
||||
mirror = mirror.replace("__MIRROR_JS__", "/assets/" + mirrorJsName);
|
||||
if (mirror.includes("__M")) throw new Error("unfilled mirror placeholder");
|
||||
const mirrorHtml = shell({
|
||||
title: "The Mirror", desc: MIRROR_DESC, path: "/mirror",
|
||||
ogImg: SITE + mirrorPlates["20"], scripts: [], bodyHtml: mirror,
|
||||
});
|
||||
|
||||
// ---- the directory (index): the chapters of what this place teaches ---------
|
||||
const DIR_DESC = "The Teaching Lab: scroll-driven teardowns of real infrastructure - container platforms, registries and the machinery of delivery, one exploded system per chapter.";
|
||||
let dir = readFileSync(join(ROOT, "pilot/index.tpl.html"), "utf8")
|
||||
.replace("__THUMB_CLUSTER__", heroUrls["__HERO_GITOPS__"])
|
||||
.replace("__THUMB_MIRROR__", mirrorPlates["20"]);
|
||||
if (dir.includes("__THUMB")) throw new Error("unfilled directory placeholder");
|
||||
const dirHtml = shell({
|
||||
title: "The Teaching Lab", desc: DIR_DESC, path: "/",
|
||||
ogImg: SITE + mirrorPlates["20"], scripts: [], bodyHtml: dir,
|
||||
});
|
||||
|
||||
writeFileSync(join(DIST, "index.html"), dirHtml);
|
||||
writeFileSync(join(DIST, "cluster.html"), html);
|
||||
writeFileSync(join(DIST, "mirror.html"), mirrorHtml);
|
||||
writeFileSync(join(DIST, "build.json"), JSON.stringify({ revision: BUILD_REV, app: appName, mirror: mirrorJsName }) + "\n");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Preflight gates: fail the build loudly rather than ship a known defect.
|
||||
@@ -157,20 +193,18 @@ writeFileSync(join(DIST, "build.json"), JSON.stringify({ revision: BUILD_REV, ap
|
||||
const fail = [];
|
||||
const must = (cond, msg) => { if (!cond) fail.push(msg); };
|
||||
|
||||
// no inline <script> may survive (CSP script-src 'self')
|
||||
must(!(html.match(/<script(?![^>]*src=)[^>]*>/gi) || []).length, "inline <script> present");
|
||||
// no unresolved template placeholders
|
||||
must(!/__[A-Z_]+__/.test(html), "unresolved __PLACEHOLDER__ in output");
|
||||
// exactly one <h1>
|
||||
must((html.match(/<h1[\s>]/gi) || []).length === 1, "expected exactly one <h1>");
|
||||
// unique ids
|
||||
const ids = [...html.matchAll(/\sid="([^"]+)"/g)].map(m => m[1]);
|
||||
must(new Set(ids).size === ids.length, "duplicate id attribute(s): " +
|
||||
ids.filter((v, k) => ids.indexOf(v) !== k).join(", "));
|
||||
// internal anchor targets resolve
|
||||
const idset = new Set(ids);
|
||||
for (const m of html.matchAll(/href="#([^"]+)"/g)) {
|
||||
if (m[1] && !idset.has(m[1])) fail.push("dangling internal link #" + m[1]);
|
||||
// per-page structural gates (CSP + document hygiene) for every page shipped
|
||||
for (const [pg, doc] of [["cluster", html], ["mirror", mirrorHtml], ["index", dirHtml]]) {
|
||||
must(!(doc.match(/<script(?![^>]*src=)[^>]*>/gi) || []).length, `${pg}: inline <script> present`);
|
||||
must(!/__[A-Z0-9_]+__/.test(doc), `${pg}: unresolved __PLACEHOLDER__ in output`);
|
||||
must((doc.match(/<h1[\s>]/gi) || []).length === 1, `${pg}: expected exactly one <h1>`);
|
||||
const pids = [...doc.matchAll(/\sid="([^"]+)"/g)].map(m => m[1]);
|
||||
must(new Set(pids).size === pids.length, `${pg}: duplicate id attribute(s): ` +
|
||||
pids.filter((v, k) => pids.indexOf(v) !== k).join(", "));
|
||||
const pidset = new Set(pids);
|
||||
for (const m of doc.matchAll(/href="#([^"]+)"/g)) {
|
||||
if (m[1] && !pidset.has(m[1])) fail.push(`${pg}: dangling internal link #` + m[1]);
|
||||
}
|
||||
}
|
||||
// banned / obsolete strings (user-visible copy)
|
||||
const banned = [
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
<style>
|
||||
:root{
|
||||
--void:#070b14; --ink:#c9d7ef; --dim:#7286a8; --line:#1b2740;
|
||||
--cyan:#3fbaf5; --magenta:#e879f9;
|
||||
--sans:ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
|
||||
--mono:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
body{margin:0;background:var(--void);color:var(--ink);font-family:var(--sans);
|
||||
-webkit-font-smoothing:antialiased}
|
||||
.wrap{max-width:56rem;margin:0 auto;padding:0 1.25rem}
|
||||
header.mast{padding:5.5rem 0 2.5rem}
|
||||
.eyebrow{font-family:var(--mono);font-size:.78rem;letter-spacing:.2em;color:var(--cyan);text-transform:uppercase;margin:0 0 .8rem}
|
||||
h1{font-size:clamp(2.2rem,6.5vw,3.6rem);line-height:1.05;margin:0 0 1rem;text-wrap:balance;font-weight:700}
|
||||
h1 em{color:var(--cyan);font-style:normal}
|
||||
.lede{max-width:38rem;color:var(--dim);font-size:1.1rem;line-height:1.6;margin:0}
|
||||
.chapters{display:grid;gap:1.4rem;margin:3rem 0 5rem}
|
||||
.card{display:grid;grid-template-columns:minmax(180px,300px) 1fr;gap:0;border:1px solid var(--line);
|
||||
border-radius:.6rem;overflow:hidden;background:#0a101d;text-decoration:none;color:inherit;
|
||||
transition:border-color .25s}
|
||||
.card:hover,.card:focus-visible{border-color:var(--cyan)}
|
||||
.card .thumb{background:#050810;display:flex;align-items:center;justify-content:center;overflow:hidden}
|
||||
.card .thumb img{width:100%;height:100%;object-fit:cover;display:block}
|
||||
.card .meta{padding:1.4rem 1.5rem}
|
||||
.card .k{font-family:var(--mono);font-size:.72rem;letter-spacing:.18em;color:var(--dim);text-transform:uppercase}
|
||||
.card h2{margin:.5rem 0 .5rem;font-size:1.6rem;line-height:1.15}
|
||||
.card h2 .arrow{color:var(--cyan);transition:transform .25s;display:inline-block}
|
||||
.card:hover h2 .arrow{transform:translateX(6px)}
|
||||
.card .outcome{margin:0;color:var(--dim);font-size:.98rem;line-height:1.55}
|
||||
.card .outcome b{color:var(--ink);font-weight:600;font-family:var(--mono);font-size:.78rem;
|
||||
letter-spacing:.14em;text-transform:uppercase;color:var(--cyan)}
|
||||
.card.ghost{border-style:dashed;opacity:.55}
|
||||
.card.ghost .thumb{min-height:130px;font-family:var(--mono);color:var(--dim);font-size:.8rem;letter-spacing:.15em}
|
||||
footer{border-top:1px solid var(--line);padding:1.4rem 0 3rem;color:var(--dim);
|
||||
font-family:var(--mono);font-size:.82rem}
|
||||
footer a{color:var(--cyan)}
|
||||
a.skip{position:absolute;left:-9999px}
|
||||
a.skip:focus{left:1rem;top:1rem;background:var(--void);color:var(--cyan);
|
||||
font-family:var(--mono);font-size:.8rem;padding:10px 16px;border:1px solid var(--cyan);border-radius:6px}
|
||||
:focus-visible{outline:2px solid var(--cyan);outline-offset:2px}
|
||||
@media (max-width:640px){.card{grid-template-columns:1fr}.card .thumb{max-height:170px}}
|
||||
</style>
|
||||
|
||||
<header class="mast wrap">
|
||||
<p class="eyebrow">bztmon · the teaching lab</p>
|
||||
<h1>Taken apart, <em>on purpose.</em></h1>
|
||||
<p class="lede">Scroll-driven teardowns of machinery this lab actually runs. Each chapter is one
|
||||
system, exploded - the diagrams move while the words explain them, and every claim traces back to
|
||||
a running estate: configs lifted from the fleet, incidents told with their receipts.</p>
|
||||
</header>
|
||||
|
||||
<nav class="wrap" aria-label="Chapters">
|
||||
<div class="chapters">
|
||||
|
||||
<a class="card" href="/cluster">
|
||||
<span class="thumb"><img src="__THUMB_CLUSTER__" alt="" width="1200" height="675" loading="eager" decoding="async"></span>
|
||||
<span class="meta">
|
||||
<span class="k">Chapter 01 · Dossier 004</span>
|
||||
<h2>The Exploded Cluster <span class="arrow">→</span></h2>
|
||||
<p class="outcome"><b>Outcome</b> — how software reaches a container platform: images,
|
||||
clusters, nodes, pods, traffic, GitOps, the image supply chain and Helm - eleven courses,
|
||||
each one machine drawn apart and wired to your scroll.</p>
|
||||
</span>
|
||||
</a>
|
||||
|
||||
<a class="card" href="/mirror">
|
||||
<span class="thumb"><img src="__THUMB_MIRROR__" alt="" width="1600" height="873" loading="eager" decoding="async"></span>
|
||||
<span class="meta">
|
||||
<span class="k">Chapter 02 · Dossier 005</span>
|
||||
<h2>The Mirror <span class="arrow">→</span></h2>
|
||||
<p class="outcome"><b>Outcome</b> — a pull-through container registry taken apart: why a
|
||||
lab runs its own mirror, the four chicken-and-egg loops hiding inside it, and the scars that
|
||||
taught each rule - told with the receipts.</p>
|
||||
</span>
|
||||
</a>
|
||||
|
||||
<div class="card ghost" aria-hidden="true">
|
||||
<span class="thumb">CHAPTER 03 · FORMING</span>
|
||||
<span class="meta">
|
||||
<span class="k">Next teardown</span>
|
||||
<h2>To be decided</h2>
|
||||
<p class="outcome">The lab keeps running; the chapters keep forming.</p>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
</div>
|
||||
</nav>
|
||||
|
||||
<footer class="wrap">
|
||||
The Teaching Lab · every page here describes systems the author operates ·
|
||||
<a href="https://www.bztmon.com">bztmon.com</a>
|
||||
</footer>
|
||||
@@ -0,0 +1,39 @@
|
||||
(function(){
|
||||
var rm = matchMedia('(prefers-reduced-motion: reduce)').matches;
|
||||
var bar = document.getElementById('bar');
|
||||
var s20 = document.querySelector('#s20 img');
|
||||
var ring = document.getElementById('ring');
|
||||
var glows = [].slice.call(document.querySelectorAll('.s22-glow'));
|
||||
var s24 = document.querySelector('#s24 img');
|
||||
function prog(el){
|
||||
var r = el.getBoundingClientRect(), vh = innerHeight;
|
||||
return Math.max(0, Math.min(1, (vh - r.top) / (vh + r.height)));
|
||||
}
|
||||
function onScroll(){
|
||||
var d = document.documentElement;
|
||||
bar.style.width = (100 * d.scrollTop / (d.scrollHeight - innerHeight)) + '%';
|
||||
if (rm) return;
|
||||
var p20 = prog(s20); /* dock-and-stow: assembles inward */
|
||||
s20.style.transform = 'scale(' + (1.07 - 0.07 * Math.min(1, p20 * 1.6)) + ') translateY(' + (26 - 26 * Math.min(1, p20 * 1.6)) + 'px)';
|
||||
var p21 = prog(ring); /* ring-walk: the cycle turns */
|
||||
ring.style.transform = 'rotate(' + (p21 * 300) + 'deg)';
|
||||
glows.forEach(function(g, i){ /* checkpoint procession: three beats */
|
||||
var p = prog(g.parentElement);
|
||||
var band = 0.18 + i * 0.22;
|
||||
g.style.opacity = Math.max(0, Math.min(0.85, (p - band) * 6));
|
||||
});
|
||||
var p24 = prog(s24); /* two-plane parallax */
|
||||
s24.style.transform = 'translateY(' + ((p24 - 0.5) * -26) + 'px) scale(1.03)';
|
||||
}
|
||||
addEventListener('scroll', onScroll, {passive: true});
|
||||
addEventListener('resize', onScroll);
|
||||
onScroll();
|
||||
if (!rm){ /* the jam breathes; nothing else does */
|
||||
var pulse = document.getElementById('pulse'), t0 = null;
|
||||
requestAnimationFrame(function breathe(t){
|
||||
if (t0 === null) t0 = t;
|
||||
pulse.style.opacity = 0.18 + 0.2 * (0.5 + 0.5 * Math.sin((t - t0) / 520));
|
||||
requestAnimationFrame(breathe);
|
||||
});
|
||||
}
|
||||
})();
|
||||
@@ -0,0 +1,540 @@
|
||||
<style>
|
||||
/* Committed single-theme: a dark cinema world. Ground painted explicitly. */
|
||||
:root{
|
||||
--void:#070b14; --void-2:#0a0f1b; --ink:#dfe6f0; --ink-dim:#8b95a7;
|
||||
--cyan:#3fbaf5; --magenta:#e879f9; --line:#1c2434; --card:#0d1424;
|
||||
--mono:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;
|
||||
--sans:ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Helvetica,Arial,sans-serif;
|
||||
}
|
||||
*{box-sizing:border-box}
|
||||
html{scroll-behavior:smooth}
|
||||
body{margin:0;background:var(--void);color:var(--ink);
|
||||
font-family:var(--sans);font-size:1.06rem;line-height:1.65}
|
||||
#bar{position:fixed;top:0;left:0;height:2px;width:0;background:linear-gradient(90deg,var(--cyan),var(--magenta));z-index:9}
|
||||
.wrap{max-width:46rem;margin:0 auto;padding:0 1.25rem}
|
||||
header.hero{min-height:92vh;display:flex;flex-direction:column;justify-content:center;padding:4rem 0 2rem}
|
||||
.eyebrow{font-family:var(--mono);font-size:.78rem;letter-spacing:.22em;color:var(--cyan);text-transform:uppercase}
|
||||
h1{font-family:var(--sans);font-stretch:semi-condensed;font-weight:700;font-size:clamp(3rem,9vw,5.2rem);
|
||||
line-height:.95;margin:.5rem 0 1rem;text-wrap:balance;letter-spacing:.01em}
|
||||
.outcome{max-width:34rem;color:var(--ink-dim);font-size:1.12rem}
|
||||
.outcome b{color:var(--ink);font-weight:600}
|
||||
.cue{margin-top:3rem;font-family:var(--mono);font-size:.75rem;color:var(--ink-dim);letter-spacing:.15em}
|
||||
.cue::after{content:"";display:block;width:1px;height:56px;margin-top:.6rem;
|
||||
background:linear-gradient(var(--cyan),transparent)}
|
||||
h2{font-family:var(--sans);font-weight:600;font-size:2rem;letter-spacing:.02em;
|
||||
margin:4.5rem 0 1rem;text-wrap:balance}
|
||||
h2 .n{color:var(--cyan);font-family:var(--mono);font-size:1.05rem;vertical-align:.35em;margin-right:.6rem;letter-spacing:.1em}
|
||||
h3{font-family:var(--sans);font-weight:600;font-size:1.3rem;margin:2.2rem 0 .6rem}
|
||||
p{margin:.9rem 0}
|
||||
.scene{margin:3rem calc(50% - 50vw);width:100vw;overflow:hidden;position:relative}
|
||||
.scene .inner{max-width:72rem;margin:0 auto;padding:0 1rem;position:relative}
|
||||
.scene img{width:100%;height:auto;display:block;border-radius:.4rem}
|
||||
.scene .cap{font-family:var(--mono);font-size:.72rem;color:var(--ink-dim);
|
||||
letter-spacing:.14em;text-transform:uppercase;text-align:center;margin-top:.7rem}
|
||||
.sayit{border-left:3px solid var(--magenta);margin:2rem 0;padding:.4rem 0 .4rem 1.1rem;
|
||||
font-size:1.28rem;font-family:var(--sans);font-weight:600;line-height:1.35;color:#fff}
|
||||
.sayit .k{display:block;font-family:var(--mono);font-size:.7rem;letter-spacing:.2em;color:var(--magenta);margin-bottom:.3rem}
|
||||
.note{background:var(--card);border:1px solid var(--line);border-left:3px solid var(--cyan);
|
||||
border-radius:.45rem;padding:1rem 1.2rem;margin:1.6rem 0;font-size:.98rem}
|
||||
.note .k{font-family:var(--mono);font-size:.72rem;letter-spacing:.18em;color:var(--cyan)}
|
||||
.clause{background:linear-gradient(135deg,rgba(232,121,249,.07),transparent 60%);
|
||||
border:1px solid var(--line);border-radius:.45rem;padding:1rem 1.2rem;margin:1.6rem 0;font-size:.98rem}
|
||||
.clause .k{font-family:var(--mono);font-size:.72rem;letter-spacing:.18em;color:var(--magenta)}
|
||||
pre{background:#050810;border:1px solid var(--line);border-radius:.5rem;padding:1rem 1.1rem;
|
||||
overflow-x:auto;font-family:var(--mono);font-size:.83rem;line-height:1.55;color:#c8d2e0}
|
||||
pre.pane{border-left:3px solid var(--cyan)}
|
||||
pre .cm{color:#5a6478}
|
||||
pre .good{color:#7fe0a7}
|
||||
pre .bad{color:#ff8fa3}
|
||||
.pane-k{font-family:var(--mono);font-size:.72rem;letter-spacing:.18em;color:var(--ink-dim);
|
||||
text-transform:uppercase;margin:1.6rem 0 .4rem}
|
||||
.diagram{background:var(--card);border:1px solid var(--line);border-radius:.55rem;
|
||||
padding:1.2rem;margin:2rem 0;overflow-x:auto}
|
||||
.diagram .k{font-family:var(--mono);font-size:.72rem;letter-spacing:.18em;color:var(--cyan);display:block;margin-bottom:.8rem}
|
||||
.diagram svg{width:100%;height:auto;display:block}
|
||||
.diagram .why{font-size:.9rem;color:var(--ink-dim);margin:.8rem 0 0}
|
||||
ol{padding-left:1.3rem}
|
||||
ol li{margin:.7rem 0}
|
||||
ul{padding-left:1.2rem}
|
||||
ul li{margin:.55rem 0}
|
||||
em{color:#fff;font-style:italic}
|
||||
.traps p{margin:1.1rem 0}
|
||||
.traps b{color:#fff}
|
||||
footer{margin:5rem 0 4rem;border-top:1px solid var(--line);padding-top:1.4rem;
|
||||
color:var(--ink-dim);font-size:.88rem;font-family:var(--mono)}
|
||||
a{color:var(--cyan)}
|
||||
:focus-visible{outline:2px solid var(--cyan);outline-offset:2px}
|
||||
.s21-ring{position:absolute;inset:0;pointer-events:none;mix-blend-mode:screen;opacity:.5;
|
||||
background:conic-gradient(from 0deg at 50% 50%, transparent 0deg, rgba(63,186,245,.28) 24deg, transparent 60deg);}
|
||||
.s22-glow{position:absolute;width:24%;aspect-ratio:1;border-radius:50%;pointer-events:none;
|
||||
mix-blend-mode:screen;opacity:0;background:radial-gradient(circle,rgba(63,186,245,.5),transparent 65%)}
|
||||
.s23-pulse{position:absolute;left:34%;top:44%;width:26%;aspect-ratio:1;border-radius:50%;pointer-events:none;
|
||||
mix-blend-mode:screen;opacity:.25;background:radial-gradient(circle,rgba(232,121,249,.55),transparent 62%)}
|
||||
.cursor::after{content:"_";color:var(--cyan);animation:blink 1.1s steps(1) infinite}
|
||||
@keyframes blink{50%{opacity:0}}
|
||||
@media (prefers-reduced-motion: reduce){
|
||||
html{scroll-behavior:auto}
|
||||
.cursor::after{animation:none}
|
||||
.s21-ring,.s23-pulse{display:none}
|
||||
.s22-glow{opacity:.35}
|
||||
}
|
||||
</style>
|
||||
|
||||
<div id="bar"></div>
|
||||
|
||||
<header class="hero wrap">
|
||||
<nav style="position:absolute;top:1.2rem;left:1.25rem;font-family:var(--mono);font-size:.75rem"><a href="/" style="text-decoration:none">← index</a></nav>
|
||||
<div class="eyebrow">Dossier / 005 · The Teaching Lab</div>
|
||||
<h1>The Mirror</h1>
|
||||
<p class="outcome"><b>Outcome</b> — a teardown of a pull-through container registry:
|
||||
why a homelab runs its own mirror, the four chicken-and-egg loops hiding inside it,
|
||||
and the scars that taught each rule. Everything on this page is running, or broke,
|
||||
in the lab that published it.</p>
|
||||
<div class="cue">SCROLL</div>
|
||||
</header>
|
||||
|
||||
<div class="wrap">
|
||||
|
||||
<h2><span class="n">00</span>Cold open — the build that "hung"</h2>
|
||||
|
||||
<div class="scene" id="s20"><div class="inner">
|
||||
<img src="__M20__" alt="An armoured way-station machine exploded into seven floating parts: hexagonal hull, roof plate, cassette shelf of glowing bricks, three intake pods, an output nozzle." width="1600" height="873" decoding="async">
|
||||
<div class="cap">Scene 20 · the way-station · motion: dock-and-stow (assembles as you scroll)</div>
|
||||
</div></div>
|
||||
|
||||
<p>Nobody sets out to run a registry. You inherit the need the day a CI build stalls
|
||||
for no reason you can see.</p>
|
||||
<p>The lab's website builds from an ordinary Dockerfile. One base image came from
|
||||
Docker Hub, pulled anonymously, and one evening the build simply... sat there. No
|
||||
error visible. No progress. Here is the accurate version: the registry does not go
|
||||
silent when it rate-limits you — it answers loudly, with an explicit 429 and the
|
||||
string <code style="font-family:var(--mono)">toomanyrequests</code>. The <em>silence</em>
|
||||
was manufactured closer to home, by a retry layer swallowing that answer and trying
|
||||
again, backoff stacked on backoff, until a routine build read like a hung machine.
|
||||
The swallower, recovered from the build script's own history, was a double retry
|
||||
stack: an outer shell wrapper allowing four attempts with a growing backoff, wrapped
|
||||
around the build tool's own three retries — up to twelve tries before anything
|
||||
surfaced. It was added, in fairness, to ride out transient 502s; nets that catch 502s
|
||||
catch 429s too, and a limit is not transient.</p>
|
||||
|
||||
<div class="pane-k">Read-only pane · receipt 00a · the silence, as committed to the repo</div>
|
||||
<pre class="pane cursor">retry() {
|
||||
local n=0 max=4
|
||||
until "$@"; do
|
||||
n=$((n+1)); [ "$n" -ge "$max" ] && { echo ">> failed after ${max} attempts" >&2; return 1; }
|
||||
echo ">> attempt ${n} failed, retrying in $((n*8))s..." >&2; sleep $((n*8))
|
||||
done
|
||||
}
|
||||
retry buildah build --retry 3 --retry-delay 5s ...
|
||||
<span class="cm"># four attempts, each containing three - the 429 answered every one of them</span></pre>
|
||||
|
||||
<p>The cause was boring and invisible: anonymous pulls from the world's most popular
|
||||
registry are rate-limited, and the base image tag was not pinned, so every single
|
||||
build asked Docker Hub the same question again. The fix for THAT bug is nearly one
|
||||
line — pin the digest. Nearly, because pinning only ends the conversation when a
|
||||
local cache already holds the answer; an ephemeral runner with a cold store still
|
||||
asks Hub for the manifest behind that digest every time. The pin stops the
|
||||
re-resolution, not the round-trip. Which surfaced the sharper question:</p>
|
||||
|
||||
<div class="sayit"><span class="k">SAY IT</span>Why does a fleet of machines ask the
|
||||
public internet for the same bytes, hundreds of times, forever?</div>
|
||||
|
||||
<p>The answer a production platform gives is a mirror: one machine that fetches an
|
||||
image once, caches it, and serves every later request from the shelf. The lab built
|
||||
one. This dossier takes it apart.</p>
|
||||
|
||||
<div class="note"><span class="k">FIELD NOTE</span><br>The mirror did not fix the red X
|
||||
in CI. That red turned out to be a separate upstream bug in the CI system's log
|
||||
finalise step — cosmetic, tolerated, documented. Two problems, one symptom.
|
||||
Diagnose them separately or you will "fix" the wrong one and declare victory.</div>
|
||||
|
||||
<h2><span class="n">01</span>Anatomy — one cache, five upstreams</h2>
|
||||
|
||||
<p>The mirror here is zot: a single OCI registry, one namespace on the platform, one
|
||||
50-gigabyte cache volume, digest-pinned like everything else in the fleet. It fronts
|
||||
five public registries — Docker Hub, GitHub's, Quay, the Kubernetes project
|
||||
registry, and NVIDIA's — and it works in one of three modes per upstream:</p>
|
||||
<ul>
|
||||
<li><b>onDemand</b> — pure pull-through. A miss fetches from upstream, caches, serves.
|
||||
The second pull is instant and never leaves the building.</li>
|
||||
<li><b>polled sync</b> — cache plus refresh on an interval, bounded by tag filters so a
|
||||
poll cannot drag a whole vendor catalogue into the shelf.</li>
|
||||
<li><b>pre-seed</b> — sync-only, for images that must be present before anything asks.</li>
|
||||
</ul>
|
||||
<p>Docker Hub gets onDemand ONLY. It has no catalogue API worth polling and the rate
|
||||
limits that started this story punish enthusiasm.</p>
|
||||
<p>The storage settings are not defaults; each one bought something:</p>
|
||||
|
||||
<pre>"storage": {
|
||||
"commit": true,
|
||||
"dedupe": true,
|
||||
"gc": true,
|
||||
"gcDelay": "1h",
|
||||
"gcInterval": "24h"
|
||||
}</pre>
|
||||
|
||||
<p><code style="font-family:var(--mono)">commit</code> forces writes to disk before
|
||||
acknowledging — crash-safety for a box that might lose power.
|
||||
<code style="font-family:var(--mono)">dedupe</code> hard-links identical layers, which
|
||||
matters enormously when five upstreams ship the same base layers under different
|
||||
names. Garbage collection is delayed so a slow client mid-pull never watches its
|
||||
blobs vanish.</p>
|
||||
|
||||
<p>The consumers point at the mirror in their runtime config. On this platform that
|
||||
is a machine-level mirror list — and the FALLBACK SEMANTICS hiding behind this
|
||||
short stanza are the most load-bearing thing in the whole dossier. Hold that thought
|
||||
for chapter 02.</p>
|
||||
|
||||
<div class="diagram"><span class="k">Diagram D1 · the pull path</span>
|
||||
<svg viewBox="0 0 760 240" role="img" aria-label="Pull path: node asks mirror; hit serves from cache; miss fetches upstream then caches; mirror down falls through to origin">
|
||||
<defs><marker id="a" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10z" fill="#3fbaf5"/></marker>
|
||||
<marker id="am" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10z" fill="#e879f9"/></marker></defs>
|
||||
<rect x="10" y="95" width="120" height="50" rx="8" fill="none" stroke="#3fbaf5"/><text x="70" y="125" fill="#dfe6f0" font-size="15" text-anchor="middle" font-family="IBM Plex Mono">node</text>
|
||||
<rect x="300" y="95" width="150" height="50" rx="8" fill="none" stroke="#3fbaf5"/><text x="375" y="118" fill="#dfe6f0" font-size="15" text-anchor="middle" font-family="IBM Plex Mono">mirror</text><text x="375" y="136" fill="#8b95a7" font-size="11" text-anchor="middle" font-family="IBM Plex Mono">(the shelf)</text>
|
||||
<rect x="600" y="30" width="150" height="50" rx="8" fill="none" stroke="#8b95a7"/><text x="675" y="60" fill="#dfe6f0" font-size="15" text-anchor="middle" font-family="IBM Plex Mono">origin</text>
|
||||
<line x1="130" y1="120" x2="295" y2="120" stroke="#3fbaf5" marker-end="url(#a)"/><text x="210" y="110" fill="#8b95a7" font-size="12" text-anchor="middle" font-family="IBM Plex Mono">pull</text>
|
||||
<path d="M450 108 Q 520 70 595 58" fill="none" stroke="#3fbaf5" stroke-dasharray="5 4" marker-end="url(#a)"/><text x="520" y="62" fill="#8b95a7" font-size="12" text-anchor="middle" font-family="IBM Plex Mono">miss: fetch + cache</text>
|
||||
<path d="M375 145 Q 375 190 320 190 L 150 190 Q 130 190 130 150" fill="none" stroke="#3fbaf5" marker-end="url(#a)"/><text x="255" y="210" fill="#8b95a7" font-size="12" text-anchor="middle" font-family="IBM Plex Mono">hit: served from cache</text>
|
||||
<path d="M120 100 Q 300 20 595 42" fill="none" stroke="#e879f9" stroke-dasharray="3 5" marker-end="url(#am)"/><text x="300" y="30" fill="#e879f9" font-size="12" text-anchor="middle" font-family="IBM Plex Mono">mirror down: implicit fallback (skipFallback: false)</text>
|
||||
</svg>
|
||||
<p class="why">The decision this explains: the fallback is the platform's implicit
|
||||
default — the design choice is refusing to disable it.</p></div>
|
||||
|
||||
<pre>machine:
|
||||
registries:
|
||||
mirrors:
|
||||
docker.io:
|
||||
endpoints:
|
||||
- https://zot.bztmon.org
|
||||
ghcr.io:
|
||||
endpoints:
|
||||
- https://zot.bztmon.org</pre>
|
||||
|
||||
<p>Read that carefully: the mirror is the ONLY listed endpoint, and that is still not
|
||||
a hard dependency — because this platform falls back to the origin registry
|
||||
IMPLICITLY unless you set <code style="font-family:var(--mono)">skipFallback: true</code>.
|
||||
The lab's first version of this config listed the upstream as an explicit second
|
||||
endpoint; the same-day refinement removed it after realising it merely rendered a
|
||||
duplicate of what the default already guaranteed. The lesson generalises: know which
|
||||
of your safety nets are things you built, and which are defaults you merely have not
|
||||
broken.</p>
|
||||
|
||||
<p>Upstream credentials — a Docker Hub login to lift rate limits, a vendor key for
|
||||
the GPU registry — live in ONE mounted secret, delivered by the cluster's secret
|
||||
operator under logical names, never in the runtime config. That split is not
|
||||
tidiness; it follows a reload asymmetry worth stating precisely. On this platform,
|
||||
the mirror ENDPOINT list hot-reloads — apply it and the runtime picks it up live,
|
||||
no reboot. Node-level registry AUTH does not: credentials in the machine config take
|
||||
effect only after a reboot. The mounted secret rotates with a restart of one pod. So
|
||||
endpoints and pod-side secrets can move cheaply; node auth is the expensive one —
|
||||
and that asymmetry, not neatness, decides where each credential lives and the whole
|
||||
rollout order in chapter 02. Both behaviours were observed on Talos v1.13.4 with
|
||||
containerd 2.2.4 (June–July 2026); treat the exact split as version-dependent and
|
||||
re-test on upgrades.</p>
|
||||
|
||||
<div class="clause"><span class="k">THE HOMELAB CLAUSE</span><br>In production this is
|
||||
not one pod with a volume. The mirror sits on dedicated storage — a NAS-class
|
||||
array or object store — sized for the whole estate's catalogue, and it is itself
|
||||
backed up, because in a disconnected site the mirror IS the software supply. The
|
||||
lab's 50Gi local volume with a nightly backup is the same organ, one size smaller.</div>
|
||||
|
||||
<h2><span class="n">02</span>Chicken and egg — four loops, four answers</h2>
|
||||
|
||||
<div class="scene" id="s21"><div class="inner">
|
||||
<img src="__M21__" alt="Six machine parts arranged on an invisible ring around empty centre - a large cyan way-station at twelve o'clock and a quarter-scale magenta replica of it among the orbiting parts." width="1600" height="893" decoding="async">
|
||||
<div class="s21-ring" id="ring"></div>
|
||||
<div class="cap">Scene 21 · the loop · motion: ring-walk (the cycle turns as you scroll)</div>
|
||||
</div></div>
|
||||
|
||||
<p>Every infrastructure service eventually meets the question: what do you depend on,
|
||||
and what happens when you ARE the dependency? The mirror has four of these loops, and
|
||||
each one got a different answer. This chapter is the reason this page exists.</p>
|
||||
|
||||
<h3>Loop 1 — the mirror's own image comes through the mirror</h3>
|
||||
<p>zot runs as a container. That container's image lives on one of the very
|
||||
registries zot mirrors. So when the node that hosts zot boots, it asks... zot. Which
|
||||
is not running, because the node is booting.</p>
|
||||
<p>The answer is the fallback you met in chapter 01: <b>mirror listed, origin
|
||||
implicit</b>. The runtime tries the mirror, fails fast, and falls through to the real
|
||||
registry because nobody set <code style="font-family:var(--mono)">skipFallback: true</code>.
|
||||
The loop breaks itself, and the thing that breaks it is a platform default doing
|
||||
quiet duty as a bootstrap protocol — the design decision here is the restraint of
|
||||
not turning it off.</p>
|
||||
|
||||
<div class="sayit"><span class="k">SAY IT</span>The fallback is not a compromise. The
|
||||
fallback IS the design.</div>
|
||||
|
||||
<div class="note"><span class="k">FIELD NOTE</span><br>This was proven by accident. A
|
||||
config change shipped with a flag the registry refused to start with
|
||||
(<code style="font-family:var(--mono)">preserveDigest</code> demands
|
||||
<code style="font-family:var(--mono)">http.compat</code> — a real pairing rule,
|
||||
learn it from this page instead of from the crash). The mirror crashlooped. And
|
||||
nothing else broke: every node quietly fell through to upstream and the fleet never
|
||||
noticed. An unplanned failover test, passed. The postmortem produced the standing
|
||||
rule: run <code style="font-family:var(--mono)">zot verify</code> on the config BEFORE
|
||||
merging, in a throwaway pod, every time.</div>
|
||||
|
||||
<h3>Loop 2 — the recovery tooling deliberately ignores the mirror</h3>
|
||||
<p>The fleet's rescue tooling — the automation that health-checks and rebuilds the
|
||||
platform — could pull its execution image through the mirror like everything else.
|
||||
It does not. Its image is cached on the operations box, outside the cluster entirely.</p>
|
||||
<p>Because a recovery tool that depends on the thing it recovers is not a recovery
|
||||
tool. It is a passenger.</p>
|
||||
|
||||
<h3>Loop 3 — you cannot turn authentication on everywhere at once</h3>
|
||||
|
||||
<div class="scene" id="s22"><div class="inner" style="position:relative">
|
||||
<img src="__M22__" alt="Four parts left to right: a waiting keyed cartridge, a small glowing key wedge, a tall gate frame with cyan inner glow, and a cartridge beyond the gate with its keyway lit." width="1600" height="873" decoding="async">
|
||||
<div class="s22-glow" style="left:16%;top:30%" data-band="0"></div>
|
||||
<div class="s22-glow" style="left:44%;top:28%" data-band="1"></div>
|
||||
<div class="s22-glow" style="left:70%;top:26%" data-band="2"></div>
|
||||
<div class="cap">Scene 22 · the proving gate · motion: checkpoint procession (three beats, left to right)</div>
|
||||
</div></div>
|
||||
|
||||
<p>The target state is a mirror that refuses anonymous pulls. The path there is a
|
||||
sequencing puzzle:</p>
|
||||
<ol>
|
||||
<li>Auth in the node's registry config requires a REBOOT to take effect — mirror
|
||||
endpoints reload live, credentials do not. (Asymmetries like this decide rollout
|
||||
order; find them before you start.)</li>
|
||||
<li>The runtime's fall-back-on-401 behaviour is unreliable enough to carry open
|
||||
upstream issues — so a node whose auth is wrong does not gracefully degrade, it
|
||||
just fails to pull.</li>
|
||||
<li>Therefore: anonymous read STAYS ON while every node gets its auth config and its
|
||||
reboot, and each node must PROVE itself before it counts. And here the gate hides a
|
||||
trap of its own: while anonymous read is still on, pulling an ordinary repo proves
|
||||
NOTHING about auth — a node whose credentials never applied sends no
|
||||
authorisation header at all, the mirror happily serves it as an anonymous reader,
|
||||
and the gate false-passes the exact node it exists to catch. The canary must
|
||||
therefore live in a repo whose access policy DENIES anonymous read, so a successful
|
||||
pull can only mean an authenticated pull. Pass THAT, and the node is trusted. Only
|
||||
when every node has passed does anonymous flip off fleet-wide.</li>
|
||||
</ol>
|
||||
|
||||
<p>The lab's live policy today is the pre-flip shape — one glob, anonymous read on
|
||||
— and the canary repo gets its own entry carrying NO anonymous policy, so its
|
||||
pulls demand credentials while the rest of the shelf stays open:</p>
|
||||
|
||||
<pre>"accessControl": {
|
||||
"repositories": {
|
||||
"**": {
|
||||
"anonymousPolicy": ["read"],
|
||||
"policies": [
|
||||
{ "users": ["zot-push"], "actions": ["read", "create", "update"] },
|
||||
{ "users": ["zot-pull"], "actions": ["read"] }
|
||||
]
|
||||
},
|
||||
"canary/**": {
|
||||
"policies": [
|
||||
{ "users": ["zot-pull", "zot-push"], "actions": ["read"] }
|
||||
]
|
||||
}
|
||||
}
|
||||
}</pre>
|
||||
|
||||
<p>That per-node proof is a hard gate precisely because the failure mode is silent. A
|
||||
node that looks fine and cannot pull is a time bomb with a pleasant dashboard.</p>
|
||||
|
||||
<div class="diagram"><span class="k">Diagram D2 · the auth-flip ladder</span>
|
||||
<svg viewBox="0 0 760 200" role="img" aria-label="Five rungs: anonymous on, auth staged, reboot, authenticated canary proof, anonymous off. A failing node loops from the proof rung back to reboot.">
|
||||
<defs><marker id="b" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10z" fill="#3fbaf5"/></marker>
|
||||
<marker id="bm" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10z" fill="#e879f9"/></marker></defs>
|
||||
<g font-family="IBM Plex Mono" font-size="11" text-anchor="middle">
|
||||
<rect x="8" y="70" width="128" height="46" rx="8" fill="none" stroke="#8b95a7"/><text x="72" y="90" fill="#dfe6f0">anon read ON</text><text x="72" y="105" fill="#8b95a7">baseline</text>
|
||||
<rect x="163" y="70" width="128" height="46" rx="8" fill="none" stroke="#3fbaf5"/><text x="227" y="90" fill="#dfe6f0">auth staged</text><text x="227" y="105" fill="#8b95a7">inert in config</text>
|
||||
<rect x="318" y="70" width="128" height="46" rx="8" fill="none" stroke="#3fbaf5"/><text x="382" y="90" fill="#dfe6f0">node reboots</text><text x="382" y="105" fill="#8b95a7">auth goes live</text>
|
||||
<rect x="473" y="70" width="128" height="46" rx="8" fill="none" stroke="#e879f9"/><text x="537" y="90" fill="#dfe6f0">canary pull</text><text x="537" y="105" fill="#e879f9">MUST authenticate</text>
|
||||
<rect x="628" y="70" width="124" height="46" rx="8" fill="none" stroke="#3fbaf5"/><text x="690" y="90" fill="#dfe6f0">all proven:</text><text x="690" y="105" fill="#8b95a7">anon OFF</text>
|
||||
<line x1="136" y1="93" x2="158" y2="93" stroke="#3fbaf5" marker-end="url(#b)"/>
|
||||
<line x1="291" y1="93" x2="313" y2="93" stroke="#3fbaf5" marker-end="url(#b)"/>
|
||||
<line x1="446" y1="93" x2="468" y2="93" stroke="#3fbaf5" marker-end="url(#b)"/>
|
||||
<line x1="601" y1="93" x2="623" y2="93" stroke="#3fbaf5" marker-end="url(#b)"/>
|
||||
<path d="M537 116 Q 537 165 460 165 L 420 165 Q 382 165 382 121" fill="none" stroke="#e879f9" stroke-dasharray="4 4" marker-end="url(#bm)"/>
|
||||
<text x="462" y="185" fill="#e879f9">401: back to the reboot rung</text></g>
|
||||
</svg>
|
||||
<p class="why">The decision this explains: the gate is per-node, never fleet-wide —
|
||||
one unproven node under a fleet-wide flip is an outage wearing a green tick.</p></div>
|
||||
|
||||
<div class="pane-k">Read-only pane · replay 23a · the crictl gate, two different nodes</div>
|
||||
<pre class="pane">node A (auth applied, rebooted):
|
||||
$ crictl pull zot.bztmon.org/canary/prompt-forge:0726R1
|
||||
<span class="good">Image is up to date for sha256:...</span> <span class="cm"><- authenticated pull; node A is trusted</span>
|
||||
node B (auth staged in config, no reboot yet):
|
||||
$ crictl pull zot.bztmon.org/canary/prompt-forge:0726R1
|
||||
<span class="bad">E... failed to pull ...: 401 Unauthorized</span>
|
||||
<span class="cm"><- node B reboots before it counts</span></pre>
|
||||
|
||||
<h3>Loop 4 — the mirror's host is also the mirror's customer</h3>
|
||||
<p>The node that hosts the mirror boots its own workloads through it — including,
|
||||
in this lab, the tunnel that serves the public website. So "restart the mirror's
|
||||
node" carries a blast radius far beyond the mirror. The runbook for that reboot
|
||||
lists every public-facing thing that rides on it, and the order they come back.</p>
|
||||
<p>Draw the dependency arrows for your own estate. The ones that surprise you are
|
||||
the ones that will page you. Sequencing, here, is a first-class engineering
|
||||
artefact.</p>
|
||||
|
||||
<div class="clause"><span class="k">THE HOMELAB CLAUSE</span><br>Production separates
|
||||
these concerns physically: the mirror on its own storage appliance, per-site edge
|
||||
nodes pulling FROM it, no workload sharing its host. The lab collapses them onto one
|
||||
node and manages the consequence with sequencing and runbooks. Same physics,
|
||||
different budget.</div>
|
||||
|
||||
<h2><span class="n">03</span>The traps — each rule has a scar</h2>
|
||||
|
||||
<div class="scene" id="s23"><div class="inner" style="position:relative">
|
||||
<img src="__M23__" alt="A serene queue: a stuck intake pod with a magenta ring, three cyan cargo bricks waiting in a diagonal line, and a small healthy pod far away." width="1600" height="873" decoding="async">
|
||||
<div class="s23-pulse" id="pulse"></div>
|
||||
<div class="cap">Scene 23 · the jammed intake · motion: the only breathing still on the page</div>
|
||||
</div></div>
|
||||
|
||||
<div class="traps">
|
||||
<p><b>"The registry is up" is not "the mirror works."</b> The worst incident in this
|
||||
mirror's life: an in-flight upstream sync wedged, and from then on EVERY pull of ANY
|
||||
image from that upstream hung — while the registry's health endpoint returned 200
|
||||
the entire time. The log line, when found, was almost poetic:</p>
|
||||
</div>
|
||||
|
||||
<div class="pane-k">Read-only pane · replay 23b · the deadlock</div>
|
||||
<pre class="pane">"image already demanded, waiting on channel"
|
||||
$ kubectl -n zot rollout restart deploy zot <span class="cm"><- the fix, instantly effective</span>
|
||||
<span class="cm"># probe an actual manifest, not the health endpoint:</span>
|
||||
$ curl -sI https://zot.bztmon.org/v2/library/busybox/manifests/latest
|
||||
<span class="good">HTTP/2 200</span> <span class="cm"><- THIS is "the mirror works"</span></pre>
|
||||
|
||||
<div class="traps">
|
||||
<p>Monitor the thing you actually need — a manifest fetch — not the process's
|
||||
opinion of itself.</p>
|
||||
<p><b>Digests can change in transit.</b> A mirror that converts image formats on sync
|
||||
breaks every digest pin and signature downstream of it — silently. The pairing
|
||||
that prevents it (<code style="font-family:var(--mono)">preserveDigest</code> +
|
||||
<code style="font-family:var(--mono)">http.compat</code>) is exactly the pairing from
|
||||
the chapter-02 crashloop. One decision, two scars.</p>
|
||||
<p><b>The cache grows until the disk ends.</b> onDemand means every image anyone ever
|
||||
pulls joins the shelf, and this registry does not evict least-recently-used blobs on
|
||||
its own — garbage collection reclaims orphaned data, not old-but-valid images.
|
||||
The receipt from the lab's live config: the storage section carries gc settings and
|
||||
NO retention section at all. That is a real answer — the current policy is "keep
|
||||
everything", chosen implicitly, and the 50-gigabyte volume has never yet been pushed
|
||||
to its high-water mark, so what happens there remains genuinely unobserved. A volume
|
||||
size is a budget, not a policy, and the two retention behaviours on this page —
|
||||
this one and the keep-rule inversion below — are ONE open policy decision: say
|
||||
explicitly what is kept, what is deletable, and what happens when the disk fills,
|
||||
before the disk answers for you.</p>
|
||||
<p><b>Retention flips its meaning on the first rule.</b> No retention policy: keep
|
||||
everything. ONE keep-rule on one repo: everything not matching is now DELETABLE.
|
||||
Adding a policy is not narrowing — it is inverting. Write explicit keep coverage
|
||||
per repo pattern, or write none — and read this trap together with the
|
||||
cache-growth one above; they are the two halves of the same unanswered question.</p>
|
||||
<p><b>Dedupe is not a toggle.</b> Flipping it on existing storage triggers a full
|
||||
relink pass under lock — pushes can hang half an hour behind it. Maintenance
|
||||
window, or leave it alone.</p>
|
||||
<p><b>A tag and a digest in the same reference used to miss the cache — and this
|
||||
trap carries its own moral about traps.</b> The historical mechanism: the runtime
|
||||
routes to mirrors by HOST, so the combined reference reached the mirror, which
|
||||
rejected it as unresolvable ("repository name not known", upstream issue zot 2584),
|
||||
and the runtime fell through to the origin — the fallback that saves chapter 02
|
||||
quietly defeating the cache. Re-tested against this lab's current registry on
|
||||
2026-08-24: the combined reference now pulls clean; the failure did not reproduce on
|
||||
v2.1.17. The rule survives in weaker form — prefer one reference form, know your
|
||||
version — but the stronger lesson is that a trap list is perishable. Re-run your
|
||||
own scars occasionally; some of them have healed.</p>
|
||||
<p><b>Credentials drift when the hash and the plaintext live apart.</b> The push
|
||||
credential is stored hashed; the drift surfaced on the first of July — plaintext
|
||||
lost, hash still standing — and with no way to prove or re-mint the pair, pushes
|
||||
stayed dead until the rebuilt rotation flow landed on the thirteenth. Twelve days.
|
||||
The receipt that it worked: three separate applications shipped to the fleet the
|
||||
SAME day the credential came back. Now hash and plaintext live side by side in the
|
||||
secret store, rotated together, by a human — the sync identity is deliberately
|
||||
read-only and CANNOT write secrets, so no automation can half-rotate them again.</p>
|
||||
<p><b>The cache is invisible in the UI.</b> The web interface shows pushed
|
||||
repositories only; on-demand cached images do not appear. Cosmetic — but the
|
||||
first time you look, you will conclude the mirror is empty. Verify with the API
|
||||
(<code style="font-family:var(--mono)">/v2/<name>/tags/list</code>), not the browser.</p>
|
||||
</div>
|
||||
|
||||
<h2><span class="n">04</span>The prod translation — same organ, grown up</h2>
|
||||
|
||||
<div class="scene" id="s24"><div class="inner">
|
||||
<img src="__M24__" alt="A monolithic armoured vault with a glowing wall of bricks visible through its parted door, attended by four small self-sufficient way-stations each holding its own bricks." width="1600" height="873" decoding="async">
|
||||
<div class="cap">Scene 24 · the vault and the way-stations · motion: two-plane parallax, calm authority</div>
|
||||
</div></div>
|
||||
|
||||
<p>Everything above is one lab, one node, one cache. Scale the pattern honestly and
|
||||
it becomes the supply chain of every disconnected or edge estate:</p>
|
||||
<ul>
|
||||
<li><b>The mirror lives on real storage</b> — an appliance with parity and
|
||||
snapshots, because at a disconnected site the mirror is not an optimisation, it is
|
||||
the only source of software that exists.</li>
|
||||
<li><b>Each remote site runs its OWN small node</b>, pulling from the central mirror
|
||||
on a schedule, so a site survives its WAN link dying with its full catalogue local.</li>
|
||||
<li><b>Authentication is day one</b>, not phase two — the lab's staged flip is
|
||||
what retrofitting looks like; greenfield does not get that luxury or that risk.</li>
|
||||
<li><b>Promotion is by digest, not by tag.</b> Production estates rewrite image
|
||||
references to pinned digests and promote by copying digests between registries —
|
||||
a tag is a suggestion, a digest is a fact. The lab's digest-pinning discipline is
|
||||
the same muscle at one-tenth the ceremony.</li>
|
||||
</ul>
|
||||
|
||||
<div class="note"><span class="k">FIELD NOTE · LOOP 1 DOES NOT SURVIVE THE AIRGAP</span><br>
|
||||
The lab's prettiest trick — mirror listed, origin implicit, the loop that breaks
|
||||
itself — assumes an upstream EXISTS to fall through to. At a genuinely
|
||||
disconnected site there is no second endpoint; if the mirror's own image is not
|
||||
already on the host, nothing will ever fetch it. Production answers this the
|
||||
unglamorous way: the mirror's image is pre-seeded into the host's local image store
|
||||
at provisioning time (or run as a static workload imported straight from disk), or
|
||||
Loop 2's recovery-box pattern is applied to the mirror itself — its image kept
|
||||
OFF the platform it serves. Loop 1's implicit fallback is a connected-world luxury;
|
||||
the airgap makes you choose Loop 2's discipline whether you like it or not.</div>
|
||||
|
||||
<div class="diagram"><span class="k">Diagram D3 · hub and spokes, WAN cut</span>
|
||||
<svg viewBox="0 0 760 230" role="img" aria-label="A central vault serving three sites; one site's WAN link is severed and that site still serves locally from its own shelf.">
|
||||
<defs><marker id="c" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0 0L10 5L0 10z" fill="#3fbaf5"/></marker></defs>
|
||||
<g font-family="IBM Plex Mono" font-size="11" text-anchor="middle">
|
||||
<rect x="320" y="20" width="120" height="60" rx="8" fill="none" stroke="#e879f9"/><text x="380" y="46" fill="#dfe6f0">central</text><text x="380" y="62" fill="#dfe6f0">vault</text>
|
||||
<rect x="60" y="150" width="130" height="52" rx="8" fill="none" stroke="#3fbaf5"/><text x="125" y="172" fill="#dfe6f0">site A node</text><text x="125" y="188" fill="#8b95a7">own shelf</text>
|
||||
<rect x="315" y="150" width="130" height="52" rx="8" fill="none" stroke="#3fbaf5"/><text x="380" y="172" fill="#dfe6f0">site B node</text><text x="380" y="188" fill="#8b95a7">own shelf</text>
|
||||
<rect x="570" y="150" width="130" height="52" rx="8" fill="none" stroke="#3fbaf5"/><text x="635" y="172" fill="#dfe6f0">site C node</text><text x="635" y="188" fill="#8b95a7">own shelf</text>
|
||||
<line x1="340" y1="80" x2="150" y2="145" stroke="#3fbaf5" stroke-dasharray="5 4" marker-end="url(#c)"/>
|
||||
<line x1="380" y1="80" x2="380" y2="145" stroke="#3fbaf5" stroke-dasharray="5 4" marker-end="url(#c)"/>
|
||||
<line x1="420" y1="80" x2="600" y2="140" stroke="#e879f9" stroke-dasharray="2 6"/>
|
||||
<text x="540" y="100" fill="#e879f9" font-size="14">✗</text><text x="560" y="118" fill="#e879f9">WAN cut</text>
|
||||
<text x="635" y="222" fill="#7fe0a7">still serving, local shelf</text>
|
||||
<text x="230" y="120" fill="#8b95a7">scheduled sync</text></g>
|
||||
</svg>
|
||||
<p class="why">The decision this explains: per-site nodes exist so a cut link idles
|
||||
NOTHING — pulls never cross the WAN at pull time at all.</p></div>
|
||||
|
||||
<div class="clause"><span class="k">THE HOMELAB CLAUSE, INVERTED</span><br>The lab is
|
||||
not a toy version of production. It is production with one of everything. The loops
|
||||
in chapter 02 exist at every scale; the only thing that changes is how expensive
|
||||
they are to ignore.</div>
|
||||
|
||||
<h2><span class="n">05</span>Close — the canary</h2>
|
||||
|
||||
<div class="scene" id="s25"><div class="inner">
|
||||
<img src="__M25__" alt="A wide flat node slab in the lower third of an otherwise empty void; a single luminous brick hovers midway in its descent toward the slab; a small pod watches from the edge." width="1600" height="873" decoding="async">
|
||||
<div class="cap">Scene 25 · the canary · motion: none. Deliberately.</div>
|
||||
</div></div>
|
||||
|
||||
<p>The mirror's proof-of-life is not a dashboard. When a node powers on, the fleet's
|
||||
gate makes it pull one known, first-party, mirror-only image through its own runtime
|
||||
— the same path a real workload would take, auth and all. Pass, and the node is a
|
||||
member of the fleet. Fail, and it is a machine that happens to be switched on.</p>
|
||||
|
||||
<div class="sayit"><span class="k">SAY IT</span>A node is not "up" when it pings. A
|
||||
node is up when it can feed itself.</div>
|
||||
|
||||
<p>That is the whole dossier in one sentence. A platform is a chain of machines
|
||||
feeding each other bytes, and the mirror is the pantry. Stock it deliberately, prove
|
||||
it constantly, and never let the recovery crew depend on it for lunch.</p>
|
||||
|
||||
<footer>
|
||||
DOSSIER / 005 · rev 3 · source of truth: the running lab · one open
|
||||
verification: negative-test the canary repo before Phase B leans on it. Motion is
|
||||
plate-level v1; per-part sliced animation arrives in a later revision.
|
||||
<a href="/">Back to the index</a>
|
||||
</footer>
|
||||
|
||||
</div>
|
||||
|
||||
<script src="__MIRROR_JS__"></script>
|
||||
Reference in New Issue
Block a user