---
title: "The first secret is the one you can't commit"
date: 2026-06-28
summary: "Secrets management has a bootstrap paradox: the credential that pulls every other secret can't itself live in git. Here's how a homelab fleet breaks the cycle — zero plaintext secrets in any repo, and a clean rule for which is the one exception."
tags: ["security", "gitops", "secrets", "kubernetes", "external-secrets"]
draft: false
hero: "/blog/secret-zero.webp"
heroAlt: "A single glowing seed-shaped key at the base of a dark circuit-board tree whose traces branch upward into hundreds of faint nodes, all drawing their glow from that one source."

---

Every "we do GitOps properly" story has a chicken-and-egg problem hiding in the first paragraph, and most
posts quietly skip it.

The pitch is clean: secrets never live in git. Instead, the cluster runs the [External Secrets
Operator](https://external-secrets.io) (ESO), which reads from a real secret store — here, a self-hosted
[Infisical](https://infisical.com) — and materialises a Kubernetes `Secret` for each app. Your repo only ever
contains an `ExternalSecret` manifest: a **pointer** ("give `immich` the value at key `IMMICH_DB_PASSWORD`"),
never a value. Beautiful. Auditable. Diff-able.

Except — ESO has to authenticate to Infisical somehow. That's a credential. Where does *it* live?

## Secret zero

You can't store the bootstrap credential in git (that's the whole point), and you can't fetch it from the
secret store (you need it *to reach* the secret store). This is **secret zero**: the one credential that the
entire chain hangs off, that has to be injected from outside the GitOps loop.

So you treat it as exactly that — special, minimal, and out-of-band:

- It's a **machine identity** scoped to *read-only*, and scoped per cluster. cave's ESO can't read alfred's
  secrets and vice-versa. A leak is blast-radius-limited to one cluster's read path.
- It's applied **once, by hand** (or by a sealed bootstrap step), never committed. Everything downstream of it
  is declarative.
- After it lands, a single `ClusterSecretStore` object — which *is* in git, because it's just a pointer to the
  store plus a reference to secret zero — wires the whole cluster up.

From there the dam breaks in the good way: every app's `ExternalSecret` resolves through that store, ESO keeps
the `Secret` in sync, and your repo stays a map of *names*, not values.

<!-- DIAGRAM: secret-zero bootstrap chain — out-of-band identity → ClusterSecretStore → ExternalSecrets → app Secrets -->
![Secret-zero bootstrap chain](/diagrams/secret-zero.svg?v=1)

## The discipline around the one exception

One out-of-band credential is fine. The trap is letting it sprawl — or fumbling its rotation. Two rules earn
their keep:

**Per-cluster, least-privilege identities.** It's tempting to mint one powerful identity and reuse it
everywhere. Don't. A read-only identity per cluster means the bootstrap secret can *read* and nothing else, in
*one* place. The write path (provisioning new secrets) is a separate, rarely-used identity that doesn't sit on
every node.

**Never revoke a shared identity until every consumer has moved off it.** This sounds obvious and is the
single easiest way to take the fleet down. When you split a shared identity into per-cluster ones, the old one
stays valid until you've *verified* each cluster is happily authenticating on its own — then, and only then,
you revoke. Revoke-first-verify-later turns a tidy-up into an outage.

## What still belongs in git (encrypted)

A purist would stop here, but reality has a few things that genuinely need to live *in* the repo — a value a
bootstrap step reads before ESO is even running. For those, the answer isn't "commit it in plaintext and feel
bad," it's **SOPS + age**: the value is encrypted in git, decryptable only by a key that lives on the
operators' machines (and the cluster), never in the repo. Same principle as secret zero — the *decryption* key
is the out-of-band thing — applied to the handful of values that can't wait for the operator to spin up.

## The shape of it

The win isn't any one tool. It's the shape: **exactly one** credential lives outside GitOps, it's read-only
and per-cluster, and it's the seed the whole tree grows from. Everything else is a pointer you can show
your worst enemy. When someone asks "where are your secrets?", the honest answer is "in the store — the repo
just knows their *names*," and that one sentence is the whole security model.

*This runs across a five-cluster Talos homelab; the ESO + Infisical wiring, the per-cluster read-only
identities, and the revoke-order discipline are the load-bearing pieces.*