Thoughts··2 min
Git Is the Source of Truth; the Runtime Is a Cache
What changes when you treat the running system as a disposable projection of a versioned history.
There’s a mental shift that, once it clicks, changes how you operate everything: the running system is not the source of truth. Git is. The cluster, the warehouse, the environment — they’re caches. A projection of a versioned, reviewed history that a reconciliation loop keeps in sync.
Why it matters
When the runtime is authoritative, every fix is a one-off. Someone SSHes in, edits a file, restarts a service — and now reality has drifted from any record of intent. The next person can’t tell what’s deliberate and what’s an accident.
When Git is authoritative, the rules invert:
- Every change has an author, a timestamp, a diff, and a review. Change control is just how you already work.
- Drift is a signal, not a surprise. A controller comparing desired state to live state will tell you when something diverged — and, if you let it, put it back.
- Rollback is
git revert. You’re never reconstructing what the system “used to” look like; it’s in the history.
The discipline it demands
This only works if you’re honest about it. The moment you “just fix it in production” without a follow-up commit, you’ve reintroduced the thing you were trying to escape. The healthy pattern is: even genuinely imperative work — a one-off backfill, an incident remediation — gets captured back into the declarative source afterward, so the system converges to a fully described state.
It’s not only for compute
The same idea generalizes well beyond Kubernetes. Configuration, policy, pipelines, DNS records, even documentation — anything you can describe as desired state can be version-controlled, reviewed, and reconciled. The reward is a system you can reason about by reading, not by archaeology.
I keep coming back to one rule of thumb: if I find myself doing something twice through a UI, it’s time to write the code, commit it, and let the loop take over.