Every month, an agency sends a client a maintenance report. Updates applied. Security scan clean. Backups verified. The client skims it, sees green, and files it.
Here is a failure mode nobody writes about: every line of that report can be true, and none of it about the machine the client is paying to have maintained.
The mechanism is boring, extremely common, and structurally invisible to the exact system built to catch it.
How it happens
Automation needs to know where things live. So there is a map — a config file, a service registry, a spreadsheet of account IDs — that translates a name into a location. The site clientsite.com lives on the install named clientsite. Routine stuff.
Now suppose a row in that map is wrong. A name points at a location with a very similar identifier. Nothing crashes. Nothing errors. The automation reads the map, connects where it is told, does its job correctly, and reports success.
Updates applied: true. Security scan: clean. Responding: 200.
All true. All about the wrong machine. Meanwhile the intended one sits untouched, accumulating exactly the updates the report says were applied.
Why the safety net does not fire
Most maintenance systems have a check for this class of problem. Every asset carries a completion timestamp, and anything that goes too long without a pass gets flagged as overdue.
That check cannot fire here. The pass did run. It ran against the wrong machine and wrote a real timestamp to the right record. The overdue detector sees a healthy, recently-serviced asset and correctly says nothing.
This is the part worth sitting with:
A false completion record is worse than a missing one. A missing record triggers the alarm. A false record silences it — and a silenced alarm is precisely the state the alarm exists to detect.
An empty field is a question. A wrong field is an answer, and systems stop asking questions once they have an answer.
Why monitoring will not find it
It will not be found by watching. It is found by counting.
The check is trivial: ask each live system what it thinks its own address is, and compare that against what the map claims. Under a minute per asset.
The reason it is almost never run is not difficulty. It is that the map is configuration, and nobody audits configuration. Configuration is the thing you check with, not the thing you check. It is the reference frame, and a reference frame is invisible from inside itself.
The general lesson, which is not about websites
In aerospace software under DO-178C, the discipline is not "test the system." It is prove the thing you tested is the thing that ships. Most of that standard's weight sits on traceability — establishing that the artifact you verified and the artifact in the field are the same artifact.
Commercial software almost never does this. We verify behaviour and assume identity. We confirm the update applied; we do not confirm what it applied to.
Every automated system has a set of inputs it treats as ground truth: a config file, an environment variable, a service registry, a spreadsheet of account IDs. Those inputs are unverified by definition — they are what verification is measured against.
What to actually do
Three questions, worth asking of any automation you rely on:
1. What does this system treat as ground truth, and when was that last verified against reality? Not "is the file correct" — is the thing the file describes the thing that exists. 2. Can a success record be written for work done somewhere else? If yes, your completion timestamps are not evidence of service. 3. Does any check compare identity, or do they all compare behaviour? Behaviour checks confirm something worked. Only an identity check confirms what it worked on.
The answer to (1) is usually "never." That is not negligence — it is what happens when a thing is load-bearing and looks like plumbing.
If you depend on automation and could not say today what it treats as ground truth, request a systems assessment. We check what your systems are pointed at before we talk about what they do.
Brent Dorsey is the founder of Geaux Digital Media, which architects, integrates, automates and builds the systems growing companies run on. Louisiana-based, working with businesses across the United States. Get a Systems Assessment →