A process that looks alive is the cheapest lie in software. It has a name in the process list, it is holding a bit of memory, it has not crashed. None of that says it is doing its job. We have watched something sit in exactly that state for days: perfectly alive, doing nothing anybody wanted.
So we hold a plain rule. If you cannot see a system run, it is not in production. It is installed. Those are different words for a reason, and every system we hand over does three things from day one to earn the first one.
It says it is alive, and it says when
Not a light that is on. A line, written every cycle, carrying the time and the last thing it finished. "Looks alive" is not evidence. A timestamp from ninety seconds ago is.
The difference shows up the night something stops. With that line you know it stopped at 02:14 and what it had just completed. Without it you know it is broken now, and you get to guess how long it has been broken. That guess is usually wrong in the expensive direction.
The stronger version of the same habit is to prefer a mark the work leaves behind: a file whose modification time moved, a record that appeared, a probe that comes back changed. A system's account of itself is a claim. The mark is evidence.
It knows what it may not write down
A log that quietly records secrets is a data leak with a timestamp on it. We have had to strip credentials out of logs because the layer underneath stored them in plain text and printed them helpfully. That is not a rare accident. It is the default behaviour of software written to be useful.
So we decide up front which fields are safe to record, and record only those. A list of things to ban fails the first time somebody adds a field nobody thought to ban, and it fails silently. A list of things allowed fails closed: anything you did not clear simply never gets written.
It writes the change down before it announces it
Every state change that matters lands in a record first, and only then does the system say it happened. The order is the whole point. A system that announces first and records after will, on the day it dies mid-step, have told you about work that never landed.
"It is running" is not the same as "we are operating it." A system you cannot watch is not in production. It is running somewhere you cannot see.
Why all three on day one
None of this is expensive while you are building. It is a logging habit, a short list of safe fields, and a record written before the announcement. It is brutal to add afterwards, because by then you are trying to reconstruct a history you never kept. Two in the morning, with something down, is a poor time to discover the system never wrote anything at all.
It is also why we will not hand over a box that only works in the demo. Software that cannot show you what it is doing is asking for your trust on nothing but hope. Hope is a fine thing. It is a poor operations plan.
Under the hood
Structured events, not print statements: one machine-readable record per event carrying time, actor, action, entity and outcome, plus a correlation identifier so a single request can be followed across every service it touches. Free text is for reading one line at a time. Structured events are for querying a thousand at a time while you hunt the one that went wrong.
Redaction by allowlist, applied before anything reaches the sink, never by denylist, and never by logging whole request or response payloads on reflex.
State transitions written to an append-only trail, stamped at the source that made the change rather than inferred by whatever reads it later, and committed ahead of the side effect. Append-only means entries can be added but not quietly rewritten, so the history holds up in an incident review.
One more thing that gets missed: silence has to be alarmable. If the liveness line stops arriving, that absence is itself an event, and something has to be watching for it. A monitor that only reacts to what it receives will never react to a system that has stopped speaking.