Blameless, and still scored: where the line actually has to sit

Time to say the uncomfortable sentence plainly, because the rest of this arc only earns your trust if it doesn't dodge it: a system that watches what people say, checks it against what actually happened, and keeps a running tally per person is one short step away from a surveillance system that scores humans. Anyone building this who tells you otherwise, breezily, without sitting in that discomfort for a while, is not being straight with you. This post is about exactly where the line sits, and why crossing it would be a completely different, and much worse, product.

The postmortem that got this right first

Software already ran this exact argument once, in a different room, and it's worth understanding how it landed before applying the lesson here. John Allspaw, then CTO of Etsy, wrote a 2012 post on the company's engineering blog — "Blameless PostMortems and a Just Culture" — that became one of the most cited pieces of writing in operations engineering, and its central move is the one this whole post depends on: separate the account of what happened and why from any decision about consequences for the person involved, because the moment those two things are fused into one conversation, people stop giving honest accounts. Nobody volunteers the full, unflattering truth about a mistake to the same room that's about to decide their performance review based on it. Fuse investigation and judgment and you don't get more accountability — you get less information, at exactly the moment you need more of it.

Sidney Dekker's Field Guide to Understanding Human Error gives the deeper theoretical version of the same move. Dekker argues for what he calls the "new view" of human error: an action that looks obviously wrong in hindsight almost always made local sense to the person taking it, given what they knew, what they were juggling, and what the system around them made easy or hard to do at that moment. The investigator's job, in this view, is not to find the point where someone should have known better — that's the "old view," and it stops the investigation exactly where it should have started. The job is to reconstruct why the action made sense from inside the situation, because that reconstruction is what actually prevents the next incident. Assigning blame explains nothing about the system that produced the failure; it just closes the file.

So does this violate that? Here's the actual argument that it doesn't.

The honest answer requires being specific about what's being scored, not asserting a vibe.

1. The subject of the score is the claim, never the character. What gets scored is whether a specific, quoted, falsifiable sentence about the world turned out to match what the world did. It does not, and structurally cannot, score whether a person is careless, talented, or trying hard — those are judgments about a human, and nothing in a resolved-claim ledger contains them. "You said X; X did not happen" and "you are bad at your job" are different speech acts, and collapsing them is a failure of whoever reads the report, not something the measurement can prevent by itself — which is exactly why it has to live inside a blameless culture, not instead of one.
2. A track record, never a single strike. Calibration is a property of many resolved claims, weighted by consequence, accumulated over time — the same design Brier's scoring rule and the Good Judgment Project's tournament both depend on (Part 21 of this arc). No meteorologist gets fired for one missed forecast, and no engineer's calibration is a verdict computed from one wrong guess. A pattern across a real number of claims is information. A single miss is noise, and treating it otherwise breaks the same statistical rule Part 22 spent its entire length on.
3. It answers "did this match," never "why did you get it wrong." This is the actual boundary with Dekker's investigation, and it's worth being precise rather than hand-waving past it: reconstructing the local rationality behind a wrong call — what the person knew, what the system made visible, what pressure they were under — is genuinely valuable work, and it is not this measurement's job. It stops at the resolved fact. Understanding why a resolved claim was wrong belongs to a blameless postmortem process, run the way Allspaw and Dekker describe, by humans, with all the context a ledger entry cannot hold.
4. The unresolvable bucket has to stay honest, or the whole thing curdles. Part 26 made the statistical case for never quietly converting "we don't know" into "wrong." Here's the ethical case: a system that punishes ambiguity the same way it punishes a clear miss teaches people to stop making checkable claims — the exact opposite of what blameless, evidence-based engineering culture is trying to build. Protecting the unresolvable count isn't a technicality. It's the difference between a scoreboard and a trap.

A postmortem asks what the system produced. A calibration score asks what a specific person's specific sentence predicted. Both are legitimate. Only one of them needs a name attached — and that's exactly the part that has to be handled with the most care, not the least.

The bright lines, stated plainly, because good intentions aren't a policy

It's easy for a piece like this to assert good intentions and stop there. That's not good enough for something this close to the line, so here is what would actually mean it had crossed it, stated as concretely as possible: calibration used, alone, to make a hiring, firing, or compensation decision, without the full human context a postmortem provides. Calibration shown to the room rather than to the individual it describes, turning a private instrument into a public ranking. A single bad stretch overriding a real track record. And — the one this arc keeps returning to because it's the easiest to get quietly wrong — the unresolvable count getting folded into "did not," even once, even under pressure to make a slide cleaner. Any of these turns a measurement of claims into a measurement of people, which is precisely the thing this whole approach exists to not be. Google's own SRE postmortem guidance draws a nearly identical line in a different context: a postmortem is a learning document, not a performance review, and the moment it's read as the second thing, people stop writing the honest version of the first.

CodeNSM's own design commitment follows directly from this: calibration is always shown with its denominator, always to the person it's about, and never used to rank anyone by how much they said in a room — the last of which matters more than it sounds, because the next part of this arc is about exactly who that rule protects.

Blameless doesn't mean consequence-free

It means the consequence lands on the system that produced the outcome, not on whoever was honest enough to describe it out loud. Keeping that promise isn't a policy you set once. It's a question that has to get asked again every time this gets used somewhere new: has the score quietly become a leash? The honest answer to that question is the actual product of this post, more than any single claim in it — and it's a question with no permanent answer, only a permanent obligation to keep asking it.

It's worth naming, plainly, who this whole design protects, because "blameless" can sound abstract until you picture the actual person it's for. It's the engineer who made a wrong call about a payment-path function, in good faith, with the information available at the time, and who needs the organization's response to that wrong call to be a better runbook and a clearer signal next time — not a quiet mark against their name that resurfaces at review season with no context attached. Allspaw's argument and Dekker's argument were never really about being nice. They were about which response actually produces a safer system next quarter, and the research on both sides of that question has been remarkably consistent for over a decade: punishment gets you silence, and silence is the one input this entire approach cannot function without.

If you take one thing from this arc's most uncomfortable part, take this: the safest position on the blameless-versus-scored question is not confidence that the line has been drawn correctly. It's discomfort, kept alive on purpose, checked against the bright lines above on a recurring basis rather than settled once in a design document and filed away. A team that stops being uneasy about this is a team that has stopped watching the exact place where the whole approach could quietly become the thing it was built to replace.