Poker players have a word for the mistake. Annie Duke, who played professionally before writing about decisions, calls it "resulting": treating the outcome as the only test of whether a decision was good. A sound decision can fail, and a careless one can get lucky. If you cannot tell the two apart, you will learn the wrong things from both.1
Most organisations cannot tell them apart, and they do not know it, because the question is never asked in the meeting where things go wrong.
The effect is old and robust
The finding has a name in psychology: outcome bias. In 1988 Jonathan Baron and John Hershey published a set of five studies in which people read about decisions made under uncertainty, such as a medical choice or a monetary gamble, together with how they turned out. The readers were told the decision-maker had no more information than they did. Even so, when the outcome was good they rated the thinking behind the decision more favourably, saw the decision-maker as more competent, and were more willing to hand over the next decision.2
The first of those experiments was small, with 20 participants. It is worth knowing that it has since been replicated with 692, and the bias appeared again, including among people who had said that outcomes should not matter when judging a decision.3 Knowing about the bias is not the same as being immune to it.
Why a record of the reasoning matters
There is a second reason to be careful with outcomes. Even with identical information, people reach different judgements. Kahneman, Sibony and Sunstein call this noise. In their account of a study at an insurance company, executives expected two randomly chosen underwriters to differ by about 10% on the same case. Measured, the difference was roughly 50%.4
Put the two findings together. Judgement varies far more than we expect, and we then grade it on results that include luck. A single result is a poor guide to whether the thinking was good. What would be a good guide is a record of the thinking, made at the time.
What an organisation teaches when it grades dice
This is our reading, not a measured result: if people are judged only on how things turn out, the rational response is to avoid any decision whose outcome they cannot control. Wait. Escalate. Convene a committee. Write the email that protects you. An organisation that punishes bad outcomes does not get better decisions. It gets fewer decisions, made later, by people protecting themselves.
Here is a test you can run tonight. Think of one decision that was well reasoned and turned out badly, and one that was careless and got lucky. What happened to each person? If the careful one was punished and the lucky one promoted, the organisation is not judging decisions. It is judging dice.
The smallest useful record
Before a significant decision, write down three things. The frame: what question are we actually answering? What we know now. What we do not know. Date it and name who decided. That is one page.
The record does two jobs. It protects the careful decision-maker whose call went the wrong way for reasons nobody could have seen. And it exposes the careless one, even when they were lucky, which is the harder thing and the more important one. Over time it teaches an organisation that what counts is the quality of the thinking.
There is a darker version that deserves its name. When something goes wrong, blame is sometimes placed where it is easiest rather than where it belongs: the forum that judges the event is chosen, a standard of evidence is applied to one person and not another, and information that arrived later is treated as if it had been available earlier. We call that manufactured culpability. A dated record of what was known is the plainest protection against it.