Dan Gardner is the New York Times best-selling author of Risk, Future Babble, Superforecasting (co-authored with Philip E. Tetlock), and How Big Things Get Done (co-authored with Bent Flyvbjerg). His books have been published in 26 countries and 20 languages. Prior to becoming an author, Gardner was an award-winning investigative journalist. More >

Sometimes A Plausible Hypothesis Is Just A Plausible Hypothesis

Shortly after the first reports of terror attacks in Norway, pundits and security experts confidently blamed Islamists. They were wrong. And in the days following the arrest of Anders Behring Breivik, they were excoriated for having let Islamophobia get the better of them.

That reaction was misguided.

Granted, some commentators were indecently eager to pin another crime on Muslims, but, prior to Breivik's arrest, it was not unreasonable to suspect the atrocities in Norway were committed by Islamists. Indeed, it would have been unreasonable not to. The mistake made by pundits and security experts was something quite different and it's unfortunate it hasn't been properly identified because it's common - and this incident could be a salutary warning for all of us.

Think back to Friday, July 22. A bomb had exploded among the government office buildings of Oslo. Early indications suggested it was a car bomb. And it was big, having torn the facades off buildings up and down the street. Then came reports of shootings on an island where the ruling Labour party has a summer camp. There was no other information.

So what could we make of it?

In a situation like this, the first thing to consider is the "base rate," which is simply the statistical probability of something being true. If 65 per cent of professional baseball players are right-handed, the base rate of right- handed professional baseball players is 65 per cent. If you are asked whether a particular player is right- or left-handed, and you know nothing else about the player, it is correct to say that the player is probably right-handed.

So what was the base rate for Islamist attacks? As I reported some time ago, the popular perception that most terrorism in Europe is carried out by Islamists is very wrong. In fact, European Union data show that Islamists are responsible for only a tiny minority of European terrorism.

That would suggest that the relevant base rate was tiny. And that it would be ridiculous to suspect Islamists.

But that's not the best base rate to use.

Terrorism comes in many forms, including vandalism, threats, kidnappings, and assassinations. Attacks in which the intention is simply to kill as many civilian bystanders as possible are actually a fairly recent innovation and those most responsible for this new and more deadly form of terrorism are Islamists. Of course Islamists are far from alone in committing mass-atrocity terrorism. But they are predominant.

That's the proper base-rate to consider. Conclusion: Prior to Breivik's arrest, based on what was then known, it was correct to say that Islamists were probably responsible. In a phrase, it was a plausible hypothesis.

Where the pundits screwed up was in going way beyond that basic reasoning.

Having established a plausible hypothesis, they piled up the facts that fit. Norway is involved in Afghanistan. Norway has a poorly integrated Muslim community. An Islamist preacher had been charged with incitement the week before.

What they did not do is acknowledge that "probable" is a long way from "certain," that there were other plausible hypotheses, and other facts to support them. As a result, only one plausible hypothesis was considered and only the facts that supported it were discussed.

Inevitably, that plausible hypothesis ceased to looked like a reasonable guess in a highly uncertain situation. No, it was a slam dunk.

Unfortunately, the slide from "plausible hypothesis" to "slam dunk" is short and quick and people are far too eager to take the trip. No matter how many times this causes serious mistakes. No matter how obviously irrational it is. Our psychology urges us to be far more confident of our conclusions than we should be.

In a famous essay, renowned sociologist Paul Lazarsfeld wrote that a survey of American servicemen returning from the Second World War had found that men from rural backgrounds had generally been in better spirits while in the military than men from urban backgrounds. Lazarsfeld then imagined how the reader would respond. "That makes perfect sense," the reader thinks. Men from rural backgrounds were used to tougher living conditions and physical labour than city men. It's obvious, isn't it?

But then Lazarsfeld reveals that the actual finding was the reverse. So what was obviously true wasn't true at all. No matter, says the reader. That conclusion, too, is easily explained: "City men are more used to working in crowded conditions and in corporations, with chains of command, strict standards of clothing and social etiquette, and so on. It's obvious!"

No matter what the facts, people effortlessly create hypotheses that seem plausible to them. But we don't treat them as mere hypotheses. To do that, we have to remain skeptical and it's very hard to doubt an explanation generated by our own brains.

No, our hypotheses aren't mere hypotheses. They're obviously true. They are slam dunks.

As former CIA director George Tenet memorably demonstrated in the case of Iraq's WMDs, with the assistance of George W. Bush, there are no slam dunks in this uncertain world. And thinking otherwise can get us into a lot of trouble.

----

In my column last Friday, I mistook Ontario legislation making Aug. 1 Emancipation Day for legislation making the August holiday Emancipation Day. (Incidentally, the August holiday isn't actually a holiday by the strictest definition, but one confusing point at a time ...) So the Ontario legislation is actually a nice, but obscure, gesture. And Ontario's politicians apparently prefer having a Generic Holiday instead of a holiday commemorating a magnificent moment in the history of Canada and the world. Why? I have no idea. Please ask them.