Dan Gardner is the New York Times best-selling author of Risk, Future Babble, Superforecasting (co-authored with Philip E. Tetlock), and How Big Things Get Done (co-authored with Bent Flyvbjerg). His books have been published in 26 countries and 20 languages. Prior to becoming an author, Gardner was an award-winning investigative journalist. More >

Clear, Confident, And Wrong

Imagine it is some time ago. You turn on the television and see an expert talking about how the economy will perform in the coming year. "I know the first half of the year is going to be tough, there's no doubt about it," he says quickly and forcefully. "But by the second half of the year, all these interest rate cuts will work their way through the system. Obviously, I think the housing market will be a lot better than it's been. The auto market. Jobs. And I think inflation's going to be held in check." The stock market will heat up, too. The Dow will likely hit an all-time high. The expert wears an expensive suit, a gold tie, and a silk pocket square. He exudes confidence. He is a Master of the Universe. Now imagine it is Wednesday of this week and another expert is on television forecasting the future. His earlier forecast was off, the expert acknowledges. He now expects only modest economic growth. "In advanced economies, temporary factors supporting growth in 2010, such as the inventory cycle and pent-up demand, have largely run their course and fiscal stimulus will shift to fiscal consolidation over the projection horizon," he says. But this is all far from certain. "Important risks remain around this outlook," he says. Three complex factors may produce stronger economic growth than expected, he notes. But three other factors may push things in the opposite direction. The first expert is certain. The second is tentative. The first expert tells a simple and clear story with a crisp conclusion. The second says things are complex and uncertain and we can only be sure what will happen when it happens. The first expert is interesting and compelling. The second is boring. Who wants to listen to "on the one hand, on the other hand"? The first expert is Charles Payne of Wall Street Strategies. He made the forecast quoted above in December, 2007. So the "second half of the year" that he thought would be fabulous was the second half of 2008. Anyone remember what happened in the second half of 2008? Right. The second expert is Mark Carney, governor of the Bank of Canada. Obviously, I don't know how accurate Carney's latest forecast will prove to be. It may miss the mark entirely. They often do, especially when they're needed most. "The record of failure to predict recessions is nearly unblemished," noted an IMF researcher. Fundamentally, the world is unpredictable and nothing will ever eliminate the uncertainty in which we live. But we can, sometimes, reduce the uncertainty. A little. If we listen to the right people. In the late 1980s, University of California psychologist Philip Tetlock got almost 300 experts such as economists and political scientists to predict the sort of thing that is routinely predicted in the news -- inflation rates, economic growth, elections, wars, the fates of nations. In all, Tetlock gathered almost 28,000 predictions. Then he waited until the predictions could be judged. As I discuss in my new book, Future Babble, the results of this grand experiment couldn't have been clearer: The average expert was about as accurate as a dart-throwing chimpanzee. (Hence, the chimpanzee on the cover, in case you were wondering.) At this point, the cynic would scoff "what do experts know?" and walk away. But Tetlock dug deeper. Some experts actually did worse than average, meaning they would have been better off flipping a coin. But others did better. They still weren't great. The world really is unpredictable. But they showed genuine, if modest, predictive insight. What made the difference? Not whether the expert was left-wing or right-wing, optimist or pessimist, or any similar factor. No, it was the style of thinking that made all the difference. Experts who had One Big Idea, who valued analytical simplicity, who came to clear conclusions, and who were most confident -- these were the experts who could have been beaten by a chimpanzee. Let's call them Charles Paynes. But experts who gathered information from many sources, who were comfortable with complexity and uncertainty, and were more prepared to admit mistakes and adjust conclusions accordingly -- these were the experts worth listening to. They are the Mark Carneys. Tetlock found something else: the more famous an expert is, the less accurate he is. Why? Because the media, and the public, want to hear from the Charles Paynes, not the Mark Carneys (unless the Mark Carney in question happens to be the governor of the Bank of Canada, naturally). Just look at business TV. It's chock-a-block with confident windbags. But experts who talk about possibilities and probabilities? Who often use the word "maybe"? Who frankly admit errors and change their thinking accordingly? They are seldom seen. The next time one of those windbags appears on television, remember this: If they were as accurate as they are confident, they'd be billionaires. And billionaires don't do TV.