Dan Gardner is the New York Times best-selling author of Risk, Future Babble, Superforecasting (co-authored with Philip E. Tetlock), and How Big Things Get Done (co-authored with Bent Flyvbjerg). His books have been published in 26 countries and 20 languages. Prior to becoming an author, Gardner was an award-winning investigative journalist. More >

The Three Rules of Crime Statistics

As a long-time student of crime policy, I didn't predict that national crime statistics released last week would show a substantial drop in most categories of crime in most parts of the country. Predicting long-term crime trends is hard. Predicting year-to-year variations is pretty much impossible. But when the news broke, I did predict the reactions. That's dead easy. Follow three basic rules and you can't go wrong. • Rule No. 1: Responsibility for crime trends depends entirely on whether those trends are good or bad. When national crime stats decline, everyone rushes to take credit. The mayor boasts his new initiative is working exactly as he said it would. The police chief proudly declares that the strategy he implemented is a great success. Social service agencies insist their new programs are responsible. And so it goes in city after city. The only thing that varies is the identity of the initiatives, strategies and programs said to be the cause of the crime drop. In one city, they're A, B and C. In another, it's D, E and F. And so on. Which suggests pretty strongly that all these claims are empty. Or at least it would suggest that if anyone noticed how blatantly contradictory these claims are, which they don't. My favourite reaction to last week's news was the line worked up by some spin monkey in the office of Ottawa Mayor Larry O'Brien: "New chief, new mayor, new approach. It works." Of course, this is utter nonsense -- not only did crime decline in cities across the country, the decline in Ottawa was actually less than the national average -- but Mr. O'Brien is a politician and in politics neither logic nor modesty is a virtue. Naturally, a quite different analysis applies when crime statistics go up. Rising crime does not mean the initiatives, strategies, and programs which would have been responsible for a decline in crime didn't work. Not at all. It means that crime is driven by deep social trends that are far beyond the very limited control of mayors, police chiefs and social service agencies. Only a simpleton would blame local officials for national trends. Credit them, yes. But blame them? Ridiculous. As John F. Kennedy said, "victory has a thousand fathers but defeat is an orphan." • Rule No. 2: The reaction of the justice system's critics to crime statistics depends entirely on the direction in which those statistics are headed. If statistics show crime is rising, the statistics are a perfectly accurate reflection of the frightening reality. If they suggest crime is falling, they are so transparently flawed that only fools, Liberals and criminologists would believe them. This rule explains Prime Minister Stephen Harper's complicated relationship with crime data. When serious violent crime rose a couple of years ago, Mr. Harper, then the opposition leader, waved the statistics about like a bloody shirt. Homicide is up 12 per cent! Guns, gangs, and drugs are out of control! But then the stats turned around and Mr. Harper started warning people not to be bamboozled by numbers. "Some try to pacify Canadians with statistics," he scoffed in a speech earlier this year. "Your personal experiences and impressions are wrong, they say; crime is not really a problem." Mr. Harper made the same case a month ago. Numbers don't matter. Just listen to the tragic stories of the victims. "These men, women, and children are not statistics," he declared. I didn't actually hear Mr. Harper give this speech but I imagine his delivery left the word "statistics" dripping with scorn. Living in his self-imposed bubble, Mr. Harper is never pressed to explain why the statistics are valid and important if they point in a politically useful direction but misleading and useless if they don't. Other critics of the justice system have been a little more refined: People don't believe the justice system can actually accomplish anything, they say, so victims don't bother reporting crimes to the police. Thus, if police-reported crime rises, you can be sure the situation is getting worse. But even if police-reported crime falls, crime is actually getting worse. There's something to this. Victim surveys do find that a large proportion of crimes are not reported to the police. But what the critics don't mention is that when people are asked why they don't report crimes, the most common answer is "didn't think it was important." Not surprisingly, the unreported stuff tends to be very minor. Critics also ignore the fact that while non-reporting rates are, indeed, rising for several crimes, they are flat for others, and falling for assault. Which provides little support for the hell-in-a-handbasket hypothesis. Finally, critics faced with positive crime statistics sometimes fall back on what I call the Homer Simpson defence: Scoff and say, as Homer did, "Facts? You can prove anything you want with facts." "The wonderful thing about statistics is that they can prove anything," writes conservative columnist Claire Hoy. Hoy was incensed by a banner headline in the Toronto Star -- based on data in last week's StatsCan release -- which read, "We're Canada's Safest City." "Mind you," rejoined Hoy, "one in five homicides in Canada occur in Toronto but hey, when you count murders as a percentage of the overall population instead of counting them as dead bodies, it's easy to brag about how 'safe' the city is." Mind you, by Hoy's logic, a village with a population of 50 in which one person goes on a rampage and kills the other 49 is safer than Toronto because the hamlet's body count is lower. I suspect the population of the village would dispute that interpretation. Or they would if they weren't all dead. Which just goes to show that while statistics cannot prove anything, they can be used to dress up a wide variety of incredibly dumb statements. • Rule No. 3: Following a major, violent incident, discussion of negative crime data increases; following a major, violent incident, discussion of positive crime data ends. Toronto enjoyed its good news for a couple of days. Then three gang members were murdered. The statistics vanished. The image of a city under siege returned. "Given the number of bullets flying around the streets of Toronto," a letter writer in Wednesday's Globe and Mail quipped, "I'd think the city's bureaucrats would be more concerned with preventing lead poisoning than skin cancer." Responding to the release of the StatsCan numbers, Toronto police chief Bill Blair urged people to pay attention and let the data inform their sense of reality. "Unfortunately," he noted, "people's perceptions are often created around a single incident or a series of incidents over a short period of time." It was the smartest thing anyone said all week.