Is StatsCan Reporting Crime Rates Accurately?

There's more to Statistics Canada's method of reporting crime statistics than a recently published study would have us believe.

A recent study by the Macdonald-Laurier Institute for Public Policy (MLI) – which I have read from start to finish – criticizes Statistics Canada's method of reporting crime statistics, suggesting that the numbers just “don’t add up.” It is clear that the MLI has close connections with the current federal government, whose ideological views this study seems to reinforce. I don't want to make too much of that, because it’s hard to argue against some of the MLI’s more generic methodological recommendations: StatsCan should report on crime more fully and openly. The truth is, everybody that produces statistical information should do so more fully and openly. But life is tough, times are hard, and money is short.

In their recent article “Can StatsCan Crime Numbers Be Trusted?” the Globe and Mail reports one of MLI’s conclusions as follows: “the annual report on crime statistics, known as Juristat, routinely revises crime statistics from previous years upward in any given year’s report, making annual crime decreases appear more significant than they are.”

The graph below shows the relevant time series from the Juristat report in question and displays the ebbs and flows in Canada's Crime Severity Index. (A chart showing the more traditional measure of crime by volume can be found here. There is no accompanying graph, but it would look similar to the one below.)

What the MLI analysts are saying is that when StatsCan reports the data point for 2009, it will be measured against a point on the graph representing 2008, which will have been upwardly revised. Thus, any decline that takes place year over year will be accented, and any increase muted. That's not surprising given the necessary process: an initial StatsCan crime report is issued, more data that was not available at the publication date comes in, and the report and accompanying graph are updated. It isn't as though there’s anything that could be done to get around this this, other than making the publication date so late that revisions are unnecessary.

But more importantly, the issue identified by the MLI really applies only to the last couple of data points on the x-axis of the graph above. StatsCan might revise its figures once or twice, but eventually they become pretty much set in stone. Thus, the revisions the MLI notes will not alter the overall shape of the graph. In other words, the revisions will not show a long-term downward trend if there is an upward trend in the underlying crime data. The downward trend in crime that StatsCan notes really is there.

There's one other point in the study I would like to touch on, because it seems representative of a number of the criticisms the MLI is making:

This is, as the MLI admits, a methodological complaint. StatsCan chops up and recombines its numbers based on a series of methodological decisions. But these are not the only decisions that could have been made, and indeed when you recut the numbers as per the MLI, you get Table 7 above. The question then becomes whether this, as the MLI asserts, is a "much better" way of chopping up and recombining the numbers, and whether the result is worthy of highlighting in the StatsCan report.

I will not defend the agency’s various methodological decisions, for they have already done so on their own behalf. I will posit, however, that there is much more to the debate than the MLI cares to admit.