I’m planning to look at this study in more depth over the next couple of weeks. On first reading, it appears to be a work of tenuous premise-begging hackery; however, it would seem fair to look at the stats in more detail before dismissal.
If anyone reading would care to do likewise (especially if they’ve got a background in academic statistics rather than just business analysis) and share their conclusions, I’d be extremely grateful.
Okay, well, I’m a mathematician with not much background in stats, but here’s a shot. One can get data from Bureau of Justice Statistics which doesn’t seem to cover as many years as the study uses, but does have the advantage of being a working website…
One thing which caught my eye was the following from pages 2-3:
For example, Rhode Island suspended the death penalty in 1972, reinstated it in 1977, and abolished it in 1984. Although many factors that affect Rhode Island’s murder rate differ in 1972 and 1984, the murder rate increased after both suspensions: by 13 percent after 1972 and by 25 percent after 1984 in one-year comparisons. The 1984 increase even reversed a declining trend in Rhode Island’s murders.
The numbers around 1984 are as follows (absolute number of murder victims):
1981 40
1982 35
1983 26
1984 33
1985 35
1986 34
There *was* a short-term trend downwards, but the death-penalty was re-instated in 1977, after which there was an *increase* in the murder-rate (indeed, 1976 has the lowest rate in the period my data covers). Over the period 1973-2002, the average yearly change (absolute change, so not taking account of positive or negative) was 16.3, with a standard derivation of 14.7. A 25% yearly change would not seem to be terribly significant.
It bothers me that the authors single this data out as supporting their claim, when to my mind, it does not support the claim at all!
I cannot make a better critique (come on Chris Lightfoot!!!!) but a starting place might be to see which of the following the authors have followed:
i) Looked at a change in the law in year N and then the change in homocide rate in year N and N+1. Conclude: the distribution across states and time is that the change is positive after a law stopping executions.
ii) Looked at the change in homocide rate between every year, and tried to see if there was any corollation between a change in the law and a change in the rate. Conclude: there is a much higher than expected change in the murder rate after a law change.
My belief is that these are very different problems: I don’t think one can conclude much from statement (i) without looking at statement (ii). I think the authors looked at (i) without looking at (ii). But I’d be happy for someone to suggest I’m wrong in this.
Cheers, –Matt
Argh, sorry, another thought. One has to work out how to model murder-rate data. I would say that it’s obviously a random variable with long-term trends (caused by trends in social circumstances etc. etc.) The authors claim as much by:
We compare the murder rate for each state immediately before and after it suspended or reinstated the death penalty. Such comparison provides the basis for a strong inference for several reasons. First, many factors that affect crime—e.g., law enforcement, judicial, demographic, and economic variables—change only slightly over a short period of time. Therefore, quick changes in a state’s murder rate following a change in its death penalty law can be attributed to the legal change.
My emphasis. Now, I think one could call this whole study into question by showing that the murder rate always shows a large year-on-year change, which would seem to directly contradict the above claim. Certainly the data I gave above for Rhode Island suggests this.
The point is that, in order to say that a change in the law caused something, you need to remove the other variables. By looking at short-term changes, this is done (probably). However, you haven’t removed the inherent randomness of the data, and if the data shows a large amount of short-term randomness, one really should address this. I don’t think this study does.
–Matt
Co-author Hasem Dezhbakhsh is a right guy as far as I am aware. He wrote a really quite good paper on "more guns, less crime" that got published in the AER, and (I see from SSRN) a paper in Applied Economics Letters that I read last year and was very impressed by. If the Daniel Levy who he wrote a lot of his early stuff with is the same Daniel Levy who wrote "How the Dismal Science Got Its Name", then Dezhbakhsh is quite possibly a rightwinger of some sort, but he’s not a hack.