We compare the murder rate for each state immediately before and after it suspended or reinstated the death penalty. Such comparison provides the basis for a strong inference for several reasons. First, many factors that affect crime—e.g., law enforcement, judicial, demographic, and economic variables—change only slightly over a short period of time. Therefore, quick changes in a state’s murder rate following a change in its death penalty law can be attributed to the legal change.

My emphasis. Now, I think one could call this whole study into question by showing that the murder rate always shows a large year-on-year change, which would seem to directly contradict the above claim. Certainly the data I gave above for Rhode Island suggests this.

The point is that, in order to say that a change in the law caused something, you need to remove the other variables. By looking at short-term changes, this is done (probably). However, you haven’t removed the inherent randomness of the data, and if the data shows a large amount of short-term randomness, one really should address this. I don’t think this study does.

–Matt

One thing which caught my eye was the following from pages 2-3:

For example, Rhode Island suspended the death penalty in 1972, reinstated it in 1977, and abolished it in 1984. Although many factors that affect Rhode Island’s murder rate differ in 1972 and 1984, the murder rate increased after both suspensions: by 13 percent after 1972 and by 25 percent after 1984 in one-year comparisons. The 1984 increase even reversed a declining trend in Rhode Island’s murders.

The numbers around 1984 are as follows (absolute number of murder victims):

1981 40
1982 35
1983 26
1984 33
1985 35
1986 34

There *was* a short-term trend downwards, but the death-penalty was re-instated in 1977, after which there was an *increase* in the murder-rate (indeed, 1976 has the lowest rate in the period my data covers). Over the period 1973-2002, the average yearly change (absolute change, so not taking account of positive or negative) was 16.3, with a standard derivation of 14.7. A 25% yearly change would not seem to be terribly significant.

It bothers me that the authors single this data out as supporting their claim, when to my mind, it does not support the claim at all!

I cannot make a better critique (come on Chris Lightfoot!!!!) but a starting place might be to see which of the following the authors have followed:

i) Looked at a change in the law in year N and then the change in homocide rate in year N and N+1. Conclude: the distribution across states and time is that the change is positive after a law stopping executions.

ii) Looked at the change in homocide rate between every year, and tried to see if there was any corollation between a change in the law and a change in the rate. Conclude: there is a much higher than expected change in the murder rate after a law change.

My belief is that these are very different problems: I don’t think one can conclude much from statement (i) without looking at statement (ii). I think the authors looked at (i) without looking at (ii). But I’d be happy for someone to suggest I’m wrong in this.

Cheers, –Matt