Good Stats     Bad Stats

An interesting post of a talk at Strata 2012 by Dr. Ben Goldacre on Statistical Analysis.

How else can I say it.

The Washington Post reported yesterday on how Chief Lanier calculates the “closure rate” for homicides in the District of Columbia. The story was first reported by DC Homicide Watch in December of last year.

Chief Lenier has been reporting a “closure rate” of 94% for 2011 and claims the the methodology used is consistent with that used by the FBI in their Uniform Crime Reports. Her calculation of the percent is the ratio of all homicides “closed” in 2011 to the number of homicides committed in 2011. Thus crimes used in the numerator are in many cases not reflected in the denominator as they may have been committed in prior years. I do believe that most elementary schools teach the percentages do require that cases in the numerator be a subset of those in the denominator. So perhaps Chief Leneir needs to review the arithmetic that she was taught in elementary school before she defends her methodology.

So what about the FBI and how they compute “closure rates?” Actually they use a different term – “clearance rates.” Their definition of “clearance rate” is given in the table descriptions for the UCR: “These tables provide the number of violent crimes, property crimes, and arsons with a breakdown of the offenses known to law enforcement and the percentage of those offenses that were cleared by arrest or exceptional means.”

What is the “true” clearance rate? That is really not a known and measurable quantity. The reason is that a crime may be cleared years after it was committed. All that can be reported at any point in time is the clearance rate or closure rate as of a given data. That rate will change in future as additional crimes are cleared and as innocent parties are later exonerated that were at on time thought to be the perpetrators of the crime.