Validity of Lancet Study

Echidne, who has been doing a series on statistics, chimes in on the Lancet study (pdf) that investigates the death rate in Iraq since our invasion, and finds an additional 400,000-900,000 deaths:

In a nutshell, the wingnuts hate the study, because the findings suggest that a near-genocide is going on in Iraq, and the moonbats defend the study ferociously, because it confirms their expectations that a near-genocide is going on in Iraq. Nobody is happy about the study findings, of course. Let me repeat that: Nobody is happy about the study findings; nobody wants to imagine that many horrible deaths and the suffering that goes along with those or the effect on the survivors.

After discussing the way in which the study was completed (which is pretty much normal public health statistical technique, mitigated by the fact that so much is a war zone), she looks at the critiques of the study. The first is simply - geez, no-one else says the number is that high!

That there is a difference in these numbers can be at least partly accounted for by the fact that the Lancet study was actively looking for deaths in the community, whereas all the other sources are based on passive reporting: stories in newspapers, checking on morgues and so on. It's pretty likely that a war-torn country has large numbers of deaths which are not reported on, especially a country like Iraq where large areas of the country are too dangerous for journalists to venture in. This does not mean that the Lancet numbers are necessarily correct, of course, but it suggests that we must take into account the different methods other death counts use before comparing the two.

Then there's the cluster sampling method, and the honesty of the people interviewed.

But the cluster sampling method is widely used for estimating deaths in conflict areas. Its weakness, compared to simple random sampling, is taken into account in the wide confidence intervals the estimates produce...

The third most common criticism has to do with the truthfulness of the survey results and respondents. The research teams asked for a death certificate in 87% of the cases and were shown one in 80% of all cases. It's unfortunate that so many people who write about the study are using the higher percentage of 92% confirmation rate. This only applied to the cases where a certificate was requested. But 80% is fairly impressive, too.

Pretty damning. Which is why the condemnation of the study from the perpetrators, those who have the blood of 655,000 (plus or minus a couple hundred thousand) on their hands is so swift, if not accurate.