Analysis: CNN expert’s civilian drone death numbers don’t add up

CNN’s national security expert Peter Bergen speaking at a recent event (Miller Center/ Flickr)

Following recent revelations by the New York Times that all military-aged males in Waziristan are considered fair game by the CIA in its drone strikes, many US journalists have been reassessing how they report on deaths in the attacks.

So when CNN’s national security analyst Peter Bergen produced a graph claiming that no civilians have been killed in Pakistan this year by US drones, his views were bound to attract criticism. Conor Friedersdorf, a columnist at The Atlantic, accused CNN and Bergen of running ‘bogus data‘, for example.

Bergen is also a director of the New America Foundation, which for more than three years has run a database on CIA drone strikes in Pakistan and produces estimates of numbers killed. That data is the most frequent source of statistics for the US media, including CNN itself. So the accuracy of its material is important.

Yet there are credible reports of civilian deaths in Pakistan this year. And unlike the New America Foundation the Bureau actively tracks those claims.

Up to July 16 for example, between three and 27 civilians have been reported killed in Pakistan this year, out of 148 – 220 deaths. Some were actively defined as civilians by news organisations including Reuters and AFP. But these are not necessarily the only civilian deaths. Ambivalent reports might sometimes refer only to ‘people’ or ‘local tribesmen’ killed. More research is needed. And of the remaining alleged militants killed, we have so far been able to name just 13 individuals.

CNN’s controversial graph suggesting the CIA has killed no civilians in Pakistan this year.

Bergen’s claim of zero reported civilian casualties this year is therefore factually inaccurate.

To be so categoric is also problematic. The Bureau’s own data shows that of at least 2,500 people killed by the CIA in Pakistan since 2004, we publicly only know the identities of around 500. Most of the others were reported to be alleged militants by local and international media. We can say no more than that.

It is not just in NAF’s 2012 data that credible reports of civilian deaths have been missed or ignored. NAF’s Pakistan data also contains many other inaccuracies. A number of confirmed strikes are omitted, for instance, and its overall estimates of those killed are significantly below even the CIA’s own count. The consequence is a skewed picture of drone activity which continues to inform many opinion-makers.

Subjective choices
On July 13 Peter Bergen responded to his recent critics in a CNN article which stated that reported civilian casualties in Pakistan are in decline – as the Bureau itself recently noted. He also repeated his claim of no civilian casualties in Pakistan this year. And he attacked the Bureau for its own recording work in this area:

The Bureau of Investigative Journalism’s high estimate of 24 civilian deaths in 2012 came in part from reports provided by an unreliable Pakistani news outlet as well as the claims of a local Taliban commander, which contradicted all other reports.

It’s worth unpicking Bergen’s claims in some detail.

His comments appear to refer to a CIA drone strike on February 9 in which local Taliban commander Badar Mansoor died. Citing just four sources, NAF’s data reports only that three to five ‘militants’, including Mansoor, died in the attack.

But this is a misrepresentation which ignores credible claims of civilian casualties, as the Bureau’s own Pakistan database makes clear.

Among 18 unique sources we cite, the Bureau links to a story by Reuters, the international news agency. Reuters notes a Taliban commander’s claims that Mansoor’s wife and child died in the February 9 attack. Local paper The News also reported that Mansoor’s wife and children were either injured or killed; and a Bureau field researcher reported anecdotal claims from the town that some of the leader’s family had died.

As the Bureau notes, these overt claims of civilian deaths on February 9 remain contested. We state that between zero and two civilians reportedly died in the strike. It is not clear either way. What cannot be stated is that no civilians died.

Bergen’s reference to an ‘unreliable Pakistani news outlet’ is also confusing. Dawn, The Nation and The News are all reputable Pakistani dailies, cited on occasion by CNN and NAF themselves. And Central Asia Online states clearly that ‘a woman and a girl child were injured’ in the strike, not killed.

In fact Bergen’s comments undermine further the credibility of the NAF data he constantly cites. A partial list of media reports has not been updated since the day of the attack – despite a number of salient facts since emerging. And as Bergen notes in his CNN article, the Reuters report of civilian deaths is rejected as a NAF source on the (inaccurate) grounds that it involved ‘the claims of a local Taliban commander, which contradicted all other reports.’

In their CNN article Bergen and co-reporter Jennifer Rowland make no mention of a second strike in which civilians were also reported killed in Pakistan this year. According to credible media, along with a number of alleged militants between three and eight worshipers died when a mosque was struck (possibly accidentally) on May 24.

That claim is independently supported by Britain’s Channel 4 News; by Pakistan’s The News (generally the most accurate local source for information on casualties); and by French news agency AFP. The Bureau cites 17 unique sources overall in its coverage, noting reports of damage to the mosque and of civilian casualties.

Bergen’s New America Foundation, relying on just four sources, says only that 10 ‘militants’ were killed in a ‘compound.’

NAF’s claims of ‘zero civilians killed’ by the CIA in Pakistan in 2012 is reached by the simple expedient of not including in its data any of the credible reports of civilian deaths.

Full of errors
When the Bureau began looking in earnest at US drone strikes in summer 2010, we started to work with NAF’s data, and that of the Long War Journal. At that time we had no interest in the time-consuming (and expensive) effort of compiling and maintaining accurate data on covert US strikes.

But the more we worked with NAF’s material, the more troubled we became. In February 2011 for example, the Bureau wrote to NAF noting a number of errors.

We pointed out a strike that it had missed entirely (November 5 2005). The Bureau also drew NAF’s attention to a number of date errors. The Foundation claimed a strike had taken place on May 14 2005, for example. In fact that attack took place on May 8th.

Bergen personally acknowledged the email, saying ‘thanks for drawing attention to these.’ Yet almost 18 months on those errors – easily verifiable – remain uncorrected.

Our concerns about the data – particularly on the question of civilian deaths – ultimately compelled us to start from scratch, re-examining every US drone strike in Pakistan to try and understand what had really been going on.

We now know, for example, that eight years in to the CIA’s bombing campaign in Pakistan, NAF still lists the wrong date (June 18 2004) for the very first strike. Citing just one source, NAF also makes no reference to the civilians killed that day, including two children.

That first attack actually took place on Thursday June 17 as CNN and many other sources correctly noted at the time. Militant commander Nek Mohammed died along with up to eight others. These included, it was widely reported, the two young sons of Sher Zaman Ashrafkhel.

On another occasion in October 2006, an attack on a seminary killed at least 81 people. New America Foundation does not count these ‘militants’ in its data, reporting that the attack was

Allegedly conducted by Pakistani military, but may have been conducted by US forces. Noted here for the record but not included in above fatality totals.’

Claims that the Pakistan military carried out this attack were long ago dismissed. A senior aide to Pakistan’s then-leader Pervez Musharraf told the Sunday Times within weeks that ‘we thought it would be less damaging if we said we did it rather than the US.’ Last August former ISI director General Asad Durrani confirmed in an interview that the CIA carried out the strike. And just weeks ago General Musharraf himself pointedly refused to deny US involvement.

There are also reports that up to 69 children died in the October 2006 attack. While some contest this claim, local media has listed the full names, ages, family details and home villages of every child reported killed.

Bergen and New America Foundation continue to make no reference to any of these salient facts. Nor do they count these 81 deaths in their figures.

‘Civilian deaths not new’
The New America Foundation regularly publishes definitive numbers on the overall civilian death tolls in Pakistan.

On March 27 for example, Bergen and co-worker Jennifer Rowland claimed that ‘according to our data, 7% of the fatalities resulting from drone strikes [in Pakistan] in 2011 were civilians.’ The duo lowered that estimate on June 10, now claiming that civilian deaths in Pakistan ‘averaged 5.5% in 2011.’

The Bureau has been unable to replicate either of NAF’s recent statistical claims from the Foundation’s published data.

In contrast, our own data shows that between 465 and 659 people died overall in 2011. Of these between 75 and 127 were reportedly civilians. Since we cannot know where, within these ranges, accurate figures lie, the best that can be said is that reported civilian deaths account for between 11% and 27% of all of those killed by the CIA in Pakistan last year.

In July 2011 the Bureau issued a major report based on its first field investigation in Pakistan. This directly challenged US claims that it wasn’t killing civilians in the tribal areas, presenting the CIA with the details of 45 civilians killed in the specified period and raising significant concerns about a further 66 deaths.

Initially the Bureau’s report received little coverage in the US, not only by the mainstream US media but also by the influential AfPak Channel, which is edited by Bergen and Rowland. When we challenged this omission, New America Foundation senior advisor Patrick Doherty shed some light in a July 19 email on why our study had been ignored:

One reason is that the tallies on civilian deaths in PAK is not particularly new. We’ve been monitoring drone strikes for a few years and tracking civilian and militant deaths. The mainstream media has been reporting on our numbers, quite thoroughly, in fact. The gotcha on John Brennan, as a result, kind of rings hollow.

In fact no US media organisation had challenged US intelligence community assertions that civilians were no longer being killed by the CIA in Pakistan. Those extraordinary claims went uncontested for six months, when the Bureau published its investigation.

Snapshot nature
Perhaps the greatest issue with New America Foundation’s data is its incomplete, snapshot nature.

Public understanding of US covert drone strikes changes all the time. That’s why the Bureau’s eight databases change constantly, incorporating the latest understanding of each attack – and seeking information from as wide a pool of credible sources as possible.

Last year, for example, we learned that a 2009 attack in Pakistan which had initially been reported as killing alleged militants, women and children appeared to have been a strike on a child suicide bomber training camp, run by the Taliban. More recently, we incorporated evidence from sworn affidavits filed in the London High Court, relating to the deaths of many civilians in March 2011.

In contrast NAF’s data represents at best a partial snapshot of an attack, often based on just a few media reports on the day. No effort appears to be made to update, amend or correct its data.

In his most recent article for CNN looking at civilian deaths in Pakistan, Bergen cites a major Associated Press investigation into drone strike casualties published in February of this year. As the Bureau reported at the time, that investigation, based on 80 witness statements, uncovered previously unknown evidence of civilian deaths in a number of strikes. We amended our data records accordingly, to reflect these findings.

NAF has yet to change any of its records – despite Bergen citing AP’s study in his own defence. So while AP reports that seven civilians died alongside seven Taliban on August 14 2010, NAF continues to state only that ‘7-13 militants were killed.’

The cumulative effect of all these omissions and errors is that NAF’s data substantially under-estimates both the overall numbers of those killed, and the reports of civilians who have died in Pakistan strikes.

In August 2010, in response to the Bureau publishing its Pakistan data, the US government issued its first overall estimate of the numbers killed in CIA drone strikes since 2001, stating that approximately 2,050 had died – all but fifty of them combatants.

Eleven months on, and 47 strikes later, a minimum of 262 further deaths have occurred in Pakistan. Yet the New America Foundation still gives a low estimate of 1,870 killed. That indicates that its estimates are some 400 below the CIA’s own numbers.

The New America Foundation has undoubtedly done valuable work in recent years in bringing to the attention of the US public the scale of America’s covert wars. Its data represents a useful snapshot of most strikes, a helpful base upon which further research can be built.

But NAF cannot claim that its data represents an accurate record of what we publicly know about US drone strikes in Pakistan and elsewhere. And without radically overhauling its methodology, Bergen and NAF cannot credibly continue to offer up such precise estimates of ‘civilian deaths’ in Pakistan.

Follow @chrisjwoods on Twitter.