Drones: Untangling the data

There is no official account of the covert war’s human cost. The CIA does not officially acknowledge or comment on its drone campaign, and the Pakistani government does not publish a count of those killed and injured.

Instead, the most comprehensive information on casualties lies in the thousands of press reports of drone strikes filed by national and international media:. The Bureau has scoured these articles to help produce its dataset.

We have cross-referenced these against information provided by researchers. Where we have found later accounts of drone victims, for example, we have attempted to match their account to particular strikes.

But initial news reports have been the Bureau’s key source – and they are often far from conclusive.

The extent of what Donald Rumsfeld called ‘known unknowns’ can be startling. Take, for example, this excerpt on strike Ob97, from Afghan newspaper Pajhwok:

… another three missiles targeted a house used by militants in Dathakhel village, some 50 kilometres west of Miramshah, on Friday, killing 11 militants, he said. The identities of the dead and the networks to which they may have been affiliated to and whether there were any high-value targets among them was not immediately clear, he said.

In other words, the reporter can confirm the location, the date, and the number of people killed – and little else.

Even where specific details are reported, they sometimes contradict details from other reports. In the havoc following a drone strike, this level of confusion is to be expected, but it can make the interpretation of events complex.

Predator drone-Flickr/The Insider Brief

Quoting the sources
Strike Ob35 took place on September 8 2009. The Bureau has identified seven English-language press reports on the strike, which all refer to between nine and 15 militants being killed. However, Geo TV reports the strike hit the house of ‘a tribesman named Ismail Khan’, while Dawn says ‘the missiles damaged the residence of one Maulvi Taib Shah’.

The strike is variously reported to have taken place in the villages of Dargamandi, Syedgai and Tabi Saidgai.

What is more troubling is that although some reports carry quotes from unnamed officials referring to the dead as ‘militants’ and saying, ‘Almost all of those killed are Taliban fighters’, the report in Dawn states, ‘Three children and two women were among the dead’.

So how can we know what really happened?
The name of the owner of the compound is an interesting detail, but is often not essential. The name of the village is more important. Place names can be reported with wildly differing spellings, and Tabi is a common prefix for a town’s name, so Syedgai and Tabi Saidgi could easily refer to the same place – Saidgai, as we have opted to spell it. Looking at the map reveals that Saidgi and Dargamandi are within a couple of miles of one another, so this strike could legitimately be reported as occurring in both places.

Then we come to the conflicting accounts of who died.

Information provided to the Bureau by Reprieve and allied lawyers in Pakistan, as well as other reputable NGOs, reveal that children have died in many strikes where the initial reports suggest that only militants had been killed: civilian deaths seem to be widely under-reported. Where any report mentions that civilians have been killed, we have included this in our data, although if we cannot name the dead, we state that these are reported deaths, not confirmed ones.

This is one example of hundreds. On days when there are multiple strikes, the picture can become even more muddled, and it can take painstaking cross-checking and comparing of reports to develop an understanding of what happened.

Related article: ‘You cannot call me lucky’ – over 1,100 people injured by drones

Even precise statements must be treated with care: in strike Ob166, on November 28 2010, agency Deutsche Presse-Agentur reports a drone strike on a vehicle, quoting an anonymous intelligence official as saying: ‘The locals have pulled out three charred bodies from the wreckage of the vehicle’.

This appears categorical – until it is compared with other reports of the strike, which consistently state that the drone missed with its first missile, which allowed the suspected militants to get out of the car and escape before the second missile struck.

Where the reports have been particularly difficult to disentangle, we have contacted the journalists involved, asking them to revisit their notes or to suggest explanations of why accounts might conflict with one another. They have often been generous and extremely helpful in their response.

Finding reports to corroborate
The level of interest in reporting drones over the seven years of strikes has ebbed and flowed: the very first drone strikes in 2004 and 2005 were barely reported as drones at all – instead, the standard reports were of operations by the Pakistani military, or of accidental explosions while making bombs.

This means that contemporary reports are scant and hard to find. As the strikes became more frequent, they were more widely reported, but Waziristan is a difficult and dangerous place to report from.

Zaffar Abbas, editor of Pakistani English-language paper Dawn, told the Bureau:

Reporting from Waziristan has remained a tricky issue for the last few years, and at times we have advised our correspondents [and] stringers to relocate themselves to nearby towns for security reasons.

Communications from the tribal regions are also difficult, he added.

Since late 2010, reporting has been particularly confused. A poor security situation and tumult elsewhere in Pakistan may have reduced the number of reporters on the ground in Waziristan.

As a result, many publications are running reports produced by agencies such as AP or Reuters, rather than original reports. Often they will combine several agency reports. While it can initially appear that there are six or seven sources for a strike, these sometimes turn out to all rely on the same two or three wire reports, giving the Bureau fewer sources to compare to one another.

Using all available sources
The expertise we had developed from analysing scores of earlier strikes was invaluable at this point, as was the input from journalists and other parties. We have also employed researchers in Waziristan to provide more detail on certain key strikes.

The reports we have used are sometimes conflicting and fragmentary, but in the absence of official data they have been an invaluable resource – and sometimes the only resource available. We are indebted to the courage and tenacity of the journalists working in Waziristan.

The Bureau has invested hundreds of hours in creating what we believe is the most comprehensive, accurate and reliable dataset on the CIA’s drone strikes in Pakistan. Where there are ambiguities, we have reported them as such; where we have reported certainties, we can point to concrete evidence to support our claims. We have discovered around 8% more strikes than are included in most tallies.

When the Bureau presented its findings to the US intelligence community, the response was that our data was ‘wide of the mark’. A counter-intelligence official told us: ‘Our information is by far the most accurate because we have real-time eyes on the targets, as well as multiple other forms of collection to assess who may have been killed.’

The Bureau has now obtained estimates from the US intelligence community of the estimated numbers the CIA believes it has killed. Yet the US reportedly has no intention of publishing in full its data.

The Bureau’s major assessment of CIA drone strikes in Pakistan is an ongoing project. If you would like to be kept informed on this issue click here to sign up to our newsletter.