Oversight and 'The Responsibility to Provide'

A few weeks ago the thinktank IPPR published a paper on the future of intelligence work by Sir David Omand, a former senior Civil Servant. Already people are treating his paper as a representative statement of future government policy. A number of broadsheets praised the paper for its insistence on more oversight of surveillance activity. In particular, he identifies 6 principles which should govern this oversight:

1. There must be a sufficient sustainable cause.
2. There must be integrity of motive.
3. The methods used must be in proportion to the seriousness of the business at hand.
4. There must be proper authority.
5. There must be a reasonable prospect of success.
6. The recourse to the methods of secret intelligence must be a last, not a first, resort.

Don't get me wrong. I definitely think more oversight is a good thing, and I don't disagree with any of the above principles. But I do think there is a major tension between the idea of oversight and another key aspect of what he wants for the future of intelligence. which so far I haven't really seen discussed at all. Early on in his paper he celebrates the way that the intelligence community has moved on from the old paradigm of the 'need to know' to the new era of 'the responsibility to provide':

Authority and information will need to be pushed down to enable local problems to be tackled at local level, but at the same time national authorities must seize the issues that have international dimensions (and local imports), such as terrorism, narcotics, illegal immigration and organised crime. The national intelligence authorities will be expected to both ensure that the local enforcement level - including police, border forces and other local authorities - have the necessary information, and to help manage the international dimensions of these domestic threats. These responsibilities are likely to accentuate the shift away from the highly restrictive 'need to know' culture of the traditional intelligence world to what US Director of National Intelligence, Mike McConnell, has called 'the responsibility to provide', a phrase that captures the spirit of the new approach to the provision of intelligence for the purposes of public protection.

Later, when talking about the dissemination of intelligence already gathered he produces an even more vivid picture of how this would ideally work in practice:

The shift away from the highly restrictive 'need to know' culture must continue. Today dissemination must be both outwards, including to partners and allies overseas, and downwards, where the issues around classification, tear-line reporting and 'fusion centres' are now well discussed in the literature...Finally, the customer community, especially in the military commands, will have to be increasingly able to rapidly pull the intelligence material needed to generate situational awareness and enjoy the 'Amazon.com' ability to find past products and perhaps be told, as you are when you search for a book on Amazon, which other products users of that item also found useful.

Doubtless people in the police and security would enjoy this 'Amazon.com' ability very much. It is easy to see the attraction of more convenient sharing of information. But how easily does this sit with the idea of oversight? Part of what makes amazon's suggestions so useful is that they are instantaneous and automatic - you are never waiting for the time it would take for a human judgement. But it's hard to square the immediacy of the technology with independent scrutiny.

The trouble is that Omand's six criteria for proper oversight seem to require human judgement. True, one could program Omand's 'amazon-style' site to only 'make suggestions' to users with 'proper authority'. But you couldn't use a computational function to assess matters like 'proportionality', 'prospects of success' or that intelligence is used 'as a last resort'.

The need for human judgement is problematic for the convenience of Omand's 'Amazon.com' vision. Ben Goldacre makes a similar point in relation to datamining in one of his recent 'Bad Science' columns. His basic point is that in any case of searching for potential suspects by identifying a potential characteristic correlated with the activity in question, the number of 'false positives', all requiring human assessment, are likely to be so vast that the search will not be worthwhile.

If human oversight was required every time intelligence was passed around, this would slow things down in a way that makes Omand's internet age hopes impractical. Proper oversight in intelligence work surely applies to who get to look at the intelligence, not just how the evidence was acquired? And if oversight requires a human judgement in every instance, this cannot be done quickly. You can have the oversight or the convenience, but not both.

I'd like to get some other people's opinions on this. Maybe you disagree with my analysis. Maybe you think I've misinterpreted Omand. And what restrictions ought there to be on the sharing of intelligence among police? Would retroactive oversight be sufficient? How could this work?