What Should We Expect of Our Spies?


May 25, 2011

This commentary originally appeared in Prospect Magazine on May 25, 2011.

American agents found Osama bin Laden, but were wrong-footed by the fall of the USSR, the Arab Spring and Iraq's lack of WMD. What counts as success?

When I worked at America's National Intelligence Council, overseeing the production of the National Intelligence Estimates (NIEs), I took comfort from being told that you could outperform professional weather forecasters simply by assuming that current conditions would continue. If it was fine, predict fine weather until it rained, then predict rain until it turned fine. That would more often be accurate than the forecasters' predictions, based on data, theory and history. And if they, with all their tools, could not predict the weather, how could anyone expect us to predict a complicated human event like the collapse of the Soviet Union?

The question behind this musing was "what should people expect of their intelligence agencies?" Not "what would they like?"; for policymakers would like perfect prescience, if not omniscience. They know that they can have neither.

Reasonably, expectations should differ according to the problem at hand. But start with that hoary case which has hung over the intelligence services for 20 years: should they have done better in foreseeing the end of the Soviet Union? After all, the premise of the west's strategy was that if Soviet expansion were contained, eventually the empire would collapse from its own internal contradictions. So some monitoring of how this policy was doing would have seemed appropriate.

In retrospect, there were signs aplenty of a sick society. Emigrés arrived with tales of Soviet toasters that were as likely to catch fire as to brown bread. The demographer Murray Feshbach presented Washington with a raft of Soviet statistics in the mid-1970s, most of which, like male life expectancy, were going in the wrong direction for a rich country. These factoids were puzzling, but we rationalised the first on the grounds that while the USSR produced poor quality household goods, its defence industry was a separate and much more efficient operation. The second we dismissed with "Russians drink too much." The political scientist Emmanuel Todd did Feshbach one better and turned the numbers into a prediction of the Soviet Union's collapse. But he suffered the double misfortune of not only being French but also writing in French, and so was not likely to make much of a dent in official Washington.

Intelligence is about creating and adjusting stories—this view has crystallised during my career as a producer and consumer of intelligence. In the 1970s and into the 1980s, the story in the heads of policymakers was of Soviet expansion abroad, not disintegration at home. Thus, those Feshbach statistics were just curious factoids. The Soviet invasion of Afghanistan, the "evil empire" and "Star Wars" were still in the future. Imagine an intelligence officer who had tried to explain to the newly elected Ronald Reagan that the Soviet problem he faced was not power but impending collapse. That analyst would soon have found himself counting Soviet submarines in the Aleutian Islands. Questions not asked or stories not imagined by policy are not likely to be answered or developed by intelligence.

The best prediction of Soviet implosion I have seen was a slightly whimsical piece written in The Times, by the British columnist Bernard Levin in September 1977. He got the process exactly right: change would come not from the bottom but from the top, from leaders who "are in every respect model Soviet functionaries. Or rather, in every respect but one: they have admitted the truth about their country to themselves, and have vowed, also to themselves, to do something about it." Levin didn't get the motivation of the high-level revolutionaries right—he imagined a deep-seated lust for freedom, rather than concern over the stagnating Soviet economy—but at least he had a story. For the sake of convenience, he picked the 200th anniversary of the French revolution as the date, 14th July 1989.

Closer to the end, CIA assessments were on the mark but still lacked a story. The agency had been pointing to a chronic slowdown in the Soviet economy since the 1970s, and a 1981 report was blunt: "The Soviet pattern in many respects conforms to that of less developed countries, and remarkable little progress toward a more modern pattern." By 1982, CIA assessments reported that Soviet defence spending had stopped growing, and the following year the agency concluded that defence spending had in fact been tailing off since 1976.

Interestingly, those who could imagine the story didn't believe it could be true. Unlike Levin, they did not believe the Soviet Union could be reformed from the top and in that they turned out to be right. The director of America's eavesdroppers, the National Security Agency, Lt Gen William Odom wrote in 1987 that Mikhail Gorbachev's programme, if followed to its logical conclusion, would lead to his political suicide and the collapse of the system. Because this did not seem what the Soviet leader had in mind, he and others, including Robert Gates, then deputy director of central intelligence, concluded that Gorbachev could not intend to do what he said he would.

In fact, the Soviet Union didn't have to end in 1991. It might still be doddering along but for the actions of that visionary bumbler, Mikhail Gorbachev, who understood his nation's weaknesses but had no idea how to deal with them, and who set in motion economic reforms that were too much pain for not much gain. In this case, what we could have expected of intelligence was not prediction but earlier and better monitoring of internal shortcomings. We could also have expected competing stories to challenge the prevailing one. Very late, in 1990, an NIE, "The Deepening Crisis in the USSR," did just that, laying out four different scenarios, or stories, for the next year.

When the Soviet Union would collapse was a mystery, not a puzzle. No one could know the answer: it depended. It was contingent. Puzzles are a very different kind of intelligence problem. They have an answer, but we may not know it. Many of the intelligence successes of the cold war were puzzle-solving about a very secretive foe: were there Soviet missiles in Cuba? How many warheads did the Soviet SS-18 missile carry?

Puzzles are not necessarily easier than mysteries—consider the decade it took to solve the puzzle of Osama bin Laden's whereabouts. But they do come with different expectations attached. Intelligence puzzles are not like jigsaws, in that we may not be sure we have the right answer. The raid on bin Laden was launched, participants in the decision said, with odds that he was actually in the compound put at no better than six in ten. But the fact that there is in principle an answer provides some concreteness to what is expected of intelligence.

That is especially so at the more tactical level of intelligence. In the simplest case, in targeting (or producing, in wonderful Pentagonese, "desired mean points of impact," or DMPIs, pronounced "dimpies"), the enemy unit either is or isn't where intelligence says it is. The intelligence will quickly be self-validating as the fighter pilot or drone targeter discovers whether the enemy unit is in fact there. The raid on bin Laden reflected the solution to a much more complicated puzzle, one that was a nice example of the various forms of collection and analysis working together. But in that case too it would have been immediately apparent to the raiders if bin Laden hadn't been there.

Another puzzle, whether Saddam Hussein's Iraq had weapons of mass destruction in 2002, drives home the point that because intelligence is a service industry, what policy officials expect from it shapes its work. In the WMD case, neither the US investigating panel nor the British Butler report found evidence that political leaders had directly pressured intelligence agencies to come to a particular conclusion. Yet it is also fair to report that some analysts on both sides of the Atlantic did feel they were under pressure to produce the "right" answer: that Saddam Hussein had weapons of mass destruction.

The interaction of intelligence and policy shaped the results in several other ways. Policy officials, particularly on the US side, when presented with a range of assessments by different agencies, cherry-picked their favourites (and sometimes grew their own cherries by giving credibility to information sources the intelligence services had discredited). As elsewhere in life, how the question was asked went a long way toward determining the answer. In this case, the question became simply "Does Saddam have WMD?" Intelligence analysis did broaden the question, but issues of how much threat, to whom and over what timeframe got lost in the "does he?" debate. Moreover, American intelligence was asked over and over about links between Iraq and al-Qaeda. It stuck to its analytic guns—the link was tenuous at best—but the repeated questions served both to elevate the debate over the issue and to contribute to intelligence's relative lack of attention to other questions.

In the end, however, the most significant part of the WMD story was what intelligence and policy shared: a deeply held mindset that Saddam must have WMD. That mindset included outsiders like me who opposed going to war, as well as other European intelligence services whose governments were not going to participate in any war. For intelligence, the mindset was compounded by history, for the previous time around, in the early 1990s, US intelligence had underestimated Iraqi WMD; it was not going to make that mistake again. In the end, if most people believe one thing, arguing for another is hard. There is little pressure to rethink the issue, and the few dissenters in intelligence are lost in the wilderness.

What should have been expected from intelligence in this case was a section in the assessments asking what was the best case that could be made that Iraq did not have WMD. That would not have made the slightest bit of difference in the rush to war, given the power of the prevailing mindset, but it would at least have offered intelligence agencies some protection from later criticism—fair enough—that they had not done their job.

What policy officials expect from intelligence also shapes how intelligence is organised and what kind of people it hires. On the American side of the Atlantic, the crown jewel of intelligence products is the President's Daily Brief (PDB), perhaps the most expensive publication per copy since Gutenberg's Bibles. Often caricatured as "CNN plus secrets," much of it is material recently collected by a spy, or satellite image, or intercepted signal, plus commentary. On the British side of the ocean, there is less of a flood of current intelligence, and the assessments of the Joint Intelligence Committee are, in my experience, often thoughtful. But on both sides, the tyranny of the immediate is apparent. As one American analyst put it to me: "We used to do analysis; now we do reporting."

The focus on the immediate, combined with the way intelligence agencies are organised, may have played some role in the failure to understand the contagion effects in the recent Arab Spring. In the US, especially, where analytic cadres are large, analysts have very specific assignments. The Egypt analysts are tightly focused on that country, perhaps even on particular aspects of it. They would not have been looking at how events in Tunisia might affect Egypt. To be fair, the media probably overstated the contagion effect of events from one Arab country to the next—but that there was some such effect seems apparent in retrospect. Worse, my bet is that if asked whether events in Tunisia might affect Egypt, even slightly, those Egypt analysts would have said "no" with more or less disdain.

In the end, what is expected of intelligence shapes what capabilities it builds—and hires. At the tactical level, there are teams of young analysts from the big US collection agencies (the National Security Agency for Signals Intelligence—SIGINT—and the National Geospatial-Intelligence Agency for Imagery—IMINT), who have been organised into "geocells." These teams have become adept at combining SIGINT and imagery, and adding what has been learned from informants in the battle zones, in order to identify events of interest and ultimately provide those targets: the DMPIs.

The demand for those DMPIs is plain enough, and the PDB's collected secrets are beguiling if not always helpful. But there is less demand from policy officials for more strategic, longer-term assessments. When asked, officials say they would like them: how could they answer otherwise? But in practice too often the response when presented with a longer-term assessment is: "That looks interesting. I'll read it when there is time." And there never is time. When I was at the National Intelligence Council (NIC) overseeing NIEs we had a good idea. We'd do a short intelligence appraisal of an important foreign policy issue, and the State department's policy planners would add a policy paper. We'd then convene the deputies—the number twos in the various foreign policy agencies—over an informal lunch. The conversation would begin with the outcome the US sought a decade out, then peel back to current policy. We got such a session on the deputies' calendar exactly once.

Lacking demand, it is not clear that intelligence agencies hire or train people who could do good strategic analysis—that is, analysis that locates choices in a wider context and perhaps in a longer timeframe. Most analysts are trained to look for measurable evidence and struggle with alternative possibilities, and are not always willing to venture beyond the facts and the level of policy description. To be sure, there are differences across agencies. The State Department's Bureau of Intelligence and Research, while small, does value deep expertise, letting analysts stay on a particular account for an entire career. By contrast, the analytic arm of the CIA believes good analysts can add value quickly as they move from account to account. As a result, it has more the feel of a newsroom than a university.

At the NIC, I came to think that, for all the technology, strategic analysis was best done in person. I came to think that our real products weren't those papers, the NIEs. Rather they were the NIOs, the National Intelligence Officers—the experts, not papers. We all think we can absorb information more efficiently by reading, but my advice to my policy colleagues was to give intelligence officers some face time. If policymakers ask for a paper, what they get will inevitably be 60 degrees off the target. In 20 minutes, though, the intelligence officers can sharpen the question, and the policy official can calibrate the expertise of the analyst. In that conversation, intelligence analysts can offer advice; they don't need to be as tightly restricted as they are on paper by the "thou shalt not traffic in policy" edict. Expectations can be calibrated on both sides of the conversation. And the result might even be better policy.

Gregory F Treverton directs the RAND Corporation's Centre for Global Risk and Security. He has worked for the US Senate, the National Security Council and the National Intelligence Council. He is the author of "Intelligence for an Age of Terror" (Cambridge University Press).

More About This Commentary

Commentary gives RAND researchers a platform to convey insights based on their professional expertise and often on their peer-reviewed research and analysis.