The latest talks in Baghdad over Iran's nuclear program have prompted the usual arguments. Iran says it has only peaceful intentions. Israeli leaders scoff at that claim. Other world powers are unsure of Iran's intentions and demand that it take steps to show that it is not pursuing nuclear weapons.
The CIA and other U.S. intelligence agencies, meanwhile, are sticking with the assessment they made in November 2007, when they reported that Iran "halted its nuclear weapons program" in 2003 and apparently had not restarted it.
The CIA faced criticism 10 years ago for reporting, incorrectly, that Saddam Hussein's Iraq had weapons of mass destruction (WMD), and the caution it has shown since its 2007 estimate on Iran reflects lessons learned from its Iraq experience.
"Because we're so focused on the issues about the shortfalls of the Iraq WMD, we need to be very open about the nature of the evidence [on Iran], how much evidence, [and] the role of our assumptions," says Peter Clement, the CIA's deputy director of intelligence for analytic programs.
"Those [issues] factored into that estimate, to say we just haven't got what it takes to make the call that, yes, [the Iranians] have made a decision to go ahead with the [nuclear weapons] program."
Clement and other CIA officials discussed Iran and other issues with NPR in a rare on-the-record session at the agency's headquarters in Langley, Va.
The CIA's guarded statements on Iran have drawn criticism from conservatives.
Former U.N. Ambassador John Bolton said the 2007 estimate was "famously distorted," and his former chief of staff, Fred Fleitz, a former CIA officer, said U.S. intelligence agencies are "unwilling to conduct a proper assessment of the Iran nuclear issue."
But erroneous intelligence judgments can have costly consequences, and CIA officials insist their cautionary approach is well advised.
Criticism Over Iraq
In 2005, then-CIA Director Porter Goss, prompted by concerns that the United States and its allies went to war in Iraq on the basis of bad intelligence, ordered an internal review of its prewar Iraq reporting.
The conclusion: There were good reasons to believe Saddam Hussein had weapons of mass destruction. He had them previously, and had used them against his own people. He had kicked U.N. weapons inspectors out of the country, and some CIA informants were reporting that he still possessed them.
CIA leaders, however, concluded that the agency had erred in not highlighting for policymakers the extent to which its conclusions were based on assumptions rather than hard evidence, and in not making clear when the agency had depended on informants whose reliability was untested.
"Even if we had a great source who was a senior official in Iraq," Clement says, "and he's telling us what he believes to be the truth, you could easily be misled."
Even intelligence reports from intercepted telephone conversations can be wrong.
"It's worthy of close analysis," Clement says. "Just because someone said something to a friend doesn't make it true."
'Obsessed About Sourcing'
The 2005 Iraq review prompted the CIA management to institute a series of changes in its approach to analysis and in the way it reported its findings.
"Writing on any controversial issue right now, I think we're very — 'obsessed' may be too strong a word, but not entirely," says Clement, a 35-year CIA veteran who has overseen much of the analytic reforms undertaken at the agency.
"I personally get very obsessed about sourcing. If we make a statement of fact, I want it to be clear where that fact came from. If we got it from a [human] source, how good is the source? Did they have firsthand access? Second-hand access? Are they someone we recently got on the payroll?" he says.
Clement believes if CIA analysts had included those observations in their reports on Iraq WMD concerns, it might have made a difference, even if the overall conclusion had been the same.
"If they had written it with all the appropriate caveats," he says, "perhaps the readers on Capitol Hill or down at the White House would have said, 'You know, reading this, I'm not getting a real strong feeling about how well-founded this estimate is.' And the answer to that would [have been], 'You're correct. It is not well-founded.' "
The post-Iraq changes at the CIA also involve new analytic techniques, highlighted in a "tradecraft primer" in use at the agency since 2009. The manual is now used at the Sherman Kent School, the agency's in-house training institute for new analysts. The manual opens with a section on the "mind-set" challenge.
"If you're only looking at [an issue] through one narrow view of the world, you're not looking at the whole picture," says John, who teaches at the Kent School. He and other CIA analysts were willing to be quoted only if their last names were not revealed.
"Your biases will get you things like a confirmation bias: 'I've seen it before, so it must be happening again.' Or an anchoring bias: 'We've come up with that conclusion, and I think it's true, and it's not going to change.'"
One exercise now in use at the CIA is called "Analysis of Competing Hypotheses." Analysts who may be inclined toward one explanation for some notable development are forced to consider alternative explanations and to tally up all the evidence that is inconsistent with their favored hypothesis.
"You're looking for the hypothesis with the least inconsistencies," says John, who's been at the CIA for 34 years. "We call it the Last Man Standing approach."
Dealing With Uncertainty
Such exercises are employed throughout the CIA's Directorate of Intelligence. Each office now includes a "tradecraft cell," staffed by specialists whose mission it is to make sure their colleagues are using all the latest analytic techniques and challenging their own judgments.
"We want to try to get analysts to be honest about the uncertainties they face or the gaps in their information," says Mark, a tradecraft specialist in the CIA's East Asia office.
With all the checks and balances now incorporated into CIA analysis, it is not surprising that the agency has been reluctant to go out on a limb with its judgment on Iran's nuclear designs.
"We have used many of these tools on the Iran program," says Clement.
The cautions, in fact, are reflected throughout the agency's intelligence work, with analysts more reluctant these days to "make a call" on an issue by offering a prediction or a sweeping judgment.
Suggesting Alternative Outcomes
In the aftermath of the Sept. 11 attacks, the agency established a "red cell" with the mission of suggesting alternative visions of the future.
"There definitely was an emphasis in years past to say, 'It is most likely going to go this way,' " says Maria, the current chief of the CIA red cell. "We still have to make those calls, but now we try to explain what factors would take it in a different direction. Could the future be 180 degrees in an opposite direction?"
In one recent example, the CIA red cell offered five separate visions of how the Middle East might be different in 18 months, ranging from a "mosaic" scenario characterized by new democracies and reformist governments, to a region-wide Sunni-Shiite conflict shaped by a Saudi-Iranian competition for influence.
The reluctance to predict one future above others is another shift in the CIA's traditional approach.
"When I came onboard as a young analyst in the '70s and '80s," says Clement, "I distinctly remember managers telling me, 'Peter, the reason they pay you the big bucks is to make an analytic call. You know, you're the expert. You've studied all this material. Make the call. Give me the answer."
The Risks Of 'Making A Call'
As history shows, however, intelligence agencies can get in trouble when they're forced to "make a call" on an important issue, like whether Saddam Hussein had WMD.
Some former CIA officials have argued that the Bush administration bears some responsibility for the mistaken conclusion on Iraq. White House officials, including Vice President Dick Cheney, they claim, were so determined to go to war in Iraq that they pressured the agency to find a reason for invading that country.
Peter Clement does not share that view. "We appreciate interaction with the policymakers," he says. "Vice President Cheney was a serious consumer."
The new CIA approach does carry some risks. One danger is that the new exercises could incorporate so much caution into CIA analysis that the agency will be paralyzed. Analysts could become so focused on questioning their conclusions that they can't come up with answers.
Robert Jervis of Columbia University, who has written widely on intelligence failures and advised the CIA on its performance, is undecided on whether the new analytic techniques will prove worthwhile.
"I could see them becoming a substitute for careful, informed thought," Jervis says, "a formula [that leads to analysts saying], 'Oh, you don't need to know anything about the country. You just plug these things in.' "
"Misapplied, they would make things worse," Jervis says. "Applied well, they will make things better. How much better? Hard to tell."
ROBERT SIEGEL, HOST:
From NPR News, this is ALL THINGS CONSIDERED. I'm Robert Siegel.
AUDIE CORNISH, HOST:
And I'm Audie Cornish. Diplomats from the U.S. and other countries are in Baghdad today, meeting with officials from Iran. The international community wants Iran to show that its nuclear program is only for peaceful purposes. The official judgment from the CIA and other U.S. intelligence agencies is that Iran, for now at least, is not building a nuclear weapon. But some governments are unconvinced.
SIEGEL: The CIA's caution on Iran, in part, reflects lessons learned from its Iraq experience. The agency has been criticized for reporting 10 years ago that Iraq had weapons of mass destruction, when in fact it did not. NPR's Tom Gjelten visited CIA headquarters recently, and in a rare on-the-record session, agency analysts discussed new safeguards against jumping to conclusions.
TOM GJELTEN, BYLINE: After being embarrassed by what they'd said incorrectly about Saddam Hussein, CIA officials ordered a full review of their Iraq reporting. They wanted to know where they had gone wrong. The conclusion: There'd been good reasons for thinking Saddam had weapons of mass destruction. He'd had them before - he'd used them against his own people; and the CIA had informants who said Saddam still had them. CIA analysts in fact said if they'd been forced to make the call again, they'd probably say the same thing, that Saddam had WMD. Peter Clement, the CIA's deputy director of intelligence for analysis, says where the agency erred was in not highlighting which of its conclusions were based on assumptions, not hard evidence, and not making clear when they'd relied on informants whose reliability was untested.
PETER CLEMENT: Even if we had a great source who was a senior official in Iraq, and he's telling us what he believes to be the truth, you could easily be misled. And so, to me, there were just so many issues with Iraq.
GJELTEN: But that judgment was wrong.
GJELTEN: Therefore, it's not all that reassuring to hear that your analysts would have come to the same conclusion if they were to do it over again.
CLEMENT: I can't disagree with that. But I would say, if they had written it with all the appropriate caveats, the readers - perhaps those on Capitol Hill or down at the White House - they would have said, you know, reading this, I'm not getting a really strong feeling about how well founded this estimate is. And the answer to that would be: You're correct. It is not well founded.
GJELTEN: The Iraq experience, Clement says, changed the way analysts characterize their intelligence, especially that coming from human sources - humint in CIA jargon.
CLEMENT: I think we're very - obsessed may be too strong a word, but not entirely. I personally get very obsessed about sourcing. If we make a statement of fact, I want it to be clear where that fact came from. And if we got it from a humint source, how good is the source? Did they have first-hand access, second-hand access? Are they someone we recently got on the payroll?
GJELTEN: These reforms involve CIA analysts, the people who prepare classified reports for the president and other officials, not CIA spies and covert-action types. In 2009, the agency prepared a new tradecraft manual for its analysts. Peter Clement, himself a 35-year agency veteran, oversaw much of the work, and he recently brought together three of his colleagues in a secure room at CIA headquarters to talk about what they do.
CLEMENT: John teaches at the Kent School. Mark is a member of a tradecraft cell, and Maria is the chief of the red cell. And the red cell actually...
GJELTEN: Only their first names are used. It's unusual for CIA officials to talk openly about their work. At the CIA's in-house school, John shows new analysts how their thinking about issues can be affected by the mindsets they bring to their work.
JOHN: Your biases will get you things like - an example would be confirmation bias: Well, I've seen it before, so it must be happening again. Or anchoring Bias: This is my viewpoint. We've come up with that conclusion, and I think it's true, and it's not going to change.
GJELTEN: One new exercise forces CIA analysts to consider alternative explanations for developments they're following. It's the ACH exercise: analysis of competing hypotheses. Iran is developing a new missile. Maybe it's to deliver a nuclear warhead. But could there be another explanation? CIA analysts now have to spell out all the contradictory evidence. Even the bureaucratic organization of the CIA's analytic work has changed. Each department now includes what's called a tradecraft cell. Mark, who's assigned to the CIA's East Asia office, says tradecraft specialists, like himself, monitor their colleagues' work and make sure they're all using the latest analytic techniques.
MARK: We want to try and get analysts to be honest about the uncertainties they face or the gaps in their information.
GJELTEN: Not just go with a judgment they assume to be true. Another change: CIA analysts sometimes push back now on the idea that they should necessarily make a prediction. There's now what's called a red cell at the agency, with the job of explaining all the different ways the future could unfold. Maria is the current red cell chief.
MARIA: There definitely was an emphasis in years past to say: It is most likely going to go this way. It probably will happen this way. And we still have to make those calls. But now, we try to explain what factors would take it in a different direction.
GJELTEN: This exercise requires what Peter Clement calls a cultural change in CIA thinking.
CLEMENT: When I came on board in the late '70s and '80s as a young analyst, I distinctly remember managers telling me, Peter, the reason they pay you the big bucks is make an analytic call. You know, you're the expert. You've studied all this material. Make the call. You know, give me the answer.
GJELTEN: As history shows, the CIA can get in trouble when it's pressured to make the call on an important question, like, does Saddam have WMD or not? Some former CIA officials have said the Bush administration bears some responsibility for the mistaken conclusion about Saddam's weapons. White House officials, they say, including Vice President Dick Cheney, were so determined to go to war in Iraq, that they pressured the agency to find a reason for invading that country.
Now comes another big question for CIA analysts. Is Iran building a nuclear weapon? Peter Clement says the new determination to avoid another Iraq-type mistake was kept in mind when the U.S. intelligence community prepared its 2007 official estimate on Iran. That estimate said Iran had halted its nuclear weapons program four years earlier and apparently not restarted it. It's been regularly reaffirmed since then.
CLEMENT: In the case of the 2007 estimate, I think because in fact we're now so focused on the shortfalls in the Iraq WMD, we need to be very open about the nature of the evidence, how much evidence. I mean, all those factored into that estimate to say: We just haven't got what it takes to make the call that, yes, they've made a decision to go ahead with the program.
GJELTEN: We now know about the consequences of making a wrong call. The risk with these new exercises is that they could build so much caution into CIA analysis that the agency will be paralyzed, so focused on questioning its conclusions that it can't come up with answers. Professor Robert Jervis, of Columbia University, has written widely about intelligence failures and advised the CIA. He's undecided on whether the new analytic techniques will prove worthwhile.
ROBERT JERVIS: I could see them becoming a substitute for careful, informed thought, a formula that, oh, you don't need to know anything about the country or anything. You just plug these things in. So misapplied, they'd make things worse. Applied well, they will make things better. How much better? Hard to tell.
GJELTEN: What we know about intelligence failures comes only from looking back at how well the CIA anticipated some event or understood something that was otherwise unclear at the time. And it's only in the future that we will know whether the CIA's new approach to analysis has sharpened its insight. Tom Gjelten, NPR News. Transcript provided by NPR, Copyright NPR.