How Petraeus changed the U.S. military

Iraq's government itself was fueling the violence because the Iraqi Ministry of the Interior was home to a number of Shia death squads.

Meanwhile, al Qaeda's brutal Iraqi affiliate was recruiting hundreds of suicide attackers from around the Middle East who went on to kill thousands in Iraq.

As a result of this mayhem, some 5 million Iraqis -- around a fifth of the population fled the country or went into internal exile.

After a 2003 tour in Iraq where Petraeus effectively pacified the area in and around Mosul in the north and a second tour where he had a less successful stint trying to reform the disbanded Iraqi army, he was assigned to run the U.S. Army's Combined Arms Center at Fort Leavenworth, Kansas, in 2006.

This was seen as something of a backwater for the rising star general who Newsweek had anointed with a cover story two years earlier headlined "Can This Man Save Iraq?"

But Petraeus saw his tour in Kansas as an opportunity to revamp the Army's counterinsurgency doctrine, something the military hadn't put any real thought into since the Vietnam War. And now it was a complex stew of Sunni and Shia insurgent groups that the U.S. military was fighting in Iraq.

So Petraeus turned his time in Kansas into a year-long exercise to rewrite the strategy and tactics of the Iraq War.

To help him, Petraeus recruited Iraq War veteran John Nagl, a Rhodes Scholar with a doctorate from Oxford who in 2002 published the book "Learning to Eat Soup with a Knife: Counterinsurgency Lessons from Malaya and Vietnam."

In November 2005, Petraeus gave a lunchtime speech at a counterinsurgency conference in Washington. Nagl recalls that Petraeus, his former history professor at West Point, "announced that he was going to write a counterinsurgency manual, and he announced that I was going to be the lead pen, which was the first time I'd heard of it."

Nagl assumed the role of managing editor of the manual and Petraeus recruited Conrad Crane, a military historian and a former West Point classmate, to be the lead writer. But there was no doubt who was in charge. Nagl recalls that Petraeus "was the driver, he was the vision, he was the copy editor, he read the whole thing twice, he turned around chapters in 24 hours with extensive edits and comments."

The writings of the French soldier-intellectual David Galula were quite influential on the group working on the manual. Galula had fought in Indochina and Algeria in the 1950s as an officer in the French army as it was attempting to stamp out nationalist insurgencies in its colonial possessions.

Around a decade later Galula published "Counterinsurgency Warfare: Theory and Practice," which distilled the lessons of fighting and observing insurgencies in the Middle East and Southeast Asia.

Galula laid down a general principle that is recognized as the core of a successful counterinsurgency strategy: "The population becomes the objective for the counterinsurgent as it was for his enemy."

This meant that seizing territory became far less important than it was in a conventional war; ensuring that people felt secure enough so they were not forced to have to side with the insurgents and, eventually, even felt secure enough to provide intelligence about them, became the prize.

Once a first draft of the counterinsurgency manual was completed, Petraeus and the lead writer, Crane, decided to convene a group of outside experts to critique it. Crane recalls, "We had a vetting conference to go over the doctrine, and I agreed with the general that we would do it, and I said, 'Yeah, let's bring in 30 smart people to talk about it'; he brought in 150. It was quite a three-ring circus out at Fort Leavenworth."

Over the course of two days officials from the CIA, State Department and leading academics and journalists such as Eliot Cohen, James Fallows and George Packer were instructed to give their critiques, which generated hundreds of pages of new ideas.

The army and Marines published the final version of the Counterinsurgency Field Manual in December 2006.

The doctrines in this new manual deeply informed how the U.S. military would fight the wars in Iraq and Afghanistan.

The manual pointed to such unsuccessful counterinsurgency practices as overemphasizing killing and capturing the enemy rather than making conditions secure for the populace, conducting large-scale operations as the norm and concentrating military forces in large bases for protection.

This was, in fact, a good description of what the U.S. military had been doing in Iraq for the past three years of the conflict and an explanation of why it was now losing the war.

Successful practices, the new manual emphasized, focused on meeting the needs and ensuring the security of the population.

In a section titled "Paradoxes" the manual made a number of recommendations that were hardly typical of prevailing U.S. military doctrine: "Sometimes doing nothing is the best reaction" and "the host nation doing something tolerably is normally better than U.S. doing it well."

Emma Sky, an Oxford graduate and Arabic speaker who became Petraeus' political adviser in Iraq, recalled that, "The biggest mindset change was for the U.S. to look at Iraqis as not the enemy, but to look at the Iraqis as people who needed protecting."

In February 2007 Petraeus was appointed by President George W. Bush as the new U.S. commander in Iraq just as 30,000 troops of "the surge" that Bush had recently ordered began arriving in Iraq.

Shortly after Petraeus' arrival he took a tour of Baghdad neighborhoods he knew from his past deployments. Petraeus later told me, "I just couldn't believe it...here's literally tumbleweed rolling down the street of what I remembered as a very prosperous, upper-middle-class, former military officers' neighborhood in northwest Baghdad. It was just. . .Wow!"

There were now well over 200 car bombings and suicide attacks every month in Iraq. Six months earlier there were around a quarter of that number. Iraq was simultaneously exploding and imploding.

Petraeus' new counterinsurgency approach got American soldiers out of their massive bases in Iraq and into Iraqi neighborhoods.

Petraeus explained this "population-centric" strategy in a letter he sent to all of the soldiers he commanded. "You can't commute to this fight...Living among the people is essential to securing them and defeating the insurgents...patrol on foot and engage the population. Situational awareness can only be guaranteed by interacting with people face-to-face, not separated by ballistic glass."

Emma Sky says that Petraeus played another key role, which was buying time in Washington for the new strategy to work by being the public face and advocate of the new approach in Iraq. "Without his strategic communications, without people's belief in Petraeus, we would never have got the time."

The greatest test of whether the political will existed to continue with the ramped-up Iraq effort were the congressional hearings held on the sixth anniversary of 9/11.

On September 11, 2007, Petraeus and Ryan Crocker, the veteran diplomat who was U.S. ambassador to Iraq, were grilled by both the Senate Foreign Relations and Armed Services committees, on which happened to sit five senators all seriously vying for the presidency-- Joe Biden, Christopher Dodd, Barack Obama, John McCain and Hillary Clinton--- one of whom would become president of the United States in just over a year.

Petraeus recalls that the hearing "Was just charged beyond belief. I mean, you could just feel the spotlight of the world on you. It was carried live in Baghdad."

Petraeus and Crocker gamely tried to present a picture of progress in Iraq, but the Democrats were having none of it. Clinton interjected at one point: "You have been made the de facto spokesman for what many of us believe to be a failed policy. Despite what I view as your rather extraordinary efforts in your testimony...I think that the reports that you provide to us really require the willing suspension of disbelief."

This is Washington-speak for you are either wrong or lying.

The day before, the duo had also testified before a joint hearing of the House Armed Services and Foreign Affairs committees. Petraeus knew it was going to be a rough day when he received a heads up that the New York Times was running a full-page ad about him, paid for by the left- wing advocacy group MoveOn.org.

Under a banner headline GENERAL PETRAEUS OR GENERAL BETRAY US?" the general was accused of "Cooking the books for the White House." The ad copy went on to assert, "Every independent report on the ground situation in Iraq shows that the surge strategy has failed...Most importantly, General Petraeus will not admit what everyone knows: Iraq is mired in an unwinnable religious civil war."

Despite these criticisms, Petraeus' cautiously worded congressional testimony about the turnaround that he was beginning to see in Iraq proved to be accurate.

The violence in Iraq, which was peaking in almost every category in the first months of 2007, steadily dropped after that. That decline was true across the board, including attacks by insurgents, civilian deaths, U.S. soldiers killed, Iraq security forces killed, car-bomb attacks and IED explosions.

In December 2006 the U.S. military map of "ethno-sectarian" violence in Baghdad was colored mostly yellow, orange and red, indicating medium to intense violence. The same map two years later was mostly colored green, indicating that the sectarian violence in Baghdad had largely subsided.

Of course, not all of this was due to the generalship of Petraeus. Other important factors such as the tribal revolt against al Qaeda's Iraqi affiliate worked in Petraeus' favor.

The Sunni Awakening movement had begun in 2006 before Petraeus arrived in Iraq, but he and his top commanders deftly managed it.

The tribal fighters of the Awakening movement ended up on the American payroll in the "Sons of Iraq" program, which by the spring of 2009 had grown to around 100,000 men. Many of those men had once been shooting at Americans; now they were shooting at al Qaeda.

Iraq today remains a dangerous place, but it is not in the grip of a civil war, and political differences are more likely to be decided by parliamentary maneuvers than by violence.

Certainly, Petraeus can claim a large share in the achievement of that outcome.

Petraeus was later tapped to try to turn around another war that wasn't going well, this time it was the war in Afghanistan and the call came from President Obama in 2010.

The jury is still out on what level of success Petraeus achieved during his tenure as the commander of U.S. and other NATO troops in Afghanistan.

As a result of the operations resulting from the "surge" of 30,000 U.S. troops into Afghanistan authorized by Obama and led in its latter stages by Petraeus, longtime Taliban havens in the southern Afghan provinces of Helmand and Kandahar have now been eliminated.

Special operations forces also have decimated the ranks of mid-level Taliban commanders to such a degree that last year the average age of Taliban commanders dropped from 35 to 25, according to U.S. military sources.

That said, the increased military pressure on the Taliban did not bring them to the negotiating table in any meaningful way as had once been hoped. And the Taliban continue to control many rural areas of the country.

Arriving at any judgment of Petraeus' record at the CIA is complicated by the fact that he was there for only around a year and, of course, most of the activities of the CIA are secret.

Some light will be shed on the agency's recent activity when the acting director of the CIA, Michael Morell, testifies next week before the Senate Intelligence Committee about the attack on the Benghazi consulate in September that killed four Americans, including the U.S. ambassador to Libya and two men who were recently revealed to be CIA employees.

It was Petraeus who was supposed to be delivering that testimony.