Charting Progress in Iraq

This topic of determing how things are going on the ground in Iraq seems to be heating up in the last few days. See for example, Sheryl Gay Stolberg's New York Times article from May 13th - See You in September, Whatever That Means where she says:

There’s just one problem. Nobody in Washington seems to agree on what progress actually means — or how, precisely, it might be measured.

While I don't agree with that premise, the article opened up the door to a ton of worthwhile commentary - much of discussing possible factors that might be useful to gauge how things are going.

Another recent contribution to this topic was David Peck's Christian Science Monitor Article on why we need yardsticks for both success and failure in Iraq. David skillfully maps his experience with coaching leaders onto the pressing leadership issues involved in our presence in Iraq. The money quote of this article for me was:

there has never been agreement on the status of the war in Iraq because there are no agreed-upon measurements.

For some perspective from a couple of years back, and maybe to help understand how much or little forward momentum we are making in talking about progress, I also suggest you take a look at this Dan Froomkin article on Nieman Watchdog from a couple of years back (May 2005): Isn’t there some way to tell if we're winning or losing in Iraq?


We've delved into this topic in previous posts (see: Most Important Iraq Data) and we plan to revisit it again soon based on these new inputs and what seems to be a growing interest in the topic.

And of course in our opinion since what gets measured gets done and what doesn't get measured doesn't get done, this particular topic is of particular importance and relevance right now.

Hat tip to Dan Froomkin for pointing out both the referenced NY Times and the CS Monitor articles in his White House Watch Blog.

Missing Metrics - What else do we need to know about how things are going in Iraq?

In looking closely at the text of Michael O'Hanlon's testimony to the Senate Foreign Relations Committee on January 10th, 2007, I noticed that he spoke of quite a few Iraqi factors on the ground that were NOT included in the list of 30 metrics reported.

A small number of the factors that showed up in the tabular data were mentioned in the logic of the text, but I was surprised by how many important factors from the text had no numerical equivalent in the table.

Here's a list of the missing factors that I found in the text with my comments in brackets.

Good year/Bad Year - <I would love to know more about what key factors Michael incorporates in his thinking in coming up with this assessment and would find it enlightening if this black and white measurement were refined and reported on say a scale from 1 to 10 or 1 to 100>

“The year 2006 was, tragically and inescapably, a bad one in Iraq.”

Sense of Progress

“The first is the failure of the 2005 election process to produce any sense of progress.”

Al-Maliki Favorabaility Rating

“In a September poll, Prime Minister al-Maliki was viewed unfavorably by 85 percent of all Sunni Arabs”

Rate of Shia attacks on Sunnis

Rate of Sunni attacks on Shia

Increasing politicization of the violence “to include many more Shia attacks on Sunnis as well as the reverse”

Iraqi state subsidies for consumer goods

“recent reductions in Iraqi state subsidies for consumer goods (which distort the economy and divert government resources)”,

The number of children being immunized.

Percentage who feel safe en route to school

Rate of private sector investment

“further slide toward chaos.”

Percent who think Economy is Poor

According to a June, 2006 poll, 59 percent call the economy poor

Percent who think security environment is poor

“and 75 percent describe the security environment as poor”.

Rate of Jobs Creation

“a massive program to create jobs”

Degree of Iraqi Leader consensus on sharing oil

“an ultimatum to Iraqi political leaders that if they fail to achieve consensus on key issues like sharing oil”,

“American support for the operation

could very soon decline”

Number of coalition forces needed to provide security.

“Coalition forces have never reached the numbers needed to provide security for the population in Iraq”

Degree of Overwork of our Soldiers and Marines

“our already overworked soldiers and Marines”.

Rate of “Success of military commanders in putting Iraqis to work

with their commander emergency response program funds”,

Degree of Support Iraqi Leaders enjoy outside their own sectarian group.

“it is clear that extremely few Iraqi leaders enjoy any real support outside of their own sectarian group.”

Degree to which Iraqi Leaders are working across sectarian lines.

“Trying to force them to work across sectarian lines must be a focus of our policy efforts, if there is to be any hope of ultimate stability in Iraq.”

The Probability that a Given Intervention will succeed

“Social scientists and military experts do not know how to assess rigorously the probabilities that such steps will succeed at this late hour in Iraq.” [Emphasis added]

Degree of Pessimism

“Overall, however, it seems fair to say that most have become quite pessimistic.”

Iraqis displaced per month

“But with 100,000 Iraqis per month being displaced from their homes, making for a total of some two million since Saddam was overthrown, ethnic cleansing is already happening.”

Bottom Line: If we want to understand what's going on in Iraq, we are going to have to work harder at bringing these missing metrics out in the open where every citizen can see them, think about them, discuss them.

What Gets Measured Gets Done

Michael O'Hanlon is a co-author (along with Jason H. Campbell) of the Brookings Iraq Index report that we have discussed in several recent posts.

In a recent Brad DeLong post

(Grasping Reality with Both Hands: Brad DeLong's Semi-Daily Journal: Matthew Yglesias and His Commenters Look at the Brookings Institution, Michael O'Hanlon, and Jeff Herf)

I came across the following quote (originally posted here as a comment to Matthew Yglesias post) and found it interesting and relevant to the mission of this site (emphasis added):

Also of note, the one guy in Brookings who was actually right about the whole mess, that is, Ivo Daalder, isn't working there anymore. As for the Hanlon's (sic) major contribution to the war of ideas, the Iraq index, the best I can say is that this is a very American idea: the idea that problems can be analyzed and solved if we only quantify them and measure them.

Posted by: Nick Kaufman on January 14, 2007 12:22 AM

My reading is that O'Hanlons' and Brookings' Iraq Index is useful exactly because

1. it works hard at identifying the key factors that give us clues to how things are really going in Iraq, and
2. it does the heavy lifting work of pulling the measurements of these factors together in a single place and making them available for others to examine and think about.

Simply quantifying and measuring key factors (while necessary) cannot be sufficient to solve the problems we face. Once we have the data, different individuals may formulate different and even contradictory analyses and that too is useful. By the way, it is clear from the Iraq Index report itself that they are only presenting some of the required data and that one would be wise to treat some of the individual measurements with due caution. So, at least for me, I don't see them saying that measurement leads automatically to solution.

The converse of the proposition that "problems can be analyzed and solved if we only quantify and measure them" is likely to be more on target. Namely something like:

If we don't identify the key factors, quantify them, measure them, make them available to others, analyze them, and discuss them, then this will severely limit our individual and collective ability to understand the problems we face.

Under this new framing (effective measurement leads to greater understanding), we also note that the identification of potential solutions will be sharply limited, much more difficult, and much more a matter of guesswork when such measurement is absent.

One other benefit of a measurement-aided approach is that when a given solution (an intervention) is initiated, the next measurements of these same key factors will provide the essential grist for determining whether that intervention is delivering on its promise.

In other words we might want to designate the idea: what gets measured, gets done as the bugle call order of the day, knowing that measurement, sharing, and collaboration can help us at each step along the way from problem definition to proof that our solutions have actually worked.

In future posts, we will make an attempt to build further on the work started in the Iraq Index and on the measurement work we can cull from other Iraq sources so as to add some substance to the theoretical statements of this post.

In my view, if we are going to find and implement a solution to the deteriorating Iraq situation, we need more quantification, measurement, sharing, analysis, and collaboration and that extensions to and improvements on the Iraq Index can help pave the way.