Thursday, June 14, 2007

Ten Trend Charts from the DoD Quarterly Report

The Dept of Defense has just published its mandated quarterly report to Congress on the situation in Iraq - Measuring Stability and Security in Iraq, June 2007 (51 pages). As they state in the Executive Summary, the "report includes specific performance indicators and measures of progress toward political, economic, and security stability in Iraq"

Included as part of the report are 10 static trend graphs which I have reproduced in this post. You can click on each for a full size image. If you would like all 10 JPG images so you can look at full size with your favorite slide viewer, you can download this zip file.

Most of the important performance indicators mentioned in the report did not rate their own static trend graph chart. As we did with the UN document and our scan of recent news articles, we plan to comb through the document and come up with a list of indicators that complement the ones we have already highlighted so as to get a sense of the breadth of important indicators that are already on our radar screen.

Chart 01 - Oil Production - too short a time span. Also missing data for May 2007.

Chart 02 Electricity - Missing data for April and May 2007.

Chart 03 - Sectarian Murders and Incidents . Missing data from May 2007. Weekly data might be important to look at.

Chart 04 - Weapons Caches Found - This is looking promising. Access to weekly and daily data could be even more revealing of ongoing trends. Missing data for May 2007.

Chart 05 - Average Weekly Attacks - Missing last 5 weeks of data. This is the source for the charts that appeared in WaPo and NYT as reported earlier today.

Chart 06 - Average Daily Casualties - Missing last 5 weeks. Bar chart format with three trends makes reading trends for Iraqi Security Forces and Coalition more difficult.

Chart 07 - Hotline Tips - Interesting new performance metric. Looks positive. Missing past month of data. Weekly or daily data reporting could be even more revealing.

Chart 08 - Confidence in Iraqi Government to Improve the Situation. This looks surprisingly to me for its stability over time. I will need to read the text about this in detail. Missing most recent month's data.

Chart 09 - Assessed Capabilities of Police - Difficult to interpret trends when only two data points per item. Stacked bars make interpretation more difficult and time consuming. This is an interesting new metric and it could benefit from higher resolution, near real time reporting.

Chart 10 - Ministry of Defense Forces Assessed Capabilities. This chart also presents some important data but suffers from same failings as previous chart - only 2 data points per indicator, stacked bars are hard to read.

10 Static charts covering a range of important indicators is more than almost anyone else except Michael O'Hanlon produces when it comes to highlighting the important trends evolving in Iraq.

So that's definite plus. Someone who spends a few minutes can examine all 10 of these (especially if they can scroll in slide show mode on full size images using this zip file).

On the minus side, the missing last month or last 5 weeks of data is a serious drawback. Many of the charts can be improved on and made more readily useful to ordinary viewers by applying some trend chart best practices principles. These include
  • use of full time range
  • use of near real time reporting for the most recent interval
  • selection of best resolution or choice of several different reporting intervals (day, week, month)
  • displaying each important indicator separately with appropriate Y axis scale
  • having sufficient data points for trends to be discernible - 2 is way too few
And worst of all, most of the important indicators have no charts at all and are only discussed in text or in tables showing the current values.

The indicators in this report are important, but this method of presentation falls short of ideal and short of what is needed and what indeed is possible. With the high number of factors discussed in this report, the only method that will actually make this kind of trend data usable to its intended primary audience (Congress) and to the secondary audiences including the media and ordinary interested citizens would be to deliver this data via a 21st century Trend Visualization Appliance.

No comments: