Many Eyes - A New Doorway for Graphical Collaboration

From our previous post we noted that:

We believe in many eyes, many hands, many voices working together on the path of mutual discovery (rather than the idea of a single expert figuring it all out and presenting it to us in final form).

To help foster exactly this kind of process, I am excited to note a new web site, Many Eyes that is now available for exactly this purpose. As they describe it on their home page: Many Eyes is a place to: view your data, ask questions, share your discoveries, and harness the collective intelligence of the net for insight and analysis.

Once you register, you can upload interesting data sets and create visualizations that you feel are meaningful. Others can view your visualization and comment. They can also create their own variant visualizations using the data you uploaded and publish these for others to view and comment. Many different visualization formats including time series graphs, bar charts, scatter charts as well as some very creative new chart types are available in an interactive fashion. Creating new charts is similar to the kinds of interactions you might experience with one of the many stock quote charting web sites such as the java charts on prophet.net

Even better, once a data set is uploaded by anyone, it is possible someone else to download that data set for further analysis on their system using their favorite tools. For example, you might want to take a set of raw time line data series and compute some additional columns of data as we did in one of our previous Iraq posts. You could then take your new data set with added columns and upload it to Many Eyes for others to work with directly.

To test out these capabilities, I uploaded the O'Hanlon indicators that we have recently posted about. You can find a visualization that shows a single indicator that I created at: Percent of Coalition Troops Appears to be Shrinking. Once you get to that page, you can select other indicators from the list box on the left side to view the other trends at work. You can also download the entire table to your computer or leave a comment or create at publish your own chart. [Unfortunately, at present the long indicator names cannot be fully seen during the selectino process. So if you want to know what they are, you can find out by clicking on the table like icon element to look at and scroll through the top row of the data set where the indicator names appear]

I see Many Eyes (what a great name) as a big step forward in thinking about, discussing, understanding, and collaborating about the most important factors in our lives. I expect I will be uploading additional data sets and creating new visualizations there in the future. I would love to hear what you think about his newly available capability.

Change Over Time: First Principles Revisted

I am enjoying reading Edward Tufte's new book: Beautiful Evidence. It's the fourth in a series of wonderful Tufte volumes that exemplify the power of visual display and it certainly lives up to its title. Beautiful Evidence, like its predecessors, is simply a beautiful and inspiring work - blending art and science and philosophy and the linkages between numeric data and their visual representations into a unifying whole.

Tufte is powerfully interested in the underlying principles. For example, here's a brief passage from his introduction to Beautiful Evidence:

Evidence is evidence, whether words, numbers, images, diagrams, still or moving. The intellectual tasks remain constant regardless of the mode of evidence: to understand and to reason about the materials at hand, and to appraise their quality, relevance, and integrity.

... The point of evidence displays is to assist the thinking of the producer and consumer alike. Evidence presentations should be created in accord with the common analytical tasks at hand, which usually involve understanding causality, making multivariate comparisons, examining relevant evidence, and assessing the credibility of evidence and conclusions.

This blog is especially concerned with a particular kind of evidence: multi-dimensional time series. We are surely interested in the underlying intellectual and analytical tasks including understanding causality, making comparisons, and cross checking conclusions.

"Beauty" on the other hand, plays a much smaller role in our efforts. It is always a delight when it makes one of its rare appearances for us, but is not a primary or even a secondary goal. In fact, many of the approaches we recommend in order to meet our primary mission by their very nature tend to lead away from beauty.

Reading Tufte always encourages me to take a step back and think about the first principles that are the foundation of the work of this blog. I want to think through and examine all that his approach has in common with the work shown in this blog and all the ways we might differ.

Here are some further thoughts about the first principles that are the basis of my continuing work with the visualization of time series data (with special emphasis today on the principles that might mitigate against "beauty").

1. Time is of the essence - both analyst time and viewer time. While making comparisons and searching for understanding and seeking out properties of the data that have never before been seen, we will give preference to those approaches that help speed up the work of the analyst, even if some or a lot of beauty must be sacrificed.
2. Following Covey, we seek first to understand the data and then to present our findings in ways that make it easiest and quickest for others to understand.
   
3. Sharing the data in a readily accessible and readily reusable form along with the presentation of any findings is perhaps the core principle of our work. It's the one that allows for collaboration and for a rapid and multi-step search for meaning.
4. We aim to make both the data and whatever graphical representations we create accessible to experts and non-experts alike.
5. We care deeply about practical tools that allow ordinary citizens to participate in this work of comparison and understanding.
6. We believe in many eyes, many hands, many voices working together on the path of mutual discovery (rather than the idea of a single expert figuring it all out and presenting it to us in final form).
7. We want to apply this approach to the most important, practical, everyday problems that we face. We want this approach to help dramatically improve decision making. And we want this to happen in near real time.
   

For example, in hindsight, the famous Napoleon' March graphic that Tufte has written extensively about is certainly a beautiful and powerful display that promotes understanding of what happened to Napoleon's army on the way to Russia and back. But of course, it was not available in time to help Napoleon make better decisions. What we are interested in is figuring out ways in which such powerful understanding might be made available soon enough (in near real time) to actually be of use to the decisions makers and to ordinary citizens as complicated and important events are unfolding. Our recent series of posts about the key indicators at work that help us understand how things are going in Iraq is a step in that direction.

To sum up, in our approach, if we have to chose, we will chose timeliness, usefulness, ready re-usability, accessibility to the ordinary citizen, tools that are available here and now, and the ability to collaborate over beauty every time.

Edward Tufte's "Beautiful Evidence", Sparklines & CSVPNG

Edward Tufte is out with a new and inspirational book: Beautiful Evidence which includes a full chapter on the idea of sparklines - small word like graphic elements that can be integrated fully with text and that allow for massive amounts of statistical data to be presented in a compact format that encourages comparison between elements.

Not many software tools are yet available to help create the sparkline graphical elements. One exception is the T4 & Friends utility: CSVPNG that we have talked about in previous posts. CSVPNG has some introduced some new basic capabilities that can let you experiment with creating your own sparklines without having to write all the code yourself. The latest version of CSVPNG can be downloaded from the Trends That Matter download page

Apprenticed Investor: Know Thyself

Apprenticed Investor: Know Thyself

Hat tip to Brad Delong for pointing out and reminding me of this excellent article by Barry Ritholtz in his Apprenticed Investor. While Barry is directing his comments at our all too human behaviors when we are investing in stocks, his discussion of pitfalls to watch out for is highly relevant to the analysis of any set of complex data (e.g. the 142 Iraq factors we have so far identified).

Take a look. It's a quick read and well worth the effort

Top 12 factors (one citizen's view)

Here's an even shorter list of what I consider to be the top 12 factors to watch right now.

This does not mean that we should only look at this subset. In my opinion, every item on the previous post's list of 34 is of high importance and worthy of being measured, tracked, and reported consistently in near real time.

In fact, while perhaps of not as high importance (and opinions will of course vary on this), almost every item on the list of 142 deserves regular attention - measurement, tracking, and reporting - so that the full picture can be brought into view.

From the Brookings Iraq Index Report

• Iraqi Deaths from All Violent Causes

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

• Crude oil production

5 trend graphs drawn from Iraq Coalition Casualties

• US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
• US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006

from Michael O'Hanlon's testimony. 30 factors

• Daily Average of Inter-ethnic Attacks
• Avg Hours/Day of Power Baghdad (prewar 20)
• Unemployment Rate (%)

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

• Rate of Jobs Creation

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

• Iraqis Kidnapped in Baghdad (table page 15)
• Number of total daily patrols (chart page 23)
• People with potable water (table page 42)
• Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

What does your short list look like?

First cut at identifying most important factors

Here's one private citizen's view on which of the 142 Iraq factors identified so far are most vital to understanding what is happening on the ground in Iraq. These are the ones that I believed need to be measured and reported on regularly so we can see the trends at work. I have picked 34 out of 142 (or about 24% of the total) so this is something like an 80/20 selection.

What does your list look like?

It would be excellent to see Michael O'Hanlon's list, or Anthony Cordesman's list, or Secretary Gate's list, or Secretary Rice's list, or Lt. Gen. Petraeus' list of the top factors they will be considering as we go forward.

From the Brookings Iraq Index Report

• Average Weekly Attacks by Time Period – 01 Jan 2004 – 11 Aug 2005
• Iraqi Deaths from All Violent Causes – Jan 06- Oct 06
• Enemy-Initiated Attacks Against the Coalition and its Partners May 2003 –Aug 2006

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

• Crude oil production for the past 10 weeks.
• Electricity production compared to electricity demand since January 2004.

5 trend graphs drawn from Iraq Coalition Casualties

• US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
• US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006

from Michael O'Hanlon's testimony. 30 factors

• Estimated Strength of Insurgency
• Estimated Strength of Shia Militias
• Daily Average of Inter-ethnic Attacks
• Iraqi Security Forces who are Politically Dependable
• Percent of Household Fuel Needs Available
• Avg Hours/Day of Power Baghdad (prewar 20)
• Unemployment Rate (%)

New indicators calculated from combinations of O'Hanlon's metrics

• total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq.
• percent of coalition troops that are not US troops
• Percent of US troops killed by IEDs
• Year over year increase in Iraqi refugees

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

• Rate of Shia attacks on Sunnis
• Rate of Sunni attacks on Shia
• The number of children being immunized.
• Rate of Jobs Creation
• Degree of Overwork of our Soldiers and Marines

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

• Civilian deaths in Baghdad (page 12 table)
• Iraqis Kidnapped in Baghdad (table page 15)
• Iraqi prison population in US Custody (table page 16)
• Iraqi prison population in Iraqi Custody (table page 16)
• Number of total daily patrols (chart page 23)
• Gasoline/Benzene production (table page 29)
• Total cost of US operations (table page 37)
• People with potable water (table page 42)
• People with sewerage system coverage (table page 42)
• High School enrollment percentage (table page 43)
• Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Combined Listing of Iraq Factors

Here's the combined list of 142 factors we have identified so far based on this series of recent posts. I have left off the links to the various sources. You can find them these links in the previous posts.

Which ones do you think are most vital to pay attention to on a regular, near-real-time basis in order to better understand what's happening on the ground?
How do you rank each on a scale of 1 to 10?
How often do you think each factor must be measured and reported?
Are there any ones you think can be dropped?

From the Brookings Iraq Index Report

1. Multiple Fatality Bombings – May 03 – Jan 07
2. Killed and Wounded in Multiple Fatality Bombings – May 03 – Jan 07
3. Average Weekly Attacks by Time Period – 01 Jan 2004 – 11 Aug 2005
4. Improvised Explosive Devices (IED’s) Detonated and Disarmed - Jan 2006 –June 2006
5. US Troops Wounded in Action March - March 2003 – Jan 2007
6. Iraqi Military and Police Killed Monthly – Apr 03 – Jan 07
7. Iraqi Deaths from All Violet Causes – Jan 06- Oct 06
8. Number of Daily Attacks by Insurgents and Militias - June 03 – Nov 06
9. Enemy-Initiated Attacks Against the Coalition and its Partners May 2003 –Aug 2006

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

1. Crude oil production for the past 10 weeks.
2. Crude oil export since February 2006
3. Electricity production compared to electricity demand since January 2004.
4. The strength of the local Iraqi currency compared to the dollar
   

GlobalSecurity.org

1. US Ground troops in Iraq from March 2004 through December 2006
2. Average daily casualties - Iraqi and Coalition from 01 April 2004 through 12 May 2006
3. Average weekly attacks by time period 01 April 2004 through 12 May 2006

5 trend graphs drawn from Iraq Coalition Casualties

1. US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
2. US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006
3. IED Fatalities per month - July 2003 - January 2007
4. Iraqi Police/Military fatalities - January 2005 - December 2006
5. Iraqi Civilian fatalities - January 2005 - December 2006

Composite Readily Reusable (RR) example based on Tabular data from GlobalSecurity.org

1. the monthly ratio of wounded to dead
2. the monthly change in troop level in Iraq
3. the monthly change in troop level in Iraq as a percent of the previous month's level
4. the three month trailing moving average of fatalities
5. the six month trailing moving average of fatalities
6. the 9 month trailing moving average of fatalities
7. the monthly total of wounded and dead combined

from Michael O'Hanlon's testimony. 30 factors

1. US Troops in Iraq (thousands)
2. Other Foreign Troops in Iraq (thousands)
3. US Troops Killed
4. US Troops Killed by IEDs
5. US Troops Wounded
6. Iraqi Army/Police Fatalities
7. Iraqi Civilian Fatalities
8. Multiple Fatality Bombings (for the month in question)
9. Estimated Strength of Insurgency
10. Estimated Strength of Shia Militias
11. Daily Average of Inter-ethnic Attacks
12. Estimated Number of Foreign Fighters
13. Number of Daily Attacks by Insurgents/Militias
14. Attacks on Oil/Gas Assets
15. Iraqi Internally Displaced Since April 2003
16. Iraqi Refugees Since April 2003 (Total)
17. Iraqi Physicians Murdered or Kidnapped
18. Iraqi Physicians who have Fled Iraq
19. Iraqi Security Forces who are Technically Proficient
20. Iraqi Security Forces who are Politically Dependable
21. Oil Production (Millions of Barrels/Day prewar 2.5)
22. Percent of Household Fuel Needs Available
23. Electricity Production (in megawatts prewar 4000)
24. Avg Hours/Day of Power Baghdad (prewar 20)
25. Unemployment Rate (%)
26. Per Capita GDP (Real Dollars prewar $900)
27. Number of Trained Judges
28. Telephone Subscribers (prewar 800000)
29. Independent Media Companies (prewar 0)
30. Iraqi Optimism (% who think things going in right direction)

New indicators calculated from combinations of O'Hanlon's metrics

1. total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq.
2. percent of coalition troops that are not US troops
3. Percent of US troops killed by IEDs
4. Year over year increase in Iraqi refugees

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

1. Good year/Bad Year (e.g. subjective on scale of 1 to 10)
   
2. Sense of Progress (e.g. subjective on scale of 1 to 10)
3. Al-Maliki Favorability Rating
4. Al-Maliki Favorability Rating for all Sunni Arabs
5. Rate of Shia attacks on Sunnis
6. Rate of Sunni attacks on Shia
7. Iraqi state subsidies for consumer goods
8. The number of children being immunized.
9. Percentage who feel safe en route to school
10. Rate of private sector investment
11. Slide toward chaos (e.g. subjective on scale of 1 to 10)
12. Percent who think Economy is Poor
13. Percent who think security environment is poor
14. Rate of Jobs Creation
15. Degree of Iraqi Leader consensus on sharing oil
16. American support for the operation
17. Number of coalition forces needed to provide security.
18. Degree of Overwork of our Soldiers and Marines
19. Rate of “Success of military commanders in putting Iraqis to work with their commander emergency response program funds”
20. Degree of Support Iraqi Leaders enjoy outside their own sectarian group.
21. Degree to which Iraqi Leaders are working across sectarian lines.
22. The Probability that a Given Intervention will succeed (e.g the most recent set of changes in strategy and tactics)
    
23. Degree of Pessimism
    
24. Iraqis displaced per month
25. Degree of Ethnic Cleansing

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

1. Percentage of US troop fatalities caused by car bombs (page 3 table)
2. Iraqi civilians killed by US troops (page 9 table)
3. Police deaths in Baghdad (page 12 table)
4. Civilian deaths in Baghdad (page 12 table)
5. Kellogg, Brown and Root Employees in Iraq (table page 14)
6. Combat arms troop strength (table page 14)
7. Combat support troop strength (table page 14)
8. Combat service support troop strength (table page 14)
9. Dept of Defense Civilians in Iraq (table page 14)
10. Total Civilian Personnel in Iraq & Kuwait (table page 14)
11. Iraqis Kidnapped Nationwide (table page 15)
12. Iraqis Kidnapped in Baghdad (table page 15)
13. Iraqi prison population in US Custody (table page 16)
14. Iraqi prison population in Iraqi Custody (table page 16)
15. Active duty troop strength in Iraq (table page 20)
16. Reserve & National Guard troop strength in Iraq (table page 20)
17. Total US Army authorized strength - Activity duty (table page 20)
18. Total US Army authorized strength - National Guard (table page 20)
19. Total US Army authorized strength - Reserve (table page 20)
20. Total US Army authorized strength - Combined Active, National Guard & Reserve (table page 20)
21. Percentage of IEDs found and disarmed (table page 23)
22. Number of daily Iraqi Patrols (chart page 23)
23. Number of daily US Patrols (chart page 23)
24. Number of daily Joint Iraqi-US Patrols (chart page 23)
25. Number of total daily patrols (chart page 23)
26. New passports issued (table page 24)
27. Percent of professional class that has left since 2003 (table page 24)
28. Iraqi refugees heading to Syria per day (table page 24)
29. Iraqi refugees heading to Jordan per day (table page 24)
30. US Air Strikes (Iraq only) (table page 26)
31. C-130 Sorties flown (Iraq and Afghanistan combined) (table page 26)
32. Diesel production (table page 29)
33. Kerosene production (table page 29)
34. Gasoline/Benzene production (table page 29)
35. LPG production (table page 29)
36. Non oil related GDP (table page 33)
37. Total cost of US operations (table page 37)
38. Cost of US defense activities (table page 37)
39. Cost of Veteran's programs (table page 37)
40. Inflation rate (table page 39)
41. Cost of imported gasoline (table page 39)
42. Domestic price of gasoline (table page 39)
43. Relative amount of car traffic (table page 40)
44. Typical waiting time for gasoline (table page 40)
45. Number of registered cars (table page 40)
46. Internet subscribers (table page 41)
47. Tons of mail sent by Iraqis (table page 41)
48. People with potable water (table page 42)
49. People with sewerage system coverage (table page 42)
50. Water treatment capacity (table page 42)
51. Annual Iraqi Medical School Graduates (table page 42)
52. Primary school enrollment percentage (table page 43)
53. High School enrollment percentage (table page 43)
54. Percentage of Iraqis who approve of attacks on US led forces (table page 44)
55. Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Does reading through this entire list inspire your creativity to identify other missing metrics or other new calculations that might be made by combining this original list?

The Most Important Factors

Ooops. One critique I left off in the previous post is that to the best of my knowledge, none of the sources that I have consulted so far has laid out what they consider to be the most important of all the factors that they mention.

It is clear that huge amounts of data gathering, analysis, and reporting go into creating many of these reports which end up mentioning 50 or more factors.

From each of these sources, it would be most helpful to see a short list of the 20% of those factors that are considered most important by each analyst or group of analysts reporting on the subject.

I will take a cut myself at this particular challenge in the next post to show you what I mean.

Are we there yet?

How well does the current set of Iraq trend charts & tables serve our needs?

Answer: I think we are a little better off than we were 10 days ago when I began this series of posts on Iraq, but we certainly are not where we need to be. We can look at the charts that are currently available from multiple sources much more quickly now than we could before by skimming these two pdf files: Combine-Iraq-Trends and vizualizing-trends-ohanlon-testimony.pdf. But each of these documents has it's own and weaknesses as discussed below. And there are a growing number of important missing metrics that as yet have no publicly available trend chart or trend table.

What are the weaknesses of the Combine-Iraq-Trends document?

Answer: This combined report consisting of trend charts drawn from 4 different sources covers a lot of important ground and shares the trend behavior of many factors that appear quite important. On the downside, the big weaknesses of this set of trend charts are numerous:

1. There is a wide variation in formats - This reduces overall understandability and slows down the process of viewing the entire set of charts. Each chart must be examined individually with careful checks to make sure that the range of the X and Y axes are are clear and to locate the legend information that tells you what kind of factor you are looking at. This is in clear violation of the Edward Tufte's idea of small multiples that we discussed in a previous post. Applying small multiple thinking to a set of trend charts means that for the viewer, once understanding is reached regarding how a single chart in a series is laid out and what graphing conventions have been used, succeeding charts can be grasped with much reduced time and effort.
   
2. Time frames of the individual charts are widely different. Some cover the period from the start of the War in 2003 right up to the present moment. Others, cover much shorter periods. For example, the IED chart covers only from January 2006-June 2006. Some charts show monthly data, others show weekly data. This is another violation of the small multiple idea and the consequence for the viewer is that it is more difficult and more time-consuming to make any comparisons between different factors.
   
3. Stale Data: Violation of the "Near Real Time" Principle In many charts such as the IED chart mentioned above, the most recent sample is many, many months ago. If something is rated as an important factor, then it won't do us any good unless we measure and report on it often and that we make the most recent sample data available as quickly as possible. A 7 month old data point of a key trend factor can substantially frustrate our ability to understand what's going on, to determine whether interventions we have taken are working, or to decide what to do next.
4. Overly short time frames. Charts such as the State Dept's Crude Oil Production chart cover an amazingly short time frame - in this case from October 30th, 2006 to January 7th, 2007. While this chart is both interesting, useful, and clear, the viewer is seriously short-changed by not being able to look at what has happened to this key trend factor for the entire period since the war began.
   
5. Unavailability of the Underlying Data. In many cases with these charts, the underlying trend data is unavailable for further review or analysis. This means that the viewer of the chart is sharply impeded from discovering different angles to examine the data (e.g. by combining several factors together in a single chart, or calculating new previously invisible factors as a combination of an original set of factors.) If the original analyst who selected the chart for presentation did not create one chart for each important view, then without the data, these new views are simply unattainable. And of course, it's never possible for the original analyst to provide all the views.
   
6. Data that is provided is Not Readily Reusable (RR). In the relatively small number of cases where the raw trend data is provided, it is painful and time-consuming to transform these data into an RR format. This slows down the effort of understanding and because of the time involved, only a small percentage of potentially interested parties ever actually make these transformations.
7. Missing Metrics, Quality of Metrics. Many of these sources mention other metrics specifically (presumably because they are considered important) but do not provide trend tables or charts. And all metrics, those that are charted and those that are not, must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in

The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”

What are the weaknesses of the Combine-Iraq-Trends document?

Answer:

1. Stale Data: Violation of the "Near Real Time" Principle - the most recent data point in all of these charts is November 2006 so the trend series displayed is not as up to date (on January 10th when this data was presented) as one might wish.
   
2. Overly long time frame between samples. Way too few samples. Having a total four samples, one year apart creates a trend table that is quite readable on an 8 by 11 sheet of paper. The price is dramatic loss of vital detail and can lead to seriously misleading conclusions.
3. Data that was originally provided was Not Readily Reusable (RR). We were able to work our way around this barrier and create trend charts from the data, but it was costly in time and will serve as an impediment to further analysis of the data for most people most of the time.
4. Missing Metrics, As we noted in an earlier post, there were many Iraq trend factors that Michael O'Hanlon considered important enough to include is his verbal testimony and in the logic of thinking through what those factors meant, but which did not appear in the attached table of 30 factors. Our view is that if a metric is important enough to mention and use for logical argument, it is imperative that the details of how that factor varies over time are provided for further review and analysis by interested parties.
   
5. Quality of Metrics. As for the case of evaluating the Combine-Iraq-Trends document, all metrics, those that are in the table (and therefore charted) and those that are in the text must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in:

   The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”How
   

What next steps could help things along and help us move closer to our goal of helping both ordinary citizens and decision makers gain a better understanding of the trends at work on the ground in Iraq?

Answer: We've made some progress, but in many ways we have only just begun. Below, I outline some steps I am planning to take with the goal of taking this up to the next level. If you can assist in any way, I would be pleased to hear from you.

1. Continue to harvest and grow our list of missing metrics and invisible indicators, for example by close examination of Dept of Defense quarterly report or the many writings of Anthony Cordesman on the subject.
2. Create a single composite list of all the factors for which we do not yet have trend data or trend charts but which we think would further aid our understanding if they were to be made available.
3. Make inquiries to the original sources to see if the raw trend data for the key factors that have been identified can be made available for further use, ideally in a readily-reusable form to begin with.
   

Further Sources of Vital Iraq Trend Data

Thanks to a hat tip from a reader, here are a few more sources of vital online information about Iraq trend data that have recently come to my attention and that I find most relevant to the ongoing discussion.

First, there is the Dept. of Defense's quarterly report to Congress about the status in Iraq - Measuring Security and Stability in Iraq. The most recent report is dated November 2006. The report includes 9 trend charts and 1 trend table. Some of these charts are variants of trend charts we have already seen from our look at the Brookings Iraq Index , Michael O'Hanlon's testimony, GlobalSecurity and the Iraq Casualty Coalition . Others, however, are new and complementary to the charts we have seen so far. For example, there is a single chart combining three separate trends (average daily casualties for Iraqi Civilians, Iraqi Security Forces and Coalition Forces) and there is another chart showing "Ethno-Sectarian Incidents and Executions".

I have not yet had the chance yet to study the most recent Dept of Defense Quarterly Report in depth. My initial scan indicates that it includes mention of dozens of other important factors whose trend patterns might help us better understand the situation on the ground, but for which no trend data or trend graphics are presented. Among these missing metrics or what might also be called "invisible indicators" or "faceless factors" are many new metrics that are not included even in our list of more than 100 metrics that we have already mentioned in this series of posts. We have it on our to do list to see if in the next couple of weeks we can bubble up these missing metrics to add to our growing list.

Second, there is a powerful, provocative and quite critical report by Anthony Cordesman analyzing the Dept of Defense's Quarterly Measuring Security and Stability in Iraq report that the DoD issued in May 2006: The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”

Here's a brief quote from the introduction [emphasis added]:

The media can provide some of this picture, as can outside experts and scholars, but only the US government has the resources and access to information that provide a comprehensive overview of the situation. The quarterly report to Congress issued by the Department of Defense, “Measuring Stability and Security in Iraq,” is supposed to be a key document to achieve this goal.

Like the State Department weekly status report on Iraq, however, it is deeply flawed. It does more than simply spin the situation to provide false assurances. It makes basic analytical and statistical mistakes, fails to define key terms, provides undefined and unverifiable survey information, and deals with key issues by omission.

This entire report is well worth reading. It focuses on the reliability and validity of the trend data we are getting - a topic that we have not yet touched on in this report but which is of course highly relevant to everything we are looking at. It would be extremely valuable to repeat this kind of evaluation of the quality of data sources with the latest DoD quarterly report, with the weekly State Dept report, with the Brookings Iraq Index and with ALL the factors that we rate as important and relevant to understanding the situation on the ground in Iraq.

In my view Anthony Cordesman brings a level of rigor to this particular report that is lacking from the other sources we have so far examined and his approach would be well worth emulating going forward.

In the meantime, just because there are questions about the quality of many of the individual data factors, that doesn't mean we should stop doing what we've been doing. This does not mean that we should stop measuring and or that we should stop regularly updating our trend tables and trend charts or that we should stop searching for missing metrics and adding them to our list.

In fact, I think you will find that it is true that by being able examine many trend factors in a time efficient manner provides powerful checks and balances that can help us discover which factors may be most likely to suffer from quality problems.

If you know of other sources of vital Iraq trend data that we are missing, please let us know.

Taking Stock of the Recent Iraq Trends Posts: A demonstration of future possibilities

So far in our recent series of posts on trends on the ground in Iraq, we have created a number of trend data files (Iraq-RR-Example.csv and ohanlon-key-factors.csv) and trend visualization documents (Combine-Iraq-Trends and vizualizing-trends-ohanlon-testimony.pdf) that we hope have helped make the situation in Iraq more understandable, more quickly, to a wider number of individual citizens. If you want a quick refresher of how easy it is to look at more than 50 important trends in Iraq, just download the two pdf documents noted above and see what you think.

We believe that this approach results in radical time savings and increased in depth and breadth of coverage and therefore understanding.

We also hope that by taking these steps and preparing this demonstration of what other new and creative things might be done with the available trend data that this work will benefit even the key decision makers who are directly or indirectly involved in the day to day situation in Iraq and even those who are regularly spending a substantial portion of their time looking at and interpreting trend data.

We would enjoy hearing your comment and feedback on whether you think this demonstration of the possibilities represents a path worth pursuing further.

Here's a more detailed summary of what we have covered and accomplished so far in this series of posts that we began on January 13th.

1. we pulled together a series of charts about key trends in Iraq drawn from several important source documents and web sites and we have made this whole set of charts more readily viewable by assembling them into a single downloadable pdf document - Combine-Iraq-Trends.
   
2. In this January 18th post we showed how we could combine raw tabular trend data from several different sources and paste it into a single readily-reusable (RR) csv file. This meant we could then easily calculate and visualize some new, important, and previously invisible trends such as the wounded to killed ratio or the three month moving average for troop fatalities and we could save the original combined trend data with the new calculated trend factors together in a single csv file: Iraq-RR-Example.csv that others could then examine and use directly.
3. We then took a more elaborate case and showed how straightforward it was to transform a table of 30 key factors from Michael O'Hanlon's testimony into a readily reusable trend data file - ohanlon-key-factors.csv. - and then use that file to generate a series of easily viewable trend charts - vizualizing-trends-ohanlon-testimony.pdf.
   
4. And finally, in our most recent posts, we demonstrated that even though we had already pulled together and presented more than 50 different trend charts, that there were still rather important factors, whose trends we had not yet looked at. In this post yesterday, we listed more than 20 important factors that were mentioned as part of the verbal argument presented by Michael O'Hanlon in his testimony, but which did not show up in his table of trend factors. And in our most recent post today, we showed how a close reading of the Brookings Iraq Index allowed us to discover more than 50 important additional factors that could help us imagine a fuller picture of what is going on in Iraq.

If you want to read the whole series, the following link will take you to the listing for the January 2007 posts: Change-over-time-January-2007.

With so many factors to consider, you might feel this is daunting. In upcoming posts we will directly address any fears you might have that handling 100 or more factors might be too difficult or beyond your reach and show you how the approach of using Readily Reusable data combined with time saving tools of which TLViz and CSVPNG are useful models can save the day.

We will also be taking a look at what we might do next to make the trend data coming out of Iraq even more useful and valuable for both ordinary citizens and for decision makers.

More Invisible Metrics: What other trends is it vital that we make readily resuable and visualizable

So far in this series of posts about better understanding the situation on the ground in Iraq, we have focused on bringing together important trends and presenting them graphically in ways aimed at making this vital information more readily available and usable to both decision makers and to ordinary citizens.

Our short term goal has been to radically reduce each viewer's cost in time spent (compared to previous formulations) while at the same time sharply increasing the depth and breadth of understanding attained.

Our longer term goal is for this increased level of understanding (now achievable and within reach of a much larger number of individuals) to be the catalyst that spurs better discussion, improved collaboration, and much deeper thinking about what is really going on, what it means, and what actions might lead to improved results in the future.

In our next post, we will give a wrap up the ground we have covered so far in this series. For now, we have some unfinished business, namely that there are many other vital Iraq trend factors that are necessary for our more complete understanding of the situation. So continuing with the list of missing factors from our previous post, below we highlight an additional 55 factors that we discovered and selected through a close reading of the Brookings Iraq Index.

In our opinion, being able to see the trend graph for each of these would add to our overall understanding of what is going on in Iraq. Admittedly, not all of these are of equal importance but the trend for each is likely to have an important story to tell if we assemble the necessary trend data and then create the chart.

1. Percentage of US troop fatalities caused by car bombs (page 3 table)
2. Iraqi civilians killed by US troops (page 9 table)
3. Police deaths in Baghdad (page 12 table)
4. Civilian deaths in Baghdad (page 12 table)
5. Kellogg, Brown and Root Employees in Iraq (table page 14)
6. Combat arms troop strength (table page 14)
7. Combat support troop strength (table page 14)
8. Combat service support troop strength (table page 14)
9. Dept of Defense Civilians in Iraq (table page 14)
10. Total Civilian Personnel in Iraq & Kuwait (table page 14)
11. Iraqis Kidnapped Nationwide (table page 15)
12. Iraqis Kidnapped in Baghdad (table page 15)
13. Iraqi prison population in US Custody (table page 16)
14. Iraqi prison population in Iraqi Custody (table page 16)
15. Active duty troop strength in Iraq (table page 20)
16. Reserve & National Guard troop strength in Iraq (table page 20)
17. Total US Army authorized strength - Activity duty (table page 20)
18. Total US Army authorized strength - National Guard (table page 20)
19. Total US Army authorized strength - Reserve (table page 20)
20. Total US Army authorized strength - Combined Active, National Guard & Reserve (table page 20)
21. Percentage of IEDs found and disarmed (table page 23)
    
22. Number of daily Iraqi Patrols (chart page 23)
23. Number of daily US Patrols (chart page 23)
24. Number of daily Joint Iraqi-US Patrols (chart page 23)
25. Number of total daily patrols (chart page 23)
26. New passports issued (table page 24)
27. Percent of professional class that has left since 2003 (table page 24)
28. Iraqi refugees heading to Syria per day (table page 24)
29. Iraqi refugees heading to Jordan per day (table page 24)
30. US Air Strikes (Iraq only) (table page 26)
31. C-130 Sorties flown (Iraq and Afghanistan combined) (table page 26)
32. Diesel production (table page 29)
    
33. Kerosene production (table page 29)
34. Gasoline/Benzene production (table page 29)
35. LPG production (table page 29)
36. Non oil related GDP (table page 33)
37. Total cost of US operations (table page 37)
38. Cost of US defense activitites (table page 37)
39. Cost of Veteran's programs (table page 37)
40. Inflation rate (table page 39)
41. Cost of imported gasoline (table page 39)
42. Domestic price of gasoline (table page 39)
43. Relative amount of car traffic (table page 40)
44. Typical waiting time for gasoline (table page 40)
45. Number of registered cars (table page 40)
46. Internet subscribers (table page 41)
47. Tons of mail sent by Iraqis (table page 41)
48. People with potable water (table page 42)
49. People with sewerage system coverage (table page 42)
50. Water treatment capacity (table page 42)
51. Annual Iraqi Medical School Graduates (table page 42)
52. Primary school enrollment percentage (table page 43)
53. High School enrollment percentage (table page 43)
54. Percentage of Iraqis who approve of attacks on US led forces (table page 44)
    
55. Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Our list of important metrics is getting longer and longer. But this is not all there is.

What other factors do you think are important? What else is missing? What else would you like to know? What else would you like to see? Do any of these factors trigger ideas about other important factors that were not even mentioned at all? Do combination of factors come to mind for you that you think would shed even more light?

And finally, of the 100 or more factors that we have touched on in this series of posts, what is your short list of the top 10 or 20 most important ones?

Missing Metrics - What else do we need to know about how things are going in Iraq?

In looking closely at the text of Michael O'Hanlon's testimony to the Senate Foreign Relations Committee on January 10th, 2007, I noticed that he spoke of quite a few Iraqi factors on the ground that were NOT included in the list of 30 metrics reported.

A small number of the factors that showed up in the tabular data were mentioned in the logic of the text, but I was surprised by how many important factors from the text had no numerical equivalent in the table.

Here's a list of the missing factors that I found in the text with my comments in brackets.

Good year/Bad Year - <I would love to know more about what key factors Michael incorporates in his thinking in coming up with this assessment and would find it enlightening if this black and white measurement were refined and reported on say a scale from 1 to 10 or 1 to 100>

“The year 2006 was, tragically and inescapably, a bad one in Iraq.”

Sense of Progress

“The first is the failure of the 2005 election process to produce any sense of progress.”

Al-Maliki Favorabaility Rating

“In a September poll, Prime Minister al-Maliki was viewed unfavorably by 85 percent of all Sunni Arabs”

Rate of Shia attacks on Sunnis

Rate of Sunni attacks on Shia

Increasing politicization of the violence “to include many more Shia attacks on Sunnis as well as the reverse”

Iraqi state subsidies for consumer goods

“recent reductions in Iraqi state subsidies for consumer goods (which distort the economy and divert government resources)”,

The number of children being immunized.

Percentage who feel safe en route to school

Rate of private sector investment

“further slide toward chaos.”

Percent who think Economy is Poor

According to a June, 2006 poll, 59 percent call the economy poor

Percent who think security environment is poor

“and 75 percent describe the security environment as poor”.

Rate of Jobs Creation

“a massive program to create jobs”

Degree of Iraqi Leader consensus on sharing oil

“an ultimatum to Iraqi political leaders that if they fail to achieve consensus on key issues like sharing oil”,

“American support for the operation

could very soon decline”

Number of coalition forces needed to provide security.

“Coalition forces have never reached the numbers needed to provide security for the population in Iraq”

Degree of Overwork of our Soldiers and Marines

“our already overworked soldiers and Marines”.

Rate of “Success of military commanders in putting Iraqis to work

with their commander emergency response program funds”,

Degree of Support Iraqi Leaders enjoy outside their own sectarian group.

“it is clear that extremely few Iraqi leaders enjoy any real support outside of their own sectarian group.”

Degree to which Iraqi Leaders are working across sectarian lines.

“Trying to force them to work across sectarian lines must be a focus of our policy efforts, if there is to be any hope of ultimate stability in Iraq.”

The Probability that a Given Intervention will succeed

“Social scientists and military experts do not know how to assess rigorously the probabilities that such steps will succeed at this late hour in Iraq.” [Emphasis added]

Degree of Pessimism

“Overall, however, it seems fair to say that most have become quite pessimistic.”

Iraqis displaced per month

“But with 100,000 Iraqis per month being displaced from their homes, making for a total of some two million since Saddam was overthrown, ethnic cleansing is already happening.”

Bottom Line: If we want to understand what's going on in Iraq, we are going to have to work harder at bringing these missing metrics out in the open where every citizen can see them, think about them, discuss them.

Tapping the Power of Iraq Readily Reusable Data

In the previous post, we showed how the tabular data from Michael O'Hanlon's testimony to the Senate Foreign Relations Committee could be converted to a Readily Reusable (RR) format as a CSV file ( ohanlon-key-factors.csv ) and how the RR format then made it relatively straightforward and inexpensive to actually look at the visualization of all thirty of the reported trends, one by one as shown in vizualizing-trends-ohanlon-testimony.pdf.

We have found that when using RR data and TLViz, the time savings we achieve make it possible to look at hundreds of trends, one after the other, in a very short time so as to gain a gestalt sense of all the reported factors at work. Typically, moving from tabular form to RR form gives a productivity saving factor of at least 10 to 1. Frequently, the productivity increase is 50 to 1 or more.

After having reviewed each of the 30 reported trend factors from Michael O'Hanlon's testimony, it became apparent that there were some other interesting trends hiding amongst the original data that could be computed with simple calculations from the original data. This is readily achievable by opening the RR csv file with a tool such as Microsoft Excel.

For example, with Excel we could create a new column of data and combine US troop strength and troop strength of other non-US coalition forces and then calculate the percentage of coalitions forces that were non-US. Similarly, we could use the original data to calculate the percentage of US troops that had been killed in that month by IEDs. We could also take factors such as the cumulative number of refugees and convert them into year over year trend data.

Below, we show you the three new trend charts that bring some previously invisible trends into the light of day where we can all see them. (please click on trend graphic for full size image.)




Bottom line. Once the trend data is in RR format, all sorts of new possibilities of understanding open up that can help us better understand what is really going on.

Visualizing the O'Hanlon Trend Data

Once we had the the data from Michael O'Hanlon's Senate Foreign Relations Committee Testimony (Jan 10, 2007) in readily reusable (RR) form, we opened that csv file with TLViz and were able to generate a set of graphs in a matter of minutes showing the individual trends in visual form including some that combined several of the metrics together.

You can see a pdf slide show of the entire set of charts we created in full screen size by clicking on: vizualizing-trends-ohanlon-testimony.pdf

Using Edward Tufte's idea of small multiples, we have combined 4 trends at a time into the pictures below (please click for full size) that show a sample of the charts from the full report.

The first 4 show US troops killed by IED, Iraq Civilian Fatalities, Multiple Fatality Bombings, and Estimated Strength of the Insurgency.

The next 4 trends show Estimated Strength of Shia Militias, Daily Average of Inter-ethnic attacks, Estimated Number of Foreign Fighters, and Number of Daily Attacks by Insurgents or Militias.



The next four small multiples show Iraqi Internally Displace, Iraq Refugees, Iraqi Physicians Murdered or Kidnapped, and Iraqi Physicians who have fled Iraq.


In this next set of trends, the first chart shows the total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq. This chart was created by combining two of the originally reported trends. The second chart also combines two original trends, this time comparing the number of Iraqi forces who are technically proficient to the number who are politically dependable. The final two charts in the small multiple set show the Percentage of Household Fuel needs that are being met and the average hours per day of electricity in Baghdad.


This final small multiple set shows the Unemployment Rate, the Per Capita GDP, the Number of Trained Judges, and the Number of Telephone Subscribers.


If you click on these charts and then press F11, you should have a full screen readable version to examine. Of course, if you want to look at each chart by itself in full screen size with easy back and forth navigation, the vizualizing-trends-ohanlon-testimony.pdf is the place to go.

One of the advantages of these charts to the previous set of combined charts that we published is that they all cover the same time period and that makes understanding what they mean just that much easier. Once you understand what the X axis timeline means in one chart, you don't have to recompute that with each chart you examine.

Once again, we suggest that you download this RR data yourself and play with it to see what else you can find, what other patterns are hiding in the data. You can download by right clicking on ohanlon-key-factors.csv. We also suggest that this could be a fine opportunity for you to test out the ways in which TLViz simplifies and speeds up trend analysis work once you have RR data. You can learn more about TLViz from our earlier posts this month and you can download a copy of the full kit for Windows PC's from TrendsThatMatter's Download Page

Iraq Trend Data: O'Hanlon's Testimony to the Foreign Relations Committee

Michael O'Hanlon who is the lead author of the Brookings Iraq Index testified last week before the Senate Foreign Relations Committee. Here's the 4 page text of his January 10th testimony.

The last two pages show a relatively easy to read and understand printed table. The table lists, all in one place, what I assume to be what O'Hanlon thinks are the 30 key factors on the ground in Iraq and how they have varied year by year from November 2003 to November 2006.

Here's a version of that table that I have edited for even better visibility and to keep it from covering two pages. (please click on image for full size view):

The four data columns of the table show the trends for those factors with the data values being captured for the month of November in each of the past 4 years. From a standpoint of printed table readability, limiting the number of trend samples to 4 makes this table much more suited for direct human consumption at the cost of reducing overall trend data quality compared to a set of trend samples for a given factor that showed how that factor changes every month.

While most people are not comfortable with reading tables, this one is actually readable and understandable with the application of a modest amount of effort. Please let us know what you think?

This table as text is of course not a readily-reusable (RR) file. However, it was relatively straightforward for us to convert it into RR format and you can download your own copy by right clicking on ohanlon-key-factors.csv. This RR file is ready for examination with Microsoft Excel or even better with the T4 & Friends tools TLViz or CSVPNG.

You will note that the layout of the data has been inverted with the data rows now representing the 4 monthly sample reading time periods, and the 30 data columns representing the 30 key factors which is the orientation required for TLViz and CSVPNG. This makes for a very wide page, but of course, when using TLViz or CSVPNG to examine the trend data, we do not actually need to read it.

In our next post, we will show you why this conversion from printed table to RR format is so helpful in the way it allows the tabular data to come alive in visual form with very little effort by using a tool such as TLViz.

Creating Readily Reusable Data from Online Tables of Trends in Iraq: a quick example

Here's a quick example of how relatively inaccessible tabular data from a web site can be converted into a readily reusable format.

Starting Point: Tabular data from GlobalSecurity

Step 1. We went to Global Security's Iraq Casualties page and found a table that included one column of monthly data for US Named Dead and second column for US Wounded. The bottom of that page also includes one chart for each of these factors that are worth examining. Note that these charts were not included in the ones we selected from Global Security in our earlier post as we already had charts on US dead and wounded that we had selected from the Iraq Casualty Coalition that we felt were more revealing. We cut and pasted the table from Global Security into Microsoft Excel and played with the data to create two clean columns of CSV data and a time stamp column.

Step 2. We went to Global Security's Boots on the Ground page and cut and pasted the two columns of data showing troops in Iraq and and troops "in theatre" to create a CSV data file which now had 4 data columns.

Step 3. We now had the data in a readily-reusable (RR) form as 4 data column CSV file. We took advantage of this to compute some new data columns derived from the original four columns, namely:

1. the monthly ratio of wounded to dead
2. the monthly change in troop level in Iraq
3. the monthly change in troop level in Iraq as a percent of the previous month's level
4. the three month trailing moving average of fatalities
5. the six month trailing moving average of fatalities
6. the 9 month trailing moving average of fatalities
7. the monthly total of wounded and dead combined

The payoff:

You can take a look at the resulting RR file we have created which now has 11 data columns by downloading it from Iraq-RR-Example.csv. Check it out with Microsoft Excel, or even better this would be a great opportunity to try out TLViz (the TimeLine Visualizer) or the CSVPNG utility - both of which are expressly designed to work with RR data in this format.

You can learn more about these powerful tools in our previous posts: here and here and here and here for TLViz and here for CSVPNG and you can download the latest versions of these powerful utilities from the TrendsThatMatter Download page.

The first payoff of an RR approach is that when tabular data is converted to RR format, it puts you back in the driver seat. You are not dependent on only seeing the charts that some other analyst selected. You can look at every factor. You are not restricted to just the time period that someone else selected. You can zoom in on a period you are most interested in.

A second payoff is that you are not restricted to looking at only the combinations of data (if any) that someone else selected. You can also combine factors together with multiple trends on a single graph. Interactive tools like TLViz make it an order of magnitude easier to create different combinations of key factors and help reveal patterns hiding in the data.

A thrid payoff is that you are not restricted to only looking at the raw data. You can add moving averages and look at the moving average by itself or create a trend graph that shows both the initial raw data and the moving average together as shown in the example below:


A fourth payoff is that you can now create new computed or derived values such as the 7 new metrics we added on to our starting point of four Global Security factors. This facility to compute or derive new trend data by calculation involving the current set of factors is extremely important at revealing the full value that the data might hold.

The final payoff is that as you gather data from other sources, you can turn this into RR format and then add new columns to extend the your existing RR file.

The example we showed in this post is pretty elementary, starting as it does with only a handful of factors. As we have seen from our work with the Iraq war data so far, there are literally dozens of important factors. If we could put them all into a single RR data set, our ability to understand what is going on would be markedly enhanced and many opportunities for discovery would open up.

We are still a long way from that goal. In our next post, we will give you a little more complex example of RR data for Iraq that involves substantially more of the relevant factors.

In the meantime, I suggest you download and take a look at the Iraq-RR-Example.csv and see what else you can discover from the 11 reported factors that we have assembled for you.

Consolidating the Iraq Trend Charts from Multiple Sources: A step toward ready-viewability

In the past few days, we've posted a total of 21 charts showing trends that matter on the ground in Iraq. These were culled and selected from 4 separate sources and posted in 4 separate posts. You can scroll down the January 2007 posts from January 17th back to January 13th and find all 21 of these charts in relatively close proximity to each other. Our theory is that by bringing them all to one relatively compact place, we make it easier for those who are interested in learning more about what is happening to have access to the combined analysis and reporting talent from the four sources.

What can we do next? We think this approach makes it easier, but it still is not easy. Due to the current limitations of this blog, the graphs that appear in the blog itself are relatively small in size and to get a full grasp of the message of each trend it is necessary to click on each trend graph to get a full size image. This is pretty unwieldy and involves a lot of extra back and forth clicks to navigate to a good sized view of each trend.

Here's a possible approach to make this a bit easier and less time consuming and require far fewer clicks and allow much better navigation back and forth between the different trend charts. What we've done is to take all 21 trend graphs to create a single down-loadable PDF document that will let you easily view all trends up to the full size your screen while navigating back and forth to the chart you want using the full navigational capabilities of Adobe Acrobat (e.g. page down, page up).

Here's the link - Combine-Iraq-Trends Please let us know what you think of this step towards making these important charts rank higher on a ready-viewability scale.

Did you find the navigation easier?
Did having easy access to all trends in full size improve understanding for you?
What other suggestions do you have to make this even easier?

The Iraq Situation and Readily-Reusable Data

One of the difficulties I have encountered so far is that the data I would really like to examine first hand is not readily reusable and in some cases is completely unavailable in any form.

The first timeline collaboration principle that you will find on the right hand column of our blog template is:

1. Share the data series with the chart. Make sure it is readily reausable

To the best of my knowledge, not one of the 4 sources we have consulted so far (Brookings Iraq Index, The State Dept's Iraq Weekly Status, GlobalSeccurity, or the Iraq Casualty Coalition) has made the data that they used for their own analysis and charting both available and readily reusable as a spreadsheet or CSV file.

Admittedly some of these sources have shared their trend data series, but the method they selected means that they are not easy or ready for reuse without investing a lot of time and effort.

For example,

• Brookings Iraq Index does include some trend data tables with important metrics and frequently labels the data values on their charts. Not all of the interesting trend data that appears in the tables is translated into a chart in the report. For example, see the table on page 5 of the January 8th report that shows details of the trends in cause of death for US Troops.
  
• The Iraq Weekly Status Report from the State Department does not include any trend data at all beyond what they show in their charts. And the time period covered by the charts they have selected is rather short in most cases. For example the crude oil production chart only covers the last 10 weeks and other charts cover less than a year. Only the electricity chart stretches back to the beginning of 2004
• Global Security includes a lot of detailed trend data in tabular form in addition to and complementing the charts they provide. The range of the data presented tabularly tends to cover the entire period from the start of the war until the present.
  
• The Iraq Casualty Coalition also provides substantial amounts of tabular trend data in addition to their charts and their data covers the entire period from the beginning of the war.

Here's the problem.
Most people (both decision makers and ordinary citizens) are not particularly good or particularly thrilled by the idea of trying to make sense of trend data contained in a printed or online table.

My experience indicates that almost all adults (I would estimate the number at 90% or more) can make good sense of trend data presented in line graphs or bar graphs). On the other hand, I estimate that fewer than 10% of the people have the skills, interest or time to extract basic information from a trend table like the one on page 5 of the Brookings Iraq Index, or the many tables you can find at GlobalSecurity or at the Iraq Casualty Coalition.

What impact does this lack -- this lack of data / lack of readily reusable data / lack of easily viewable trend data for all key factors -- have on our ability to understand what's going on?

The way this plays out is that for 99% of the population, they will get their understanding of the trends from their most trusted source.

For a small percentage of people (I am guessing under 5% of the population), that trusted source might be one of the ones we have mentioned so far in this series on Iraq Trends or a similar detailed view of the situation that provides some depth and breadth and doesn't rely on just one or two metrics.

For this small group, if their source is Brookings, and they spend the time wading through the 30 page report, they will have a good chance of making good sense of the charts that Brookings selected. Similarly if their source is the State Department and if they wade through the Weekly Status report they will get a picture of the trends that the State Department selected but not the ones that Brookings was paying attention to. If their source is GlobalSecurity or the Iraq Casualty Coalition, they will get a good grasp of the charts presented at those sites.

Even in this small group that goes and gets their trend understanding from one of these useful and detailed sources (or other similar venues) a small percentage will dig any further into the tabular data that is available. The reason: primarily lack of time and secondarily lack of experience possibly combined with self conceived notions that reading tables is hard.

My reading is that few in this small group will consult and combine the trends from multiple sources like those mentioned above to get a broader perspective. Of course, if you have been reading this series of posts, we have done some of that lifting work for you and made it easier (still not easy in my opinion) to become more widely informed of the composite set of trends that are considered important drawn from a mix of analysts who may have vastly different axes to grind.

For a good portion of the rest of the population beyond this small group, the trusted source for trend data will turn out to be television and other mainstream media. Based on what I have seen from the MSM, this group of individuals will likely be exposed to quite limited trend information in any form and will often end up only being exposed to non-specific, non-numerical statements about the situation on the ground.


What to do? What to do?
Glad you asked. The answer for me is straightforward: ready reusability.

Ready reusability for the Iraq Trend data is easy to say and hard (time consuming, error prone) to do right now. In the next couple of posts, I will give you an example of what I mean by transforming some of the tabular data from expert sources into a readaily reusable form. Once the data is ready to go, I will provide some examples of why I think this is so important.

One More Source of Iraq Trend Graphics - Iraq Coalition Casualties

So far, in our previous posts on Iraq we have pulled together a set of 16 different How Are Things Going in Iraq Charts from three different sources. We have:

• 9 trends from the Brookings Iraq Index
• 4 trends from the Department of State Weekly Iraq Status Report
• 3 trends from Global Security.org

Complementing these are another 5 trend graphs drawn from Iraq Coalition Casualties and these cover some topics not previously addressed in the earlier charts (e.g. IED fatalities). they Also present some important information again but in a different format thereby bringing out additional details and nuances not readily visible earlier - for example adding notations for the timing of important events to the chart showing US Soldiers killed.

The 5 charts show:

1. US Soliders Killed with moving avg & event timing - March 2003 - December 30, 2006
   
2. US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006
3. IED Fatalities per month - July 2003 - January 2007
4. Iraqi Police/Military fatalities - January 2005 - December 2006
5. Iraqi Civilian fatalities - January 2005 - December 2006

This brings us to a total of 21 charts. I think you will see that by arranging these together in a somewhat more convenient and time-saving format than the separate originals, your understanding of what's happening in Iraq has been boosted. I think you will also see that being able to look at all 21 one of these in a relatively short period of time by scrolling down this blog allows you to develop more of a gestalt understanding of how all the factors interact and impact on each other.

• How many charts is enough?
• Are we there yet?
  
• Do we already have everything we need to understand the situation?
• Is it relatively easy for decision makers or interested citizens to get rapid access to all of the most important factors?
• Do we now have the full deck of cards that will allow us to make wise choices?
• Have we arranged things so that automatic monitoring and reporting of key factors enables us to determine quickly whether the actions we take are having the desired effect as we go forward?

I think not. My two cents is that we are not there yet and in the next few posts, I will aim at making some further improvements that make these important trends more accessible to both decision makers and ordinary citizens. I will also lay out a plan for even further improvements that could be available to us in the future if we begin to take appropriate decisive action now.