Longer term view of Iraqi Oil Production

Here's a longer term (10 year view) of Iraqi Oil Production in a single chart (Source is Oil Market Report, provided by the International Energy Agency). With the more complete time range view, it becomes easier to put the most recent production data as reported in the Iraq Weekly Status Report into broader perspective.

Hat tip for this link to Scott Johnson.

Michael O'Hanlon Year over Year Iraq Update

Here's a recent Michael O'Hanlon New York Times article which includes year over year table showing some 17 trend metrics from Feb 2004, year by year through Feb 2007.

The State of Iraq: An Update:

We plan to graph some of the more interesting of these trends in a future post to see what patterns we can discover that may be hiding in the data once we have transformed the posted table into a more readily reusable format for further investigation and discussion.

The Brookings Iraq Index Archive

Here's the link for the Brookings Iraq Index Archive: Iraq Index Archive that lets you access their entire history of this important report going back to Nov 17th, 2003.

Latest Iraq Trends from the Brookings Institution

The Brookings Institution continues its weekly Iraq Trend reporting. Here are some interesting charts gleaned from their March 22nd edition. Note the link to the Brookings Latest Iraq Trends Report will always take you to their most recent version.

Security Stations Established. This is a new chart reflecting changes underway as a result of the "surge" showing the number of security stations (cumulative) that have been established in Baghdad. The new goal is to reach 70 stations (original goal of 30).


Patrols Per Week. This new chart shows patrols carried per week and shows an impressive jump up in recent weeks.

IED Fatalities & Percentage. This dual axis chart shows both the number of IED fatalities (red bars, left scale) and the percentage of all fatalities that are attributable to IEDs (blue line, right scale).

Targetted Attacks. This chart shows average weekly attack based on who the attacks target - Coalition forces, Iraqi forces, or others.

Oil Production: Iraq Weekly Status Report - March 21, 2007

Here's the latest Iraq Oil production chart from the Iraq Weekly Status Report, March 21, 2007
showing production at or close to the Oil Ministry Goal for 2007 and holding quite steady .for the past 7 weeks after a slowdown period at the beginning of January.

To completely understand the meaning of current production and this year's goal, it would be helpful if the full range of production data since the beginning of the war were also available in addition to the zoomed in most recent view that is presented.

With that in mind, I assembled a mini time series history (each about 3 months apart) of this chart as it appears in weekly reports stretching back to 2005. Note that the next to the last chart overlaps the charts before and after to more clearly show the January 2007 dip.

Ideally, all this data would be on a single chart and much more readily digestible.

Iraq Reconstruction Management Office: Employment Chart

Here's latest employment chart from the US State Department's Iraq Reconstruction Management Office (IRMO) Report dated March 20th, 2007.

It shows Iraqi employment in US Govt. projects and what appears to be a rapidly declining trend. It would interesting to determine whether the change in this factor was having an impact on other key success factors in Iraq.

If you would like to be on an email distribution list for the full weekly IRMO report which also contains detailed updates about Electricity, Oil, Water, and Sanitation project. you can send your request to:
jddurso AT earthlink DOT net

More BBC trend charts for Iraq

BBC NEWS | In Depth | Iraq violence, in figures

The BBC link above is the source for these two easy to read charts -- the first showing the varying levels of US troop strength and the second showing the US Military Death Toll.

Good points of the BBC presentation:

1) Full range of Data. Both charts go back to the beginning of the conflict in 2003 and are up to date as of Feb 2007.

2) Combining Data from Multiple Sources. The BBC has combined data sources obtaining the troop strength information from the Brookings Institution and fatality figures from the Iraq Coalition Casualty Count. The charts are presented on the page so the single Time Axis can be used for both for more ready comparison.

3) Detailed Interactive Event Description Integrated with the Trend Data.
The BBC original interactive chart has the option to click on the event arrows to get a brief snapshot of some of the important events and see how these fit into the overall picture and relate to the underlying trends.

More Timelines in Motion

BBC NEWS | In Depth | Baghdad: Mapping the violence

This interactive graphic from the BBC will repay your careful interactive examination.

Some things to try.

1) Drag the timeline slider slowly to see how the level of violence has changed month to month.

2) Also click on Ethnic Areas and then click back and forth between Current and Pre-2006 to see the dramatic changes in ethnic mix that have taken place already and to understand where the predominance of violence is taking place.

It will be interesting to compare the March 2007 view of violence when it is ready to the pictures of violence in December 2006, January 2007 and February 2007 to better gauge the degree to which the "surge" may be having a positive effect.

This would be even better if, like Gapminder, there were a PLAY button and some Speed control. Next Month and Previous Month buttons would also be handy as it is a little tricky dragging the timeline control exactly one month forward or back or clicking on exactly a given hash mark along the timeline of months.

Hat tip to: Data Mining: Text Mining, Visualization and Social Media

And speaking of gapminder

Thanks to a hat tip from Pat McConnell, I've learned that the latest news is it that GOOGLE has acquired gapminder's Trendalyzer software. Here's the scoop from the the gapminder home page.

Make sense of the world by having fun with statistics

Gapminder and Google share an enthusiasm for technology that makes data easily accessible and understandable to the world. Gapminder’s Trendalyzer software unveils the beauty of statistics by converting boring numbers into enjoyable interactive animations. We believe that Google’s acquisition of Trendalyzer will speed up the achievement of this noble goal. Trendalyzer’s developers have left Gapminder to join Google in Mountain View, where Google intends to improve and scale up Trendalyzer, and make it freely available to those who seek access to statistics.

The Stockholm-based Gapminder Foundation will continue to spearhead the use of new technology for data animations. The goal is to promote a fact-based worldview by bringing statistical story-telling to new levels. In collaboration with producers of accurate statistics that are eager to give the public free access to databases, Gapminder hopes to recruit and inspire many users of public statistics.

If you haven't seen Trendalyzer in action, you are in for a treat. You can check it out for yourself with this powerful example.

Trendalyzer in simple terms is a timeline in motion machine which has all sorts of potential to help us discover previously unseen patterns. Google's stated intentions to continue to advance on this already powerful design is an exciting development for anyone interested in understanding the behavior of important factors and how they change over time.

timelines in motion

A nice example of timelines in motion accompanied by an audio track from Brad Delong.

The combination of a timeline change in motion and a guided audio track is powerful.

To make this even better, I would have liked it to zip along at a faster pace and it would be great to have a higher resolution graphic quality so that the X and Y axes could be more readily examined and understood.

You won't be surprised to hear me saying that making the data available in a readily re-usable format would sweeten things even further.

Providing a mechanism similar to gapminder to be able to watch this change over time with the viewer in control of the speed would be even better.

What do you think?

Many Eyes - A New Doorway for Graphical Collaboration

From our previous post we noted that:

We believe in many eyes, many hands, many voices working together on the path of mutual discovery (rather than the idea of a single expert figuring it all out and presenting it to us in final form).

To help foster exactly this kind of process, I am excited to note a new web site, Many Eyes that is now available for exactly this purpose. As they describe it on their home page: Many Eyes is a place to: view your data, ask questions, share your discoveries, and harness the collective intelligence of the net for insight and analysis.

Once you register, you can upload interesting data sets and create visualizations that you feel are meaningful. Others can view your visualization and comment. They can also create their own variant visualizations using the data you uploaded and publish these for others to view and comment. Many different visualization formats including time series graphs, bar charts, scatter charts as well as some very creative new chart types are available in an interactive fashion. Creating new charts is similar to the kinds of interactions you might experience with one of the many stock quote charting web sites such as the java charts on prophet.net

Even better, once a data set is uploaded by anyone, it is possible someone else to download that data set for further analysis on their system using their favorite tools. For example, you might want to take a set of raw time line data series and compute some additional columns of data as we did in one of our previous Iraq posts. You could then take your new data set with added columns and upload it to Many Eyes for others to work with directly.

To test out these capabilities, I uploaded the O'Hanlon indicators that we have recently posted about. You can find a visualization that shows a single indicator that I created at: Percent of Coalition Troops Appears to be Shrinking. Once you get to that page, you can select other indicators from the list box on the left side to view the other trends at work. You can also download the entire table to your computer or leave a comment or create at publish your own chart. [Unfortunately, at present the long indicator names cannot be fully seen during the selectino process. So if you want to know what they are, you can find out by clicking on the table like icon element to look at and scroll through the top row of the data set where the indicator names appear]

I see Many Eyes (what a great name) as a big step forward in thinking about, discussing, understanding, and collaborating about the most important factors in our lives. I expect I will be uploading additional data sets and creating new visualizations there in the future. I would love to hear what you think about his newly available capability.

Change Over Time: First Principles Revisted

I am enjoying reading Edward Tufte's new book: Beautiful Evidence. It's the fourth in a series of wonderful Tufte volumes that exemplify the power of visual display and it certainly lives up to its title. Beautiful Evidence, like its predecessors, is simply a beautiful and inspiring work - blending art and science and philosophy and the linkages between numeric data and their visual representations into a unifying whole.

Tufte is powerfully interested in the underlying principles. For example, here's a brief passage from his introduction to Beautiful Evidence:

Evidence is evidence, whether words, numbers, images, diagrams, still or moving. The intellectual tasks remain constant regardless of the mode of evidence: to understand and to reason about the materials at hand, and to appraise their quality, relevance, and integrity.

... The point of evidence displays is to assist the thinking of the producer and consumer alike. Evidence presentations should be created in accord with the common analytical tasks at hand, which usually involve understanding causality, making multivariate comparisons, examining relevant evidence, and assessing the credibility of evidence and conclusions.

This blog is especially concerned with a particular kind of evidence: multi-dimensional time series. We are surely interested in the underlying intellectual and analytical tasks including understanding causality, making comparisons, and cross checking conclusions.

"Beauty" on the other hand, plays a much smaller role in our efforts. It is always a delight when it makes one of its rare appearances for us, but is not a primary or even a secondary goal. In fact, many of the approaches we recommend in order to meet our primary mission by their very nature tend to lead away from beauty.

Reading Tufte always encourages me to take a step back and think about the first principles that are the foundation of the work of this blog. I want to think through and examine all that his approach has in common with the work shown in this blog and all the ways we might differ.

Here are some further thoughts about the first principles that are the basis of my continuing work with the visualization of time series data (with special emphasis today on the principles that might mitigate against "beauty").

1. Time is of the essence - both analyst time and viewer time. While making comparisons and searching for understanding and seeking out properties of the data that have never before been seen, we will give preference to those approaches that help speed up the work of the analyst, even if some or a lot of beauty must be sacrificed.
2. Following Covey, we seek first to understand the data and then to present our findings in ways that make it easiest and quickest for others to understand.
   
3. Sharing the data in a readily accessible and readily reusable form along with the presentation of any findings is perhaps the core principle of our work. It's the one that allows for collaboration and for a rapid and multi-step search for meaning.
4. We aim to make both the data and whatever graphical representations we create accessible to experts and non-experts alike.
5. We care deeply about practical tools that allow ordinary citizens to participate in this work of comparison and understanding.
6. We believe in many eyes, many hands, many voices working together on the path of mutual discovery (rather than the idea of a single expert figuring it all out and presenting it to us in final form).
7. We want to apply this approach to the most important, practical, everyday problems that we face. We want this approach to help dramatically improve decision making. And we want this to happen in near real time.
   

For example, in hindsight, the famous Napoleon' March graphic that Tufte has written extensively about is certainly a beautiful and powerful display that promotes understanding of what happened to Napoleon's army on the way to Russia and back. But of course, it was not available in time to help Napoleon make better decisions. What we are interested in is figuring out ways in which such powerful understanding might be made available soon enough (in near real time) to actually be of use to the decisions makers and to ordinary citizens as complicated and important events are unfolding. Our recent series of posts about the key indicators at work that help us understand how things are going in Iraq is a step in that direction.

To sum up, in our approach, if we have to chose, we will chose timeliness, usefulness, ready re-usability, accessibility to the ordinary citizen, tools that are available here and now, and the ability to collaborate over beauty every time.

Edward Tufte's "Beautiful Evidence", Sparklines & CSVPNG

Edward Tufte is out with a new and inspirational book: Beautiful Evidence which includes a full chapter on the idea of sparklines - small word like graphic elements that can be integrated fully with text and that allow for massive amounts of statistical data to be presented in a compact format that encourages comparison between elements.

Not many software tools are yet available to help create the sparkline graphical elements. One exception is the T4 & Friends utility: CSVPNG that we have talked about in previous posts. CSVPNG has some introduced some new basic capabilities that can let you experiment with creating your own sparklines without having to write all the code yourself. The latest version of CSVPNG can be downloaded from the Trends That Matter download page

Apprenticed Investor: Know Thyself

Apprenticed Investor: Know Thyself

Hat tip to Brad Delong for pointing out and reminding me of this excellent article by Barry Ritholtz in his Apprenticed Investor. While Barry is directing his comments at our all too human behaviors when we are investing in stocks, his discussion of pitfalls to watch out for is highly relevant to the analysis of any set of complex data (e.g. the 142 Iraq factors we have so far identified).

Take a look. It's a quick read and well worth the effort

Top 12 factors (one citizen's view)

Here's an even shorter list of what I consider to be the top 12 factors to watch right now.

This does not mean that we should only look at this subset. In my opinion, every item on the previous post's list of 34 is of high importance and worthy of being measured, tracked, and reported consistently in near real time.

In fact, while perhaps of not as high importance (and opinions will of course vary on this), almost every item on the list of 142 deserves regular attention - measurement, tracking, and reporting - so that the full picture can be brought into view.

From the Brookings Iraq Index Report

• Iraqi Deaths from All Violent Causes

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

• Crude oil production

5 trend graphs drawn from Iraq Coalition Casualties

• US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
• US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006

from Michael O'Hanlon's testimony. 30 factors

• Daily Average of Inter-ethnic Attacks
• Avg Hours/Day of Power Baghdad (prewar 20)
• Unemployment Rate (%)

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

• Rate of Jobs Creation

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

• Iraqis Kidnapped in Baghdad (table page 15)
• Number of total daily patrols (chart page 23)
• People with potable water (table page 42)
• Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

What does your short list look like?

First cut at identifying most important factors

Here's one private citizen's view on which of the 142 Iraq factors identified so far are most vital to understanding what is happening on the ground in Iraq. These are the ones that I believed need to be measured and reported on regularly so we can see the trends at work. I have picked 34 out of 142 (or about 24% of the total) so this is something like an 80/20 selection.

What does your list look like?

It would be excellent to see Michael O'Hanlon's list, or Anthony Cordesman's list, or Secretary Gate's list, or Secretary Rice's list, or Lt. Gen. Petraeus' list of the top factors they will be considering as we go forward.

From the Brookings Iraq Index Report

• Average Weekly Attacks by Time Period – 01 Jan 2004 – 11 Aug 2005
• Iraqi Deaths from All Violent Causes – Jan 06- Oct 06
• Enemy-Initiated Attacks Against the Coalition and its Partners May 2003 –Aug 2006

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

• Crude oil production for the past 10 weeks.
• Electricity production compared to electricity demand since January 2004.

5 trend graphs drawn from Iraq Coalition Casualties

• US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
• US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006

from Michael O'Hanlon's testimony. 30 factors

• Estimated Strength of Insurgency
• Estimated Strength of Shia Militias
• Daily Average of Inter-ethnic Attacks
• Iraqi Security Forces who are Politically Dependable
• Percent of Household Fuel Needs Available
• Avg Hours/Day of Power Baghdad (prewar 20)
• Unemployment Rate (%)

New indicators calculated from combinations of O'Hanlon's metrics

• total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq.
• percent of coalition troops that are not US troops
• Percent of US troops killed by IEDs
• Year over year increase in Iraqi refugees

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

• Rate of Shia attacks on Sunnis
• Rate of Sunni attacks on Shia
• The number of children being immunized.
• Rate of Jobs Creation
• Degree of Overwork of our Soldiers and Marines

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

• Civilian deaths in Baghdad (page 12 table)
• Iraqis Kidnapped in Baghdad (table page 15)
• Iraqi prison population in US Custody (table page 16)
• Iraqi prison population in Iraqi Custody (table page 16)
• Number of total daily patrols (chart page 23)
• Gasoline/Benzene production (table page 29)
• Total cost of US operations (table page 37)
• People with potable water (table page 42)
• People with sewerage system coverage (table page 42)
• High School enrollment percentage (table page 43)
• Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Combined Listing of Iraq Factors

Here's the combined list of 142 factors we have identified so far based on this series of recent posts. I have left off the links to the various sources. You can find them these links in the previous posts.

Which ones do you think are most vital to pay attention to on a regular, near-real-time basis in order to better understand what's happening on the ground?
How do you rank each on a scale of 1 to 10?
How often do you think each factor must be measured and reported?
Are there any ones you think can be dropped?

From the Brookings Iraq Index Report

1. Multiple Fatality Bombings – May 03 – Jan 07
2. Killed and Wounded in Multiple Fatality Bombings – May 03 – Jan 07
3. Average Weekly Attacks by Time Period – 01 Jan 2004 – 11 Aug 2005
4. Improvised Explosive Devices (IED’s) Detonated and Disarmed - Jan 2006 –June 2006
5. US Troops Wounded in Action March - March 2003 – Jan 2007
6. Iraqi Military and Police Killed Monthly – Apr 03 – Jan 07
7. Iraqi Deaths from All Violet Causes – Jan 06- Oct 06
8. Number of Daily Attacks by Insurgents and Militias - June 03 – Nov 06
9. Enemy-Initiated Attacks Against the Coalition and its Partners May 2003 –Aug 2006

U.S Dept. of State's Iraq Weekly Status Report (30 pages, January 10, 2007).

1. Crude oil production for the past 10 weeks.
2. Crude oil export since February 2006
3. Electricity production compared to electricity demand since January 2004.
4. The strength of the local Iraqi currency compared to the dollar
   

GlobalSecurity.org

1. US Ground troops in Iraq from March 2004 through December 2006
2. Average daily casualties - Iraqi and Coalition from 01 April 2004 through 12 May 2006
3. Average weekly attacks by time period 01 April 2004 through 12 May 2006

5 trend graphs drawn from Iraq Coalition Casualties

1. US Soldiers Killed with moving avg & event timing - March 2003 - December 30, 2006
2. US Soldiers Wounded with moving avg - Sept 6, 2003 - Dec 29, 2006
3. IED Fatalities per month - July 2003 - January 2007
4. Iraqi Police/Military fatalities - January 2005 - December 2006
5. Iraqi Civilian fatalities - January 2005 - December 2006

Composite Readily Reusable (RR) example based on Tabular data from GlobalSecurity.org

1. the monthly ratio of wounded to dead
2. the monthly change in troop level in Iraq
3. the monthly change in troop level in Iraq as a percent of the previous month's level
4. the three month trailing moving average of fatalities
5. the six month trailing moving average of fatalities
6. the 9 month trailing moving average of fatalities
7. the monthly total of wounded and dead combined

from Michael O'Hanlon's testimony. 30 factors

1. US Troops in Iraq (thousands)
2. Other Foreign Troops in Iraq (thousands)
3. US Troops Killed
4. US Troops Killed by IEDs
5. US Troops Wounded
6. Iraqi Army/Police Fatalities
7. Iraqi Civilian Fatalities
8. Multiple Fatality Bombings (for the month in question)
9. Estimated Strength of Insurgency
10. Estimated Strength of Shia Militias
11. Daily Average of Inter-ethnic Attacks
12. Estimated Number of Foreign Fighters
13. Number of Daily Attacks by Insurgents/Militias
14. Attacks on Oil/Gas Assets
15. Iraqi Internally Displaced Since April 2003
16. Iraqi Refugees Since April 2003 (Total)
17. Iraqi Physicians Murdered or Kidnapped
18. Iraqi Physicians who have Fled Iraq
19. Iraqi Security Forces who are Technically Proficient
20. Iraqi Security Forces who are Politically Dependable
21. Oil Production (Millions of Barrels/Day prewar 2.5)
22. Percent of Household Fuel Needs Available
23. Electricity Production (in megawatts prewar 4000)
24. Avg Hours/Day of Power Baghdad (prewar 20)
25. Unemployment Rate (%)
26. Per Capita GDP (Real Dollars prewar $900)
27. Number of Trained Judges
28. Telephone Subscribers (prewar 800000)
29. Independent Media Companies (prewar 0)
30. Iraqi Optimism (% who think things going in right direction)

New indicators calculated from combinations of O'Hanlon's metrics

1. total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq.
2. percent of coalition troops that are not US troops
3. Percent of US troops killed by IEDs
4. Year over year increase in Iraqi refugees

Additional factors (missing metrics) contained in the text of O'Hanlon's testimony

1. Good year/Bad Year (e.g. subjective on scale of 1 to 10)
   
2. Sense of Progress (e.g. subjective on scale of 1 to 10)
3. Al-Maliki Favorability Rating
4. Al-Maliki Favorability Rating for all Sunni Arabs
5. Rate of Shia attacks on Sunnis
6. Rate of Sunni attacks on Shia
7. Iraqi state subsidies for consumer goods
8. The number of children being immunized.
9. Percentage who feel safe en route to school
10. Rate of private sector investment
11. Slide toward chaos (e.g. subjective on scale of 1 to 10)
12. Percent who think Economy is Poor
13. Percent who think security environment is poor
14. Rate of Jobs Creation
15. Degree of Iraqi Leader consensus on sharing oil
16. American support for the operation
17. Number of coalition forces needed to provide security.
18. Degree of Overwork of our Soldiers and Marines
19. Rate of “Success of military commanders in putting Iraqis to work with their commander emergency response program funds”
20. Degree of Support Iraqi Leaders enjoy outside their own sectarian group.
21. Degree to which Iraqi Leaders are working across sectarian lines.
22. The Probability that a Given Intervention will succeed (e.g the most recent set of changes in strategy and tactics)
    
23. Degree of Pessimism
    
24. Iraqis displaced per month
25. Degree of Ethnic Cleansing

55 missing metrics/invisible indicators/faceless factors from Brookings Iraq Index

1. Percentage of US troop fatalities caused by car bombs (page 3 table)
2. Iraqi civilians killed by US troops (page 9 table)
3. Police deaths in Baghdad (page 12 table)
4. Civilian deaths in Baghdad (page 12 table)
5. Kellogg, Brown and Root Employees in Iraq (table page 14)
6. Combat arms troop strength (table page 14)
7. Combat support troop strength (table page 14)
8. Combat service support troop strength (table page 14)
9. Dept of Defense Civilians in Iraq (table page 14)
10. Total Civilian Personnel in Iraq & Kuwait (table page 14)
11. Iraqis Kidnapped Nationwide (table page 15)
12. Iraqis Kidnapped in Baghdad (table page 15)
13. Iraqi prison population in US Custody (table page 16)
14. Iraqi prison population in Iraqi Custody (table page 16)
15. Active duty troop strength in Iraq (table page 20)
16. Reserve & National Guard troop strength in Iraq (table page 20)
17. Total US Army authorized strength - Activity duty (table page 20)
18. Total US Army authorized strength - National Guard (table page 20)
19. Total US Army authorized strength - Reserve (table page 20)
20. Total US Army authorized strength - Combined Active, National Guard & Reserve (table page 20)
21. Percentage of IEDs found and disarmed (table page 23)
22. Number of daily Iraqi Patrols (chart page 23)
23. Number of daily US Patrols (chart page 23)
24. Number of daily Joint Iraqi-US Patrols (chart page 23)
25. Number of total daily patrols (chart page 23)
26. New passports issued (table page 24)
27. Percent of professional class that has left since 2003 (table page 24)
28. Iraqi refugees heading to Syria per day (table page 24)
29. Iraqi refugees heading to Jordan per day (table page 24)
30. US Air Strikes (Iraq only) (table page 26)
31. C-130 Sorties flown (Iraq and Afghanistan combined) (table page 26)
32. Diesel production (table page 29)
33. Kerosene production (table page 29)
34. Gasoline/Benzene production (table page 29)
35. LPG production (table page 29)
36. Non oil related GDP (table page 33)
37. Total cost of US operations (table page 37)
38. Cost of US defense activities (table page 37)
39. Cost of Veteran's programs (table page 37)
40. Inflation rate (table page 39)
41. Cost of imported gasoline (table page 39)
42. Domestic price of gasoline (table page 39)
43. Relative amount of car traffic (table page 40)
44. Typical waiting time for gasoline (table page 40)
45. Number of registered cars (table page 40)
46. Internet subscribers (table page 41)
47. Tons of mail sent by Iraqis (table page 41)
48. People with potable water (table page 42)
49. People with sewerage system coverage (table page 42)
50. Water treatment capacity (table page 42)
51. Annual Iraqi Medical School Graduates (table page 42)
52. Primary school enrollment percentage (table page 43)
53. High School enrollment percentage (table page 43)
54. Percentage of Iraqis who approve of attacks on US led forces (table page 44)
55. Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Does reading through this entire list inspire your creativity to identify other missing metrics or other new calculations that might be made by combining this original list?

The Most Important Factors

Ooops. One critique I left off in the previous post is that to the best of my knowledge, none of the sources that I have consulted so far has laid out what they consider to be the most important of all the factors that they mention.

It is clear that huge amounts of data gathering, analysis, and reporting go into creating many of these reports which end up mentioning 50 or more factors.

From each of these sources, it would be most helpful to see a short list of the 20% of those factors that are considered most important by each analyst or group of analysts reporting on the subject.

I will take a cut myself at this particular challenge in the next post to show you what I mean.

Are we there yet?

How well does the current set of Iraq trend charts & tables serve our needs?

Answer: I think we are a little better off than we were 10 days ago when I began this series of posts on Iraq, but we certainly are not where we need to be. We can look at the charts that are currently available from multiple sources much more quickly now than we could before by skimming these two pdf files: Combine-Iraq-Trends and vizualizing-trends-ohanlon-testimony.pdf. But each of these documents has it's own and weaknesses as discussed below. And there are a growing number of important missing metrics that as yet have no publicly available trend chart or trend table.

What are the weaknesses of the Combine-Iraq-Trends document?

Answer: This combined report consisting of trend charts drawn from 4 different sources covers a lot of important ground and shares the trend behavior of many factors that appear quite important. On the downside, the big weaknesses of this set of trend charts are numerous:

1. There is a wide variation in formats - This reduces overall understandability and slows down the process of viewing the entire set of charts. Each chart must be examined individually with careful checks to make sure that the range of the X and Y axes are are clear and to locate the legend information that tells you what kind of factor you are looking at. This is in clear violation of the Edward Tufte's idea of small multiples that we discussed in a previous post. Applying small multiple thinking to a set of trend charts means that for the viewer, once understanding is reached regarding how a single chart in a series is laid out and what graphing conventions have been used, succeeding charts can be grasped with much reduced time and effort.
   
2. Time frames of the individual charts are widely different. Some cover the period from the start of the War in 2003 right up to the present moment. Others, cover much shorter periods. For example, the IED chart covers only from January 2006-June 2006. Some charts show monthly data, others show weekly data. This is another violation of the small multiple idea and the consequence for the viewer is that it is more difficult and more time-consuming to make any comparisons between different factors.
   
3. Stale Data: Violation of the "Near Real Time" Principle In many charts such as the IED chart mentioned above, the most recent sample is many, many months ago. If something is rated as an important factor, then it won't do us any good unless we measure and report on it often and that we make the most recent sample data available as quickly as possible. A 7 month old data point of a key trend factor can substantially frustrate our ability to understand what's going on, to determine whether interventions we have taken are working, or to decide what to do next.
4. Overly short time frames. Charts such as the State Dept's Crude Oil Production chart cover an amazingly short time frame - in this case from October 30th, 2006 to January 7th, 2007. While this chart is both interesting, useful, and clear, the viewer is seriously short-changed by not being able to look at what has happened to this key trend factor for the entire period since the war began.
   
5. Unavailability of the Underlying Data. In many cases with these charts, the underlying trend data is unavailable for further review or analysis. This means that the viewer of the chart is sharply impeded from discovering different angles to examine the data (e.g. by combining several factors together in a single chart, or calculating new previously invisible factors as a combination of an original set of factors.) If the original analyst who selected the chart for presentation did not create one chart for each important view, then without the data, these new views are simply unattainable. And of course, it's never possible for the original analyst to provide all the views.
   
6. Data that is provided is Not Readily Reusable (RR). In the relatively small number of cases where the raw trend data is provided, it is painful and time-consuming to transform these data into an RR format. This slows down the effort of understanding and because of the time involved, only a small percentage of potentially interested parties ever actually make these transformations.
7. Missing Metrics, Quality of Metrics. Many of these sources mention other metrics specifically (presumably because they are considered important) but do not provide trend tables or charts. And all metrics, those that are charted and those that are not, must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in

The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”

What are the weaknesses of the Combine-Iraq-Trends document?

Answer:

1. Stale Data: Violation of the "Near Real Time" Principle - the most recent data point in all of these charts is November 2006 so the trend series displayed is not as up to date (on January 10th when this data was presented) as one might wish.
   
2. Overly long time frame between samples. Way too few samples. Having a total four samples, one year apart creates a trend table that is quite readable on an 8 by 11 sheet of paper. The price is dramatic loss of vital detail and can lead to seriously misleading conclusions.
3. Data that was originally provided was Not Readily Reusable (RR). We were able to work our way around this barrier and create trend charts from the data, but it was costly in time and will serve as an impediment to further analysis of the data for most people most of the time.
4. Missing Metrics, As we noted in an earlier post, there were many Iraq trend factors that Michael O'Hanlon considered important enough to include is his verbal testimony and in the logic of thinking through what those factors meant, but which did not appear in the attached table of 30 factors. Our view is that if a metric is important enough to mention and use for logical argument, it is imperative that the details of how that factor varies over time are provided for further review and analysis by interested parties.
   
5. Quality of Metrics. As for the case of evaluating the Combine-Iraq-Trends document, all metrics, those that are in the table (and therefore charted) and those that are in the text must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in:

   The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”How
   

What next steps could help things along and help us move closer to our goal of helping both ordinary citizens and decision makers gain a better understanding of the trends at work on the ground in Iraq?

Answer: We've made some progress, but in many ways we have only just begun. Below, I outline some steps I am planning to take with the goal of taking this up to the next level. If you can assist in any way, I would be pleased to hear from you.

1. Continue to harvest and grow our list of missing metrics and invisible indicators, for example by close examination of Dept of Defense quarterly report or the many writings of Anthony Cordesman on the subject.
2. Create a single composite list of all the factors for which we do not yet have trend data or trend charts but which we think would further aid our understanding if they were to be made available.
3. Make inquiries to the original sources to see if the raw trend data for the key factors that have been identified can be made available for further use, ideally in a readily-reusable form to begin with.
   

Further Sources of Vital Iraq Trend Data

Thanks to a hat tip from a reader, here are a few more sources of vital online information about Iraq trend data that have recently come to my attention and that I find most relevant to the ongoing discussion.

First, there is the Dept. of Defense's quarterly report to Congress about the status in Iraq - Measuring Security and Stability in Iraq. The most recent report is dated November 2006. The report includes 9 trend charts and 1 trend table. Some of these charts are variants of trend charts we have already seen from our look at the Brookings Iraq Index , Michael O'Hanlon's testimony, GlobalSecurity and the Iraq Casualty Coalition . Others, however, are new and complementary to the charts we have seen so far. For example, there is a single chart combining three separate trends (average daily casualties for Iraqi Civilians, Iraqi Security Forces and Coalition Forces) and there is another chart showing "Ethno-Sectarian Incidents and Executions".

I have not yet had the chance yet to study the most recent Dept of Defense Quarterly Report in depth. My initial scan indicates that it includes mention of dozens of other important factors whose trend patterns might help us better understand the situation on the ground, but for which no trend data or trend graphics are presented. Among these missing metrics or what might also be called "invisible indicators" or "faceless factors" are many new metrics that are not included even in our list of more than 100 metrics that we have already mentioned in this series of posts. We have it on our to do list to see if in the next couple of weeks we can bubble up these missing metrics to add to our growing list.

Second, there is a powerful, provocative and quite critical report by Anthony Cordesman analyzing the Dept of Defense's Quarterly Measuring Security and Stability in Iraq report that the DoD issued in May 2006: The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”

Here's a brief quote from the introduction [emphasis added]:

The media can provide some of this picture, as can outside experts and scholars, but only the US government has the resources and access to information that provide a comprehensive overview of the situation. The quarterly report to Congress issued by the Department of Defense, “Measuring Stability and Security in Iraq,” is supposed to be a key document to achieve this goal.

Like the State Department weekly status report on Iraq, however, it is deeply flawed. It does more than simply spin the situation to provide false assurances. It makes basic analytical and statistical mistakes, fails to define key terms, provides undefined and unverifiable survey information, and deals with key issues by omission.

This entire report is well worth reading. It focuses on the reliability and validity of the trend data we are getting - a topic that we have not yet touched on in this report but which is of course highly relevant to everything we are looking at. It would be extremely valuable to repeat this kind of evaluation of the quality of data sources with the latest DoD quarterly report, with the weekly State Dept report, with the Brookings Iraq Index and with ALL the factors that we rate as important and relevant to understanding the situation on the ground in Iraq.

In my view Anthony Cordesman brings a level of rigor to this particular report that is lacking from the other sources we have so far examined and his approach would be well worth emulating going forward.

In the meantime, just because there are questions about the quality of many of the individual data factors, that doesn't mean we should stop doing what we've been doing. This does not mean that we should stop measuring and or that we should stop regularly updating our trend tables and trend charts or that we should stop searching for missing metrics and adding them to our list.

In fact, I think you will find that it is true that by being able examine many trend factors in a time efficient manner provides powerful checks and balances that can help us discover which factors may be most likely to suffer from quality problems.

If you know of other sources of vital Iraq trend data that we are missing, please let us know.

Taking Stock of the Recent Iraq Trends Posts: A demonstration of future possibilities

So far in our recent series of posts on trends on the ground in Iraq, we have created a number of trend data files (Iraq-RR-Example.csv and ohanlon-key-factors.csv) and trend visualization documents (Combine-Iraq-Trends and vizualizing-trends-ohanlon-testimony.pdf) that we hope have helped make the situation in Iraq more understandable, more quickly, to a wider number of individual citizens. If you want a quick refresher of how easy it is to look at more than 50 important trends in Iraq, just download the two pdf documents noted above and see what you think.

We believe that this approach results in radical time savings and increased in depth and breadth of coverage and therefore understanding.

We also hope that by taking these steps and preparing this demonstration of what other new and creative things might be done with the available trend data that this work will benefit even the key decision makers who are directly or indirectly involved in the day to day situation in Iraq and even those who are regularly spending a substantial portion of their time looking at and interpreting trend data.

We would enjoy hearing your comment and feedback on whether you think this demonstration of the possibilities represents a path worth pursuing further.

Here's a more detailed summary of what we have covered and accomplished so far in this series of posts that we began on January 13th.

1. we pulled together a series of charts about key trends in Iraq drawn from several important source documents and web sites and we have made this whole set of charts more readily viewable by assembling them into a single downloadable pdf document - Combine-Iraq-Trends.
   
2. In this January 18th post we showed how we could combine raw tabular trend data from several different sources and paste it into a single readily-reusable (RR) csv file. This meant we could then easily calculate and visualize some new, important, and previously invisible trends such as the wounded to killed ratio or the three month moving average for troop fatalities and we could save the original combined trend data with the new calculated trend factors together in a single csv file: Iraq-RR-Example.csv that others could then examine and use directly.
3. We then took a more elaborate case and showed how straightforward it was to transform a table of 30 key factors from Michael O'Hanlon's testimony into a readily reusable trend data file - ohanlon-key-factors.csv. - and then use that file to generate a series of easily viewable trend charts - vizualizing-trends-ohanlon-testimony.pdf.
   
4. And finally, in our most recent posts, we demonstrated that even though we had already pulled together and presented more than 50 different trend charts, that there were still rather important factors, whose trends we had not yet looked at. In this post yesterday, we listed more than 20 important factors that were mentioned as part of the verbal argument presented by Michael O'Hanlon in his testimony, but which did not show up in his table of trend factors. And in our most recent post today, we showed how a close reading of the Brookings Iraq Index allowed us to discover more than 50 important additional factors that could help us imagine a fuller picture of what is going on in Iraq.

If you want to read the whole series, the following link will take you to the listing for the January 2007 posts: Change-over-time-January-2007.

With so many factors to consider, you might feel this is daunting. In upcoming posts we will directly address any fears you might have that handling 100 or more factors might be too difficult or beyond your reach and show you how the approach of using Readily Reusable data combined with time saving tools of which TLViz and CSVPNG are useful models can save the day.

We will also be taking a look at what we might do next to make the trend data coming out of Iraq even more useful and valuable for both ordinary citizens and for decision makers.

More Invisible Metrics: What other trends is it vital that we make readily resuable and visualizable

So far in this series of posts about better understanding the situation on the ground in Iraq, we have focused on bringing together important trends and presenting them graphically in ways aimed at making this vital information more readily available and usable to both decision makers and to ordinary citizens.

Our short term goal has been to radically reduce each viewer's cost in time spent (compared to previous formulations) while at the same time sharply increasing the depth and breadth of understanding attained.

Our longer term goal is for this increased level of understanding (now achievable and within reach of a much larger number of individuals) to be the catalyst that spurs better discussion, improved collaboration, and much deeper thinking about what is really going on, what it means, and what actions might lead to improved results in the future.

In our next post, we will give a wrap up the ground we have covered so far in this series. For now, we have some unfinished business, namely that there are many other vital Iraq trend factors that are necessary for our more complete understanding of the situation. So continuing with the list of missing factors from our previous post, below we highlight an additional 55 factors that we discovered and selected through a close reading of the Brookings Iraq Index.

In our opinion, being able to see the trend graph for each of these would add to our overall understanding of what is going on in Iraq. Admittedly, not all of these are of equal importance but the trend for each is likely to have an important story to tell if we assemble the necessary trend data and then create the chart.

1. Percentage of US troop fatalities caused by car bombs (page 3 table)
2. Iraqi civilians killed by US troops (page 9 table)
3. Police deaths in Baghdad (page 12 table)
4. Civilian deaths in Baghdad (page 12 table)
5. Kellogg, Brown and Root Employees in Iraq (table page 14)
6. Combat arms troop strength (table page 14)
7. Combat support troop strength (table page 14)
8. Combat service support troop strength (table page 14)
9. Dept of Defense Civilians in Iraq (table page 14)
10. Total Civilian Personnel in Iraq & Kuwait (table page 14)
11. Iraqis Kidnapped Nationwide (table page 15)
12. Iraqis Kidnapped in Baghdad (table page 15)
13. Iraqi prison population in US Custody (table page 16)
14. Iraqi prison population in Iraqi Custody (table page 16)
15. Active duty troop strength in Iraq (table page 20)
16. Reserve & National Guard troop strength in Iraq (table page 20)
17. Total US Army authorized strength - Activity duty (table page 20)
18. Total US Army authorized strength - National Guard (table page 20)
19. Total US Army authorized strength - Reserve (table page 20)
20. Total US Army authorized strength - Combined Active, National Guard & Reserve (table page 20)
21. Percentage of IEDs found and disarmed (table page 23)
    
22. Number of daily Iraqi Patrols (chart page 23)
23. Number of daily US Patrols (chart page 23)
24. Number of daily Joint Iraqi-US Patrols (chart page 23)
25. Number of total daily patrols (chart page 23)
26. New passports issued (table page 24)
27. Percent of professional class that has left since 2003 (table page 24)
28. Iraqi refugees heading to Syria per day (table page 24)
29. Iraqi refugees heading to Jordan per day (table page 24)
30. US Air Strikes (Iraq only) (table page 26)
31. C-130 Sorties flown (Iraq and Afghanistan combined) (table page 26)
32. Diesel production (table page 29)
    
33. Kerosene production (table page 29)
34. Gasoline/Benzene production (table page 29)
35. LPG production (table page 29)
36. Non oil related GDP (table page 33)
37. Total cost of US operations (table page 37)
38. Cost of US defense activitites (table page 37)
39. Cost of Veteran's programs (table page 37)
40. Inflation rate (table page 39)
41. Cost of imported gasoline (table page 39)
42. Domestic price of gasoline (table page 39)
43. Relative amount of car traffic (table page 40)
44. Typical waiting time for gasoline (table page 40)
45. Number of registered cars (table page 40)
46. Internet subscribers (table page 41)
47. Tons of mail sent by Iraqis (table page 41)
48. People with potable water (table page 42)
49. People with sewerage system coverage (table page 42)
50. Water treatment capacity (table page 42)
51. Annual Iraqi Medical School Graduates (table page 42)
52. Primary school enrollment percentage (table page 43)
53. High School enrollment percentage (table page 43)
54. Percentage of Iraqis who approve of attacks on US led forces (table page 44)
    
55. Percentage of Iraqis who strongly oppose presence of Coalition troops (table page 51)

Our list of important metrics is getting longer and longer. But this is not all there is.

What other factors do you think are important? What else is missing? What else would you like to know? What else would you like to see? Do any of these factors trigger ideas about other important factors that were not even mentioned at all? Do combination of factors come to mind for you that you think would shed even more light?

And finally, of the 100 or more factors that we have touched on in this series of posts, what is your short list of the top 10 or 20 most important ones?

Missing Metrics - What else do we need to know about how things are going in Iraq?

In looking closely at the text of Michael O'Hanlon's testimony to the Senate Foreign Relations Committee on January 10th, 2007, I noticed that he spoke of quite a few Iraqi factors on the ground that were NOT included in the list of 30 metrics reported.

A small number of the factors that showed up in the tabular data were mentioned in the logic of the text, but I was surprised by how many important factors from the text had no numerical equivalent in the table.

Here's a list of the missing factors that I found in the text with my comments in brackets.

Good year/Bad Year - <I would love to know more about what key factors Michael incorporates in his thinking in coming up with this assessment and would find it enlightening if this black and white measurement were refined and reported on say a scale from 1 to 10 or 1 to 100>

“The year 2006 was, tragically and inescapably, a bad one in Iraq.”

Sense of Progress

“The first is the failure of the 2005 election process to produce any sense of progress.”

Al-Maliki Favorabaility Rating

“In a September poll, Prime Minister al-Maliki was viewed unfavorably by 85 percent of all Sunni Arabs”

Rate of Shia attacks on Sunnis

Rate of Sunni attacks on Shia

Increasing politicization of the violence “to include many more Shia attacks on Sunnis as well as the reverse”

Iraqi state subsidies for consumer goods

“recent reductions in Iraqi state subsidies for consumer goods (which distort the economy and divert government resources)”,

The number of children being immunized.

Percentage who feel safe en route to school

Rate of private sector investment

“further slide toward chaos.”

Percent who think Economy is Poor

According to a June, 2006 poll, 59 percent call the economy poor

Percent who think security environment is poor

“and 75 percent describe the security environment as poor”.

Rate of Jobs Creation

“a massive program to create jobs”

Degree of Iraqi Leader consensus on sharing oil

“an ultimatum to Iraqi political leaders that if they fail to achieve consensus on key issues like sharing oil”,

“American support for the operation

could very soon decline”

Number of coalition forces needed to provide security.

“Coalition forces have never reached the numbers needed to provide security for the population in Iraq”

Degree of Overwork of our Soldiers and Marines

“our already overworked soldiers and Marines”.

Rate of “Success of military commanders in putting Iraqis to work

with their commander emergency response program funds”,

Degree of Support Iraqi Leaders enjoy outside their own sectarian group.

“it is clear that extremely few Iraqi leaders enjoy any real support outside of their own sectarian group.”

Degree to which Iraqi Leaders are working across sectarian lines.

“Trying to force them to work across sectarian lines must be a focus of our policy efforts, if there is to be any hope of ultimate stability in Iraq.”

The Probability that a Given Intervention will succeed

“Social scientists and military experts do not know how to assess rigorously the probabilities that such steps will succeed at this late hour in Iraq.” [Emphasis added]

Degree of Pessimism

“Overall, however, it seems fair to say that most have become quite pessimistic.”

Iraqis displaced per month

“But with 100,000 Iraqis per month being displaced from their homes, making for a total of some two million since Saddam was overthrown, ethnic cleansing is already happening.”

Bottom Line: If we want to understand what's going on in Iraq, we are going to have to work harder at bringing these missing metrics out in the open where every citizen can see them, think about them, discuss them.

Tapping the Power of Iraq Readily Reusable Data

In the previous post, we showed how the tabular data from Michael O'Hanlon's testimony to the Senate Foreign Relations Committee could be converted to a Readily Reusable (RR) format as a CSV file ( ohanlon-key-factors.csv ) and how the RR format then made it relatively straightforward and inexpensive to actually look at the visualization of all thirty of the reported trends, one by one as shown in vizualizing-trends-ohanlon-testimony.pdf.

We have found that when using RR data and TLViz, the time savings we achieve make it possible to look at hundreds of trends, one after the other, in a very short time so as to gain a gestalt sense of all the reported factors at work. Typically, moving from tabular form to RR form gives a productivity saving factor of at least 10 to 1. Frequently, the productivity increase is 50 to 1 or more.

After having reviewed each of the 30 reported trend factors from Michael O'Hanlon's testimony, it became apparent that there were some other interesting trends hiding amongst the original data that could be computed with simple calculations from the original data. This is readily achievable by opening the RR csv file with a tool such as Microsoft Excel.

For example, with Excel we could create a new column of data and combine US troop strength and troop strength of other non-US coalition forces and then calculate the percentage of coalitions forces that were non-US. Similarly, we could use the original data to calculate the percentage of US troops that had been killed in that month by IEDs. We could also take factors such as the cumulative number of refugees and convert them into year over year trend data.

Below, we show you the three new trend charts that bring some previously invisible trends into the light of day where we can all see them. (please click on trend graphic for full size image.)




Bottom line. Once the trend data is in RR format, all sorts of new possibilities of understanding open up that can help us better understand what is really going on.

Visualizing the O'Hanlon Trend Data

Once we had the the data from Michael O'Hanlon's Senate Foreign Relations Committee Testimony (Jan 10, 2007) in readily reusable (RR) form, we opened that csv file with TLViz and were able to generate a set of graphs in a matter of minutes showing the individual trends in visual form including some that combined several of the metrics together.

You can see a pdf slide show of the entire set of charts we created in full screen size by clicking on: vizualizing-trends-ohanlon-testimony.pdf

Using Edward Tufte's idea of small multiples, we have combined 4 trends at a time into the pictures below (please click for full size) that show a sample of the charts from the full report.

The first 4 show US troops killed by IED, Iraq Civilian Fatalities, Multiple Fatality Bombings, and Estimated Strength of the Insurgency.

The next 4 trends show Estimated Strength of Shia Militias, Daily Average of Inter-ethnic attacks, Estimated Number of Foreign Fighters, and Number of Daily Attacks by Insurgents or Militias.



The next four small multiples show Iraqi Internally Displace, Iraq Refugees, Iraqi Physicians Murdered or Kidnapped, and Iraqi Physicians who have fled Iraq.


In this next set of trends, the first chart shows the total number of Iraqi Physicians who have murdered, kidnapped, or who have left Iraq. This chart was created by combining two of the originally reported trends. The second chart also combines two original trends, this time comparing the number of Iraqi forces who are technically proficient to the number who are politically dependable. The final two charts in the small multiple set show the Percentage of Household Fuel needs that are being met and the average hours per day of electricity in Baghdad.


This final small multiple set shows the Unemployment Rate, the Per Capita GDP, the Number of Trained Judges, and the Number of Telephone Subscribers.


If you click on these charts and then press F11, you should have a full screen readable version to examine. Of course, if you want to look at each chart by itself in full screen size with easy back and forth navigation, the vizualizing-trends-ohanlon-testimony.pdf is the place to go.

One of the advantages of these charts to the previous set of combined charts that we published is that they all cover the same time period and that makes understanding what they mean just that much easier. Once you understand what the X axis timeline means in one chart, you don't have to recompute that with each chart you examine.

Once again, we suggest that you download this RR data yourself and play with it to see what else you can find, what other patterns are hiding in the data. You can download by right clicking on ohanlon-key-factors.csv. We also suggest that this could be a fine opportunity for you to test out the ways in which TLViz simplifies and speeds up trend analysis work once you have RR data. You can learn more about TLViz from our earlier posts this month and you can download a copy of the full kit for Windows PC's from TrendsThatMatter's Download Page