In the two previous posts, we have spoken to the concerns of Rep Kingston and Michael O'Hanlon as captured in the Sheryl Stolberg New York Times See You in September ... article.
To be successful in understanding what is happening in Iraq, we need to listen to both their voices.
A Sufficient Number of Factors. We need to pay heed to O'Hanlon and make sure we have looked at enough different factors.
Manageable Metrics. We have to hear Rep. Kingston's concern and worry about being overwhelmed with too much data or rendered ineffective by not having enough time by making whatever data we report on "Manageable" within the constraints of the time and resources that are available to us.
Eating our Cake. It is possible to meet both of these goals. We are not doing it today, but we are surrounded daily with visible proofs that equally difficult trending situations (see the St. Louis Fed Reserve FRED application as an example of what's possible) can be brought within our grasp and control by the judicious application of 21st century Trend Visualization methods.
Along the Way. Paying attention to our view of the best practices and principles of trend graphic excellence will provide the third ingredient that helps make our efforts in this arena more effective and more readily sharable with others. For a discussion of these, check out the Trend Visualization Principles displayed near the top of the right column of this blog or follow this link to trend principles for a review of some of our other thoughts on this subject.
Showing posts with label Michael O'Hanlon. Show all posts
Showing posts with label Michael O'Hanlon. Show all posts
Thursday, June 14, 2007
Dangerous to have too few metrics or the wrong metrics
In a previous post: Revisiting "See You In September" I noted:
Dangerous to Rely on Too Few of Them. I agree entirely. For Iraq, we need to be able to look at at least 50-100 of the most important factors efficiently, all within a relatively small period of time. For expert use, a much higher number of metrics must be available for review in an efficient and effective manner. Michael O'Hanlon's weekly Iraq Index is one of the best examples out there that pays attention to the principle of not relying on too few metrics.
The Metrics were the Wrong Ones. This is a solvable challenge this time around. First, follow the principle above and make sure that we look at ALL of the important factors. How do we decide? We ask all the experts in all the different areas (security, economy, health, ...) to name what they consider to be the most important indicators. Then we capture and archive and report on all the factors that have been suggested, even if only once. Maybe there will be some "wrong ones" in the mix, but when we have the full set of data to work with, the chances of being led astray will be markedly reduced.
Can’t be exactly precise about which indicators are the conclusive ones. We don't have to be precise. Different people will have different opinions. If we capture all of them, record a history of each trend, and make this data available in Static Graphical format, AND as machine readable data suitable for input into a Trend Visualization application, then we can begin to have good and constructive discussions about what different trends mean and what we should do about them.
There is a way forward that addresses Michael O'Hanlon's concernsThe relevant quotes from O'Hanlon were:
- "metrics are grist for a fact based debate but history shows it is dangerous to rely on too few of them"
- "metrics were used in Vietnam and we had the wrong ones"
- [the Vietnam metrics] "did net harm to the debate"
- "we can’t be exactly precise about which indicators are the conclusive ones”
Dangerous to Rely on Too Few of Them. I agree entirely. For Iraq, we need to be able to look at at least 50-100 of the most important factors efficiently, all within a relatively small period of time. For expert use, a much higher number of metrics must be available for review in an efficient and effective manner. Michael O'Hanlon's weekly Iraq Index is one of the best examples out there that pays attention to the principle of not relying on too few metrics.
The Metrics were the Wrong Ones. This is a solvable challenge this time around. First, follow the principle above and make sure that we look at ALL of the important factors. How do we decide? We ask all the experts in all the different areas (security, economy, health, ...) to name what they consider to be the most important indicators. Then we capture and archive and report on all the factors that have been suggested, even if only once. Maybe there will be some "wrong ones" in the mix, but when we have the full set of data to work with, the chances of being led astray will be markedly reduced.
Can’t be exactly precise about which indicators are the conclusive ones. We don't have to be precise. Different people will have different opinions. If we capture all of them, record a history of each trend, and make this data available in Static Graphical format, AND as machine readable data suitable for input into a Trend Visualization application, then we can begin to have good and constructive discussions about what different trends mean and what we should do about them.
Tuesday, June 12, 2007
Revisiting "See You In September"
I want to return to the Sheryl Gay Stolberg article
Following along the lines of the Forecast on Iraq post we'll once again de-construct and highlight some of the language and see what else that reveals. For fun, I have color coded the quoted text as follows.
Orange - vague metrics (e.g. progress, confidence, winning, losing)
Purple - vague quantification (e.g. up, down, better, more)
Green - specific metrics (e.g. sectarian murders, monthly bomb rate). Unfortunately, none of these have any specific numbers associated with them. Some have vague quantification and others have non at all.
The language we use to talk about the important aspects of our lives makes a difference. When we use language about an evolving situation this way (vague metrics coupled with imprecise quantification, and lack of any clear trend quantification and visualization), we are going to have a tough time making sense of what's actually happening, what it means, and what our best next step might be.
If we continue along this path of mushy metrics, vague quantification, and numerous instances of what appears to be learned helplessness, come September, we won't be in any better state to evaluate the situation than we are now
I have some more thoughts on some of the key quotes from above for a future post. There is a way forward that addresses Michael O'Hanlon's concerns and answers Rep. Kingston's call for assistance.
See You in September, Whatever That Meanspublished in the New York Times on May 13th, 2007. There are many more important clues we can discover hiding the text. I did not have time to unravel them all during the first Charting Progress in Iraq post on this subject.
Following along the lines of the Forecast on Iraq post we'll once again de-construct and highlight some of the language and see what else that reveals. For fun, I have color coded the quoted text as follows.
Orange - vague metrics (e.g. progress, confidence, winning, losing)
Purple - vague quantification (e.g. up, down, better, more)
Green - specific metrics (e.g. sectarian murders, monthly bomb rate). Unfortunately, none of these have any specific numbers associated with them. Some have vague quantification and others have non at all.
- "decisive progress"
- "looking better"
- “Sectarian murders are down”
- "car bombings ... are up"
- "deaths of American soldiers are up"
- "some in Washington are grasping for a more complete and accurate way to quantify progress"
- “No one knows how to define progress in such a mixed-up situation”
- “We’re having trouble measuring it."
- [Iraq Index includes] "the monthly car-bomb rate"
- [Iraq Index includes] "how many foreign nationals are kidnapped"
- [Iraq Index includes] "how many Iraqis have electricity"
- [Iraq Index includes] "how many Iraqis have ... Internet access"
- [the Iraq index] "is long on numbers and short on analysis"
- "some signs of hope"
- "more grounds for worry than for confidence"
- [Rep Kingston (R-Georgia) wants to] "winnow down the indicators to a manageable number -- say, fewer than a dozen"
- [Rep Kingston (R-Georgia) wants to create] "a standard bi-partisan metric"
- [with such a metric in place] "then you could say who’s winning and losing”
- [Micheal O'Hanlon says] "metrics are grist for a fact based debate but history shows it is dangerous to rely on too few of them"
- [Micheal O'Hanlon says] "metrics were used in Vietnam and we had the wrong ones"
- [the Vietnam metrics] "did net harm to the debate"
- [Micheal O'Hanlon says] "we can’t be exactly precise about which indicators are the conclusive ones”
- "such an index would be politically unpalatable to the White House, which does not want to back itself into a corner by agreeing to someone else’s standard for progress"
- "the White House says the only progress report that counts is the one from Gen. David H. Petraeus, the new top commander in Iraq, and Ryan Crocker, the new ambassador"
- "is the current strategy for waging war going well or not?"
- [Stephen Biddle] did fault the White House for not being more open with the public about its own idea of what constitutes progress
- “By being unbelievably vague about everything,” he said, “they’re making it very hard for congressmen and senators to go to their constituents and say, ‘Look, here’s why things are going better than you might imagine.’ ”
- "Some say measuring progress is simple: you will know it when you see it."
- [Metrics suggested by Senator Robert Bennett (R-Utah)] I want to see life starting to come back,” “I want to see people in markets. I want to see couples strolling down the street, folks sitting at outdoor cafes.”
- [Metrics suggested by Senator Susan Collins (R-Maine) ] “a significant reduction in violence and attacks accompanied by a transfer of more and more authority to the Iraqi forces.”
- [Senator Collins] “the difficult question is going to be if the analysis is mixed, and I suspect it may well be.
- [Senator Lamar Alexander (R-Tennessee) says] “I don’t see any way for us to maintain a long-term presence in Iraq without more bipartisan support.”
- [Rep. Kingston] "hopes to come up with some useful way of figuring out ...whether [to] remain supportive for a little while longer"
- [Rep. Kingston has] " heard three years of nearly happy-talk in testimony"
- “We always seem to be about to be around this elusive corner, but we never get there.”
The language we use to talk about the important aspects of our lives makes a difference. When we use language about an evolving situation this way (vague metrics coupled with imprecise quantification, and lack of any clear trend quantification and visualization), we are going to have a tough time making sense of what's actually happening, what it means, and what our best next step might be.
If we continue along this path of mushy metrics, vague quantification, and numerous instances of what appears to be learned helplessness, come September, we won't be in any better state to evaluate the situation than we are now
I have some more thoughts on some of the key quotes from above for a future post. There is a way forward that addresses Michael O'Hanlon's concerns and answers Rep. Kingston's call for assistance.
Thursday, June 7, 2007
Why Wait Till September in Iraq
We Don't Have To Wait. There's a dangerous meme circulating that says we have to wait till September before we can learn whether the "surge" in Iraq is working. This, in my opinion is simply not true.
Many dimensions of the current situation in Iraq can and are being measured and reported on in near real time. The work of Michael O'Hanlon on the weekly Iraq Index at the Brookings Institution is one excellent proof that it is possible to know were we stand literally week by week and even day by day.
Digestible Trend Data. One principal area where we are not as good as we need to be today is in making sure that the trend data that exists is readily available in a form that is digestible by the intended audience. This intended audience includes the Congress, the main stream news media, the blogosphere, and especially ordinary citizens. Some really important data such as the example of Iraqi prison population that we showed in a post earlier today may only appear in tables. My guess is that most people are not willing or able to get what they need directly from the Iraq Index or similar reports. Those who are willing and able may not have nearly enough time. We can and must make it easier so that more citizens can be included in the process.
Assembling All Key Factors in One Place. A second principal impediment is that that the data is dispersed among many sources. We need to pull it all together so that each of us then becomes able to make a reasoned evaluation of how we are doing right now (without waiting till September) . My efforts at assembling some key trend graphs in a series of posts today and in the summary zip file is a small demonstration of what is possible.
Identifying and Acquiring Any Missing but Still Important Factors. A third thing holding us back from being able to evaluate for ourselves how things are going right now is that some key factors are not being reported in easily findable places.
Twenty First Century Visualization Tools. The fourth factor limiting our possibilities today is that most trend data is presented statically. This severely limits the number of factors that an ordinary person is going to be willing and able to look at to only a handful. On the other hand, with a modern Trend Visualization Viewing Station on the order of what's already available for stock market data from the likes of Big Charts or Prophet Net means that ordinary folk can look at and understand dozens and dozens of factors in a very short space of time.
We Don't Have to Wait. All these difficulties that are blocking us can be remedied without waiting till September. Our failure to remedy them over the past four years meant that it took that long for it to sink in that what we were doing wasn't really working. It's easy to posture that everything is ok when there is no hard data to prove you wrong.
In fact, if we don't begin to remedy them now, we will be in the same boat come September. We will find that we still do not have our arms around the entire situation. The facts we need will still not have been assembled and reported in an easily digestible way. And it will still be way too easy to claim that things are going well, or that we still don't know enough to tell.
Anyone interested in helping blast our way through these obstacles to understanding where we are in near real time, please let me know so we can join forces and see if we can make a difference.
The Data is Out There. Let's go get it and assemble it and make it easy to use and then share it with all interested parties.
Many dimensions of the current situation in Iraq can and are being measured and reported on in near real time. The work of Michael O'Hanlon on the weekly Iraq Index at the Brookings Institution is one excellent proof that it is possible to know were we stand literally week by week and even day by day.
Digestible Trend Data. One principal area where we are not as good as we need to be today is in making sure that the trend data that exists is readily available in a form that is digestible by the intended audience. This intended audience includes the Congress, the main stream news media, the blogosphere, and especially ordinary citizens. Some really important data such as the example of Iraqi prison population that we showed in a post earlier today may only appear in tables. My guess is that most people are not willing or able to get what they need directly from the Iraq Index or similar reports. Those who are willing and able may not have nearly enough time. We can and must make it easier so that more citizens can be included in the process.
Assembling All Key Factors in One Place. A second principal impediment is that that the data is dispersed among many sources. We need to pull it all together so that each of us then becomes able to make a reasoned evaluation of how we are doing right now (without waiting till September) . My efforts at assembling some key trend graphs in a series of posts today and in the summary zip file is a small demonstration of what is possible.
Identifying and Acquiring Any Missing but Still Important Factors. A third thing holding us back from being able to evaluate for ourselves how things are going right now is that some key factors are not being reported in easily findable places.
Twenty First Century Visualization Tools. The fourth factor limiting our possibilities today is that most trend data is presented statically. This severely limits the number of factors that an ordinary person is going to be willing and able to look at to only a handful. On the other hand, with a modern Trend Visualization Viewing Station on the order of what's already available for stock market data from the likes of Big Charts or Prophet Net means that ordinary folk can look at and understand dozens and dozens of factors in a very short space of time.
We Don't Have to Wait. All these difficulties that are blocking us can be remedied without waiting till September. Our failure to remedy them over the past four years meant that it took that long for it to sink in that what we were doing wasn't really working. It's easy to posture that everything is ok when there is no hard data to prove you wrong.
In fact, if we don't begin to remedy them now, we will be in the same boat come September. We will find that we still do not have our arms around the entire situation. The facts we need will still not have been assembled and reported in an easily digestible way. And it will still be way too easy to claim that things are going well, or that we still don't know enough to tell.
Anyone interested in helping blast our way through these obstacles to understanding where we are in near real time, please let me know so we can join forces and see if we can make a difference.
The Data is Out There. Let's go get it and assemble it and make it easy to use and then share it with all interested parties.
Slide Show Viewing of Today's Trends
We've covered quite a bit of ground today by reviewing several important documents that have included graphical representations of the trends at work in Iraq today.
We looked at:
down the road, I will be exploring what other blog posting capabilities are available that will let me present the data laid out the way I want it. In the meantime, I have assembled all 26 jpg and gif charts into a single zip file that I posted here in my data download area.
Once you unzip to a directory of your choice, you can enjoy a full size slide show with your favorite slide viewer - for example, Microsoft Office Picture Manager or IRFANVIEW.
I'd like to hear back whether you think this represents a useful, time saving approach for you. I am interested in learning whether you find having all the trend graphs assembled in one place makes reading them and understanding them more rapid than when looking at the individual reports. And finally, I would enjoy discovering how much value you think is added by combining charts from a wide range of resources and pulling them together in one place.
We looked at:
- The Special Inspector General for Iraq Reconstruction's April 2007 Report to Congress
- The Dept. of State's Iraq Weekly Status Report for May 30th, 2007
- A Matthew Yglesias blog post on Electricity goals from Dept of State's December 2004 Iraq Weekly Status and compared it to the most recent data on the same electricity factor from the most recent report.
- The most recent Iraq Index from Michael O'Hanlon at Brookings Institution
- Additional troop strength charts created from Tabular data in the Iraq Index report
down the road, I will be exploring what other blog posting capabilities are available that will let me present the data laid out the way I want it. In the meantime, I have assembled all 26 jpg and gif charts into a single zip file that I posted here in my data download area.
Once you unzip to a directory of your choice, you can enjoy a full size slide show with your favorite slide viewer - for example, Microsoft Office Picture Manager or IRFANVIEW.
I'd like to hear back whether you think this represents a useful, time saving approach for you. I am interested in learning whether you find having all the trend graphs assembled in one place makes reading them and understanding them more rapid than when looking at the individual reports. And finally, I would enjoy discovering how much value you think is added by combining charts from a wide range of resources and pulling them together in one place.
Iraq Index from Brookings Inst.
Here is a sampling of what I think are the most important trend charts from the Brookings Institution's June 4th edition of Michael O'Hanlon's Iraq Index. This weekly report is still covers the widest range of Iraq trend metrics in a single place that I know of. Many of these are displayed in trend charts. Other important metrics appear in tabular form or in text description. Check out the whole report (60+ pages) for all the details.
Chart 1 - Iraqi Military and Police appear to be becoming more involved in recent months based on this chart showing their fatality rate rising.
Chart 2 - Slight downturn overall recently but still high. I would prefer to see these three factors separately so I could discern individual trends. Would also be useful to be able to normalize a given category's rate as a percentage of the total rate of attack. Rich data such as these are good when presented statically, but even better when there is a chance to interact with them (e.g. as we did in yesterday's post showing the Wall Street Journal's interactive chart on housing trends).
This chart shows the recent rapid ramp up in security stations and outposts
Stacked bars can be interesting. In addition, it regularly proves helpful to be able to examine the individual components.
This data looks like it is a very important dis-aggregation that will bring added insight to what's going on. It would be great if the data from previous years could be reconstructed and added to give more perspective.
This important chart covers the full period from the beginning of the war. My own preference would be to drop the labels with the totals for each month. I think these clutter up the chart and would be best to keep in a table for those who are interested. Since the month to month is so variable, a 3 or 4 month moving average might be a useful addition. My own preference would be to plot the non-hostile fatalities separately for greater clarity regarding any trends that might be present in that data. The disparity of scale between the two factors makes interpretation of the smaller values more difficult.
Another important chart which also covers the full time range since the beginning of the war. I believe a moving average would also help present this data more thoroughly.
While his data on the growing prison population is important and seriously under-reported elsewhere, it only appears as a table in this month's report. A chart for this factor would prove even more helpful. In addition, it would be useful to know the trends in other related criminal justice factors such as:
Chart 1 - Iraqi Military and Police appear to be becoming more involved in recent months based on this chart showing their fatality rate rising.
Chart 2 - Slight downturn overall recently but still high. I would prefer to see these three factors separately so I could discern individual trends. Would also be useful to be able to normalize a given category's rate as a percentage of the total rate of attack. Rich data such as these are good when presented statically, but even better when there is a chance to interact with them (e.g. as we did in yesterday's post showing the Wall Street Journal's interactive chart on housing trends). 
This chart shows the recent rapid ramp up in security stations and outposts
Stacked bars can be interesting. In addition, it regularly proves helpful to be able to examine the individual components.
This data looks like it is a very important dis-aggregation that will bring added insight to what's going on. It would be great if the data from previous years could be reconstructed and added to give more perspective.
This important chart covers the full period from the beginning of the war. My own preference would be to drop the labels with the totals for each month. I think these clutter up the chart and would be best to keep in a table for those who are interested. Since the month to month is so variable, a 3 or 4 month moving average might be a useful addition. My own preference would be to plot the non-hostile fatalities separately for greater clarity regarding any trends that might be present in that data. The disparity of scale between the two factors makes interpretation of the smaller values more difficult.
Another important chart which also covers the full time range since the beginning of the war. I believe a moving average would also help present this data more thoroughly.
While his data on the growing prison population is important and seriously under-reported elsewhere, it only appears as a table in this month's report. A chart for this factor would prove even more helpful. In addition, it would be useful to know the trends in other related criminal justice factors such as:- number in custody who have been charged and indicted
- number currently undergoing trial
- number found guilty, sentenced and serving time
- number awaiting indictment
- number awaiting trial
- average number of months before indictment/trial
Another full time period chart. The pair of factors and the dual y axis work well together in this case. My view is that a moving average would be helpful in this situation as well.
Iraq Weekly Status Report May 30, 2007
Here are some charts taken from the Department of State's May 30th, 2007 Iraq Weekly Status Report.
This first chart covers from January 2004. It appears as if demand is rising while production is falling and cannot seem to crack the 100,000 mark.
Also notice the sharp downward spikes for the light blue daily supply data. This tells me that it would be worthwhile to be able to zoom in on the supply factor and look at how supply is changing over time but using a much higher resolution than daily. An hourly trend chart for month's worth of data, or a daily chart using a 1 minute resolution would surely reveal some insights not currently available.
Similarly, when an aggregate trend factor show such sharp variation, it is frequently the case that creating additional trend charts that dis-aggregate the data can prove most helpful in understanding more fully what is happening.
This second chart makes it clear that the Iraqi currency has strengthened relative to the dollar over the past 12 months.
This third chart clearly shows the rising trend in Iraqi Commercial Bond interest rates over the past year.
This fourth chart suffers from showing far too brief a time period of only the most recent 10 weeks. Contrast this to the Oil Production chart in the previous post that showed the past 4 years and also provide useful trend data for production from the North and from the South.
This chart also has a much short sweep of time than needed for understanding the underlying trends. The related chart in the previous post showed refinery trends going back 4 years. This chart does have the positive feature that it groups all the bars for a given type of refined product together which is an improvement over the way the bars were used in the previous post from the Reconstruction report to Congress.
It's curious to note that there are no trend charts in this report from the State Department that show trends related to the first two "key areas identified as pillars of US government policy for victory in Iraq." Namely,
This first chart covers from January 2004. It appears as if demand is rising while production is falling and cannot seem to crack the 100,000 mark.
Also notice the sharp downward spikes for the light blue daily supply data. This tells me that it would be worthwhile to be able to zoom in on the supply factor and look at how supply is changing over time but using a much higher resolution than daily. An hourly trend chart for month's worth of data, or a daily chart using a 1 minute resolution would surely reveal some insights not currently available.
Similarly, when an aggregate trend factor show such sharp variation, it is frequently the case that creating additional trend charts that dis-aggregate the data can prove most helpful in understanding more fully what is happening.
This second chart makes it clear that the Iraqi currency has strengthened relative to the dollar over the past 12 months.
This third chart clearly shows the rising trend in Iraqi Commercial Bond interest rates over the past year.
This fourth chart suffers from showing far too brief a time period of only the most recent 10 weeks. Contrast this to the Oil Production chart in the previous post that showed the past 4 years and also provide useful trend data for production from the North and from the South.
This chart also has a much short sweep of time than needed for understanding the underlying trends. The related chart in the previous post showed refinery trends going back 4 years. This chart does have the positive feature that it groups all the bars for a given type of refined product together which is an improvement over the way the bars were used in the previous post from the Reconstruction report to Congress.
It's curious to note that there are no trend charts in this report from the State Department that show trends related to the first two "key areas identified as pillars of US government policy for victory in Iraq." Namely,
- "Defeat the terrorists and neutralize the insurgents"
- "Transition Iraq to security self reliance."
Thursday, May 3, 2007
Latest Brookings Iraq Index - Key Factors for Gauging the Success of the Surge
Here's the link that will always get you to the latest edition of the highly useful Brookings Institution Iraq Index that is one of the best places that I know of for learning about the trends on the ground in Iraq.
In the April 30th, 2007 edition, there is a useful essay on "Tracking the Surge" on page 4.
Unfortunately, the charts and tables that follow for the most part do not match up with the key factors noted in the essay, nor do they make it easy to for interested citizens to see the trends at work for themselves.
Here are some of the key factors that I gleaned (scraped) from the text.
And, reviewing the whole report, there are literally dozens of other important factors that show up in charts or tables.
Wishlist 1: It would be really valuable if one could turn the page of the Iraq Index report and see the chart or charts for each of the these items one by one, in the same sequence as they appeared in the essay and with the same terminology, so as to get a sense for the degree of progress that is being made, to see which things are lagging, to see what is getting worse and by how much, and so on.
This, in my mind would provide a giant step upward in the usability and the understandability of the important work that provides the basis for these weekly reports.
Comment: Many of the tables and charts in the weekly report are out of date. They represent important factors but no new data has been available to update them for quite some time.
Wishlist 2. It would be helpful if outdated tables and charts were either placed in an appendix, or if the chart could be updated to show the current end date and the missing data. Otherwise it is easy to misread the right hand side of the charts to think that the data and the trends represent the past few weeks or months instead of sometime in June 2006, or May 2005.
Comment: The chart formating for just about every chart is different and each chart seems to have its own unique starting and ending time intervals.
Wishlist 3. A move to more consistent chart formats and time intervals would greatly enhance the readability and usability of these reports.
In the April 30th, 2007 edition, there is a useful essay on "Tracking the Surge" on page 4.
Unfortunately, the charts and tables that follow for the most part do not match up with the key factors noted in the essay, nor do they make it easy to for interested citizens to see the trends at work for themselves.
Here are some of the key factors that I gleaned (scraped) from the text.
- number of US brigades in place
- number of "joint security stations" established
- extra-judicial killings
- level of civil warfare
- willingness of Shia militias to lie low
- willingness of Sunni tribal leaders in al-Anbar to collaborate in opposing al Qaeda
- level of violence in al-Anbar
- daily attacks in and around Ramadi
- resilience level of al Qaeda and related terrorist elements
- rate of use of vehicle bombs
- rate of use of vest bombs
- percentage of casualties that are Shia
- progress on the hydrocarbon law through the Iraqi parliament
- progress towards reforming de-Bathification to allow return of lower level Baathist to public life
- state of the Iraqi economy
- Oil revenue received
- Foreign aid received
- Money available in federal coffers
- Performance level of public utilities (presumable electricity, water, sewerage, sanitation, petrol supplies, public transportation)
- How well schools are functioning
- The overall state of the health system infrastructure
- Level of unemployment
And, reviewing the whole report, there are literally dozens of other important factors that show up in charts or tables.
Wishlist 1: It would be really valuable if one could turn the page of the Iraq Index report and see the chart or charts for each of the these items one by one, in the same sequence as they appeared in the essay and with the same terminology, so as to get a sense for the degree of progress that is being made, to see which things are lagging, to see what is getting worse and by how much, and so on.
This, in my mind would provide a giant step upward in the usability and the understandability of the important work that provides the basis for these weekly reports.
Comment: Many of the tables and charts in the weekly report are out of date. They represent important factors but no new data has been available to update them for quite some time.
Wishlist 2. It would be helpful if outdated tables and charts were either placed in an appendix, or if the chart could be updated to show the current end date and the missing data. Otherwise it is easy to misread the right hand side of the charts to think that the data and the trends represent the past few weeks or months instead of sometime in June 2006, or May 2005.
Comment: The chart formating for just about every chart is different and each chart seems to have its own unique starting and ending time intervals.
Wishlist 3. A move to more consistent chart formats and time intervals would greatly enhance the readability and usability of these reports.
Wednesday, April 11, 2007
Iraq Trend Data Set now posted to Swivel and Many Eyes
A while back we mentioned the great new data sharing and visualization web site called Many Eyes and we pointed out an Iraq Trend data set we had uploaded so that others could more readily examine and analyze it.
We've recently come across another great new data sharing and visualization web site called Swivel and have uploaded the same data set there for comparison purposes at: http://www.swivel.com/data_sets/show/1004780.
Both Many Eyes and Swivel fall into a category I have started calling Trend Visualization Appliances. The amazing Gapminder work also fits into this category as do all the various stock market visualization tools currently available.
The old fashioned form of trend visualization appliances (e.g. a standard report including graphical output) can still be pretty useful when managed carefully. For further comparison, here is the link to a PDF report that graphically presents the same data - vizualizing-trends-ohanlon-testimony.pdf.
Similarly, for data sharing, the old fashioned URL links to csv files offer an alternative to the WEB 2.0 mechanisms such as Many Eyes and Swivel. Here's the Iraq trend data the old fashioned way: ohanlon-key-factors.csv
So many choices, so little time. How can we decide which is best for our purposes?
I'll be revisiting this topic, but for now, my own criteria for deciding which tools I will use at a given moment are:
1) ease of use,
2) shortness of learning curve, and
3) personal productivity and time saving -- the speed at which I can navigate through complex trend data sets to discover previously hidden patterns.
What do you think?
We've recently come across another great new data sharing and visualization web site called Swivel and have uploaded the same data set there for comparison purposes at: http://www.swivel.com/data_sets/show/1004780.
Both Many Eyes and Swivel fall into a category I have started calling Trend Visualization Appliances. The amazing Gapminder work also fits into this category as do all the various stock market visualization tools currently available.
The old fashioned form of trend visualization appliances (e.g. a standard report including graphical output) can still be pretty useful when managed carefully. For further comparison, here is the link to a PDF report that graphically presents the same data - vizualizing-trends-ohanlon-testimony.pdf.
Similarly, for data sharing, the old fashioned URL links to csv files offer an alternative to the WEB 2.0 mechanisms such as Many Eyes and Swivel. Here's the Iraq trend data the old fashioned way: ohanlon-key-factors.csv
So many choices, so little time. How can we decide which is best for our purposes?
I'll be revisiting this topic, but for now, my own criteria for deciding which tools I will use at a given moment are:
1) ease of use,
2) shortness of learning curve, and
3) personal productivity and time saving -- the speed at which I can navigate through complex trend data sets to discover previously hidden patterns.
What do you think?
Saturday, March 24, 2007
Michael O'Hanlon Year over Year Iraq Update
Here's a recent Michael O'Hanlon New York Times article which includes year over year table showing some 17 trend metrics from Feb 2004, year by year through Feb 2007.
The State of Iraq: An Update:
We plan to graph some of the more interesting of these trends in a future post to see what patterns we can discover that may be hiding in the data once we have transformed the posted table into a more readily reusable format for further investigation and discussion.
The State of Iraq: An Update:
We plan to graph some of the more interesting of these trends in a future post to see what patterns we can discover that may be hiding in the data once we have transformed the posted table into a more readily reusable format for further investigation and discussion.
Subscribe to:
Posts (Atom)