NY Times charts based on DoD report

Here's the New York Times graphic that accompanied their story this morning on the DoD report to Congress.

Their stacked bar chart is similar to the unstacked bar chart from the Washington post.

Geographic charts are popular these days but I find the pair of charts in this example difficult to interpret. There are some trends going on with respect to readiness to assume security responsibility, but one has to spend quite a bit of time to figure out what they are from these maps. The eye has to dart back and forth between the two maps to make a before and after comparison. If I am reading these correctly, it looks like some serious backsliding has occurred since Nov 2006. And of course, having only a pair of data points for each trend makes any interpretations or extrapolations risky.

The sectarian incident chart at the top right is useful. I would prefer to see a longer time frame and data that is more up to the minute (e.g. at least through the end of May 2007).

The first chart showing the daily casualties also falls short in only reporting data up through May 4th, 2007. While real time reporting is rarely needed in this kind of situation, important factors such as casualties must be reported in near real time for them to be truly useful.

Timelines in the News - Deja Vu

Here's a post on Electric Goalposts from Matthew Yglesias from December 8th, 2004 commenting on the electricity trends contained in the State Departments Weekly Iraq Status Report dated December 1st, 2004.

The money quote is:

But for goalposts purposes, let's note that we are not now at the goal, have never been at the goal, and are not even moving in the direction of the goal. Instead, Iraqis have less electricity (but way more phone lines, interestingly enough) than they had before the war.

Here's the chart from December 1, 2004 followed by the one from May 30th 2007. Not much seems to have changed in the intervening 30 months.


I found the commentary added into the first chart to be most helpful and useful as an aid to understanding what was really happening. A picture is worth a thousand words but even so, many time a few extra words can add even more value.


Both these charts rate high because they show a complete time range. Of course, it would sure be interesting to see what the Iraqi electricity chart looked like starting back in 1980 so that the effects of the first Gulf War when, if memory serves, severe damage was done to the Iraqi infrastructure. It would be useful to compare the recovery or non-recovery following that war to the current recovery. If anyone has that data, please let me know.

Monthly trade deficit

Another great chart from Calculated Risk

You can find the full post here

Is the "Surge" working?

Here's a before and after the 'surge' graphic from this morning's New York Time article on: Patterns of War Shift in Iraq Amid Buildup of U.S. Force that sheds some light on the degree to which the "surge" is working.

Disaggregating the figures by area is helpful at clarifying better what is happening on the ground. For example, the increase in Baghdad deaths and Diyala deaths due to IED both look substantial while the drop off in total deaths in Anbar Province is also striking.

By selecting a 7 week Before Period and a 7 week After Period, this graphic sets up the possiblity of direct and straightforward Apples to Apples comparisons .

I would also have liked to see all the same key factors represented on this chart (such as Baghdad deaths by IED, Baghdad deaths by other causes, Anbar deaths by IED) also available as a week by week trend table and charts with the time frame stretching back to the beginning of the war so that the trends showing up in the current casualty levels could be put put into a broader context.

Of course, adding in other factors such as US troops wounded, Iraqi forces deaths and wounded, and Iraqi civilian deaths and wounded would give a much fuller picture of the degree to which the "surge" has so far changed things in Iraq.

Selective Perception, Cognitive Bias, & the Recency Effect

Barry Ritholtz has an excellent post this morning commenting on NonFarm Payroll (NFP) increase of 180,000 from yesterday's Bureau of Labor Statistics Employment Situation Report.

His comments on selective perception, cognitive bias, the "recency effect" and the soft prejudice of low expectations apply well beyond the NFP example.

Let's begin with a quick word on cognitive bias. Humans are guilty of this -- selectively perceiving and recalling what agrees with their world view. We are all guilty of this, and while we cannot escape it, being aware of it at least allows some measure of recognition, and perhaps, adaptation to the phenomena.

Let's use the NFP data as an example: Consider another bias, the Human tendency to overemphasize more recent data versus the totality of information and the overall trend. This is a cognitive bias known as the recency effect. Despite the overwhelming evidence showing this to be a generally weak jobs recovery (the worst since WWII), our primate brains interpret a single good data point as proof of something better. ...

Regardless, we also see some of the soft prejudice of low expectations in yesterday's data: 180k is hardly a rockin' strong number, relative to population growth,. Put that into the context of recent expansions such as the 1990s. Oh, and in that more recent period, there was no BLS Birth/Death adjustment, responsible for nearly a million fictitious jobs in 2006.

Unfortunately, the two trend chart he shows only go back three years (suffering themselves from the recency effect) so we can't compare the relatively weak current numbers to the NFP increases of the 1990s. The two charts shown in the Bureau of Labor Statistics Employment Situation Report also only go back 3 years.

So, below I have posted a 15 year view showing year over year percentage change for the seasonally adjusted value of the growth in NonFarm Payroll. It's pretty clear from this longer term chart that our growth in non-farm employment during this latest expansion never got back to the steady, multi-year 2.5% growth rate of the late 1990's and appears to be declining again.

Of course, in a subject area as complex as employment, an equally important danger in my opinion is the almost universal tendency these days to stick with just a few headline factors. If we truly want to understand employment, I believe we have to make it easy to visualize and then examine a full range of factors over a substantial time period.

For example, the recent drop off shown above does not seem to square with the reported drop in the headline unemployment rate. Maybe some other factor of the thousands of factors recorded by Bureau of Labor Statistics (BLS) can help explain it.

Too often, really important factors aren't mentioned at all, and the ones that are mentioned are not accompanied by their corresponding trend charts.

Here's a link to a short list of the most popular BLS statistics which might be a useful starting point for a deeper analysis.

More BBC trend charts for Iraq

BBC NEWS | In Depth | Iraq violence, in figures

The BBC link above is the source for these two easy to read charts -- the first showing the varying levels of US troop strength and the second showing the US Military Death Toll.

Good points of the BBC presentation:

1) Full range of Data. Both charts go back to the beginning of the conflict in 2003 and are up to date as of Feb 2007.

2) Combining Data from Multiple Sources. The BBC has combined data sources obtaining the troop strength information from the Brookings Institution and fatality figures from the Iraq Coalition Casualty Count. The charts are presented on the page so the single Time Axis can be used for both for more ready comparison.

3) Detailed Interactive Event Description Integrated with the Trend Data.
The BBC original interactive chart has the option to click on the event arrows to get a brief snapshot of some of the important events and see how these fit into the overall picture and relate to the underlying trends.

Change Over Time: First Principles Revisted

I am enjoying reading Edward Tufte's new book: Beautiful Evidence. It's the fourth in a series of wonderful Tufte volumes that exemplify the power of visual display and it certainly lives up to its title. Beautiful Evidence, like its predecessors, is simply a beautiful and inspiring work - blending art and science and philosophy and the linkages between numeric data and their visual representations into a unifying whole.

Tufte is powerfully interested in the underlying principles. For example, here's a brief passage from his introduction to Beautiful Evidence:

Evidence is evidence, whether words, numbers, images, diagrams, still or moving. The intellectual tasks remain constant regardless of the mode of evidence: to understand and to reason about the materials at hand, and to appraise their quality, relevance, and integrity.

... The point of evidence displays is to assist the thinking of the producer and consumer alike. Evidence presentations should be created in accord with the common analytical tasks at hand, which usually involve understanding causality, making multivariate comparisons, examining relevant evidence, and assessing the credibility of evidence and conclusions.

This blog is especially concerned with a particular kind of evidence: multi-dimensional time series. We are surely interested in the underlying intellectual and analytical tasks including understanding causality, making comparisons, and cross checking conclusions.

"Beauty" on the other hand, plays a much smaller role in our efforts. It is always a delight when it makes one of its rare appearances for us, but is not a primary or even a secondary goal. In fact, many of the approaches we recommend in order to meet our primary mission by their very nature tend to lead away from beauty.

Reading Tufte always encourages me to take a step back and think about the first principles that are the foundation of the work of this blog. I want to think through and examine all that his approach has in common with the work shown in this blog and all the ways we might differ.

Here are some further thoughts about the first principles that are the basis of my continuing work with the visualization of time series data (with special emphasis today on the principles that might mitigate against "beauty").

1. Time is of the essence - both analyst time and viewer time. While making comparisons and searching for understanding and seeking out properties of the data that have never before been seen, we will give preference to those approaches that help speed up the work of the analyst, even if some or a lot of beauty must be sacrificed.
2. Following Covey, we seek first to understand the data and then to present our findings in ways that make it easiest and quickest for others to understand.
   
3. Sharing the data in a readily accessible and readily reusable form along with the presentation of any findings is perhaps the core principle of our work. It's the one that allows for collaboration and for a rapid and multi-step search for meaning.
4. We aim to make both the data and whatever graphical representations we create accessible to experts and non-experts alike.
5. We care deeply about practical tools that allow ordinary citizens to participate in this work of comparison and understanding.
6. We believe in many eyes, many hands, many voices working together on the path of mutual discovery (rather than the idea of a single expert figuring it all out and presenting it to us in final form).
7. We want to apply this approach to the most important, practical, everyday problems that we face. We want this approach to help dramatically improve decision making. And we want this to happen in near real time.
   

For example, in hindsight, the famous Napoleon' March graphic that Tufte has written extensively about is certainly a beautiful and powerful display that promotes understanding of what happened to Napoleon's army on the way to Russia and back. But of course, it was not available in time to help Napoleon make better decisions. What we are interested in is figuring out ways in which such powerful understanding might be made available soon enough (in near real time) to actually be of use to the decisions makers and to ordinary citizens as complicated and important events are unfolding. Our recent series of posts about the key indicators at work that help us understand how things are going in Iraq is a step in that direction.

To sum up, in our approach, if we have to chose, we will chose timeliness, usefulness, ready re-usability, accessibility to the ordinary citizen, tools that are available here and now, and the ability to collaborate over beauty every time.

Are we there yet?

How well does the current set of Iraq trend charts & tables serve our needs?

Answer: I think we are a little better off than we were 10 days ago when I began this series of posts on Iraq, but we certainly are not where we need to be. We can look at the charts that are currently available from multiple sources much more quickly now than we could before by skimming these two pdf files: Combine-Iraq-Trends and vizualizing-trends-ohanlon-testimony.pdf. But each of these documents has it's own and weaknesses as discussed below. And there are a growing number of important missing metrics that as yet have no publicly available trend chart or trend table.

What are the weaknesses of the Combine-Iraq-Trends document?

Answer: This combined report consisting of trend charts drawn from 4 different sources covers a lot of important ground and shares the trend behavior of many factors that appear quite important. On the downside, the big weaknesses of this set of trend charts are numerous:

1. There is a wide variation in formats - This reduces overall understandability and slows down the process of viewing the entire set of charts. Each chart must be examined individually with careful checks to make sure that the range of the X and Y axes are are clear and to locate the legend information that tells you what kind of factor you are looking at. This is in clear violation of the Edward Tufte's idea of small multiples that we discussed in a previous post. Applying small multiple thinking to a set of trend charts means that for the viewer, once understanding is reached regarding how a single chart in a series is laid out and what graphing conventions have been used, succeeding charts can be grasped with much reduced time and effort.
   
2. Time frames of the individual charts are widely different. Some cover the period from the start of the War in 2003 right up to the present moment. Others, cover much shorter periods. For example, the IED chart covers only from January 2006-June 2006. Some charts show monthly data, others show weekly data. This is another violation of the small multiple idea and the consequence for the viewer is that it is more difficult and more time-consuming to make any comparisons between different factors.
   
3. Stale Data: Violation of the "Near Real Time" Principle In many charts such as the IED chart mentioned above, the most recent sample is many, many months ago. If something is rated as an important factor, then it won't do us any good unless we measure and report on it often and that we make the most recent sample data available as quickly as possible. A 7 month old data point of a key trend factor can substantially frustrate our ability to understand what's going on, to determine whether interventions we have taken are working, or to decide what to do next.
4. Overly short time frames. Charts such as the State Dept's Crude Oil Production chart cover an amazingly short time frame - in this case from October 30th, 2006 to January 7th, 2007. While this chart is both interesting, useful, and clear, the viewer is seriously short-changed by not being able to look at what has happened to this key trend factor for the entire period since the war began.
   
5. Unavailability of the Underlying Data. In many cases with these charts, the underlying trend data is unavailable for further review or analysis. This means that the viewer of the chart is sharply impeded from discovering different angles to examine the data (e.g. by combining several factors together in a single chart, or calculating new previously invisible factors as a combination of an original set of factors.) If the original analyst who selected the chart for presentation did not create one chart for each important view, then without the data, these new views are simply unattainable. And of course, it's never possible for the original analyst to provide all the views.
   
6. Data that is provided is Not Readily Reusable (RR). In the relatively small number of cases where the raw trend data is provided, it is painful and time-consuming to transform these data into an RR format. This slows down the effort of understanding and because of the time involved, only a small percentage of potentially interested parties ever actually make these transformations.
7. Missing Metrics, Quality of Metrics. Many of these sources mention other metrics specifically (presumably because they are considered important) but do not provide trend tables or charts. And all metrics, those that are charted and those that are not, must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in

The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”

What are the weaknesses of the Combine-Iraq-Trends document?

Answer:

1. Stale Data: Violation of the "Near Real Time" Principle - the most recent data point in all of these charts is November 2006 so the trend series displayed is not as up to date (on January 10th when this data was presented) as one might wish.
   
2. Overly long time frame between samples. Way too few samples. Having a total four samples, one year apart creates a trend table that is quite readable on an 8 by 11 sheet of paper. The price is dramatic loss of vital detail and can lead to seriously misleading conclusions.
3. Data that was originally provided was Not Readily Reusable (RR). We were able to work our way around this barrier and create trend charts from the data, but it was costly in time and will serve as an impediment to further analysis of the data for most people most of the time.
4. Missing Metrics, As we noted in an earlier post, there were many Iraq trend factors that Michael O'Hanlon considered important enough to include is his verbal testimony and in the logic of thinking through what those factors meant, but which did not appear in the attached table of 30 factors. Our view is that if a metric is important enough to mention and use for logical argument, it is imperative that the details of how that factor varies over time are provided for further review and analysis by interested parties.
   
5. Quality of Metrics. As for the case of evaluating the Combine-Iraq-Trends document, all metrics, those that are in the table (and therefore charted) and those that are in the text must be evaluated against the quality criteria outlined by Anthony Cordesman as noted in:

   The Quarterly Report on “Measuring Stability and Security in Iraq:” Fact, Fallacy, and an Overall Grade of “F”How
   

What next steps could help things along and help us move closer to our goal of helping both ordinary citizens and decision makers gain a better understanding of the trends at work on the ground in Iraq?

Answer: We've made some progress, but in many ways we have only just begun. Below, I outline some steps I am planning to take with the goal of taking this up to the next level. If you can assist in any way, I would be pleased to hear from you.

1. Continue to harvest and grow our list of missing metrics and invisible indicators, for example by close examination of Dept of Defense quarterly report or the many writings of Anthony Cordesman on the subject.
2. Create a single composite list of all the factors for which we do not yet have trend data or trend charts but which we think would further aid our understanding if they were to be made available.
3. Make inquiries to the original sources to see if the raw trend data for the key factors that have been identified can be made available for further use, ideally in a readily-reusable form to begin with.
   

Comparing Nasdaq to S & P - another nice chart

Here's another nice chart from: MarketBeat Blog - WSJ.com : Getting Technical on Tech comparing the Nasdaq indesx (NDX) to the S & P index over the past 10 days.



These kinds of simple two way comparisons create charts that are easy to read and understand.

Nice Chart of the impact of Apple's Launch

Nice Chart. simple, direct, powerful. The proverbial one that's worth a thousand words all on its own.



Hat tip again to Barry Ritholtz over at The Big Picture for an excellent chart ("via Andrew' via Yahoo charts) today showing the impact of Apple's Launch announcements today on the intra-day price of Apple's stock compared to some of its competitors (Palm & Rimm)