Showing posts with label aggregated metrics. Show all posts
Showing posts with label aggregated metrics. Show all posts

Sunday, September 18, 2011

Providing some Context for looking at CPI charts

We have posted many charts this past week looking at CPI behavior through a variety of angles and lenses.  Our sources have been light on providing context and we have not filled in this area either.

So here is a first cut at the context that we were holding loosely in mind as we evaluated the incoming collection of charts and created a few of our own using the St. Louis Federal Reserves' FRED software.

To begin, CPI stands for Consumer Price Index.  In our view, the emphasis is on the Consumer (real people living out their lives) and the ways in which changes in Prices (specifically price inflation) might impact the standard of living for those people.

Central Questions:  The question we would like to be able to answer more fully is:
How does the cumulative change in prices affect the buying power of a range of different classes of consumers over time?
In particular, we want to be able to figure out to what degree the Cumulative Impact of Price Inflation changes each different class of consumer's standard of living over time.
Principles.   The underlying principle that we believe to be at work here is that: if cumulative CPI goes up relative to the wage trend for households representing a given class of consumers, then the Standard of Living for that class would go down in proportion.  We are interested in looking at the cumulative change over a variety of longer periods: 2 years, 3 years, 5 years, 10 years, 15 years, 20 years

A second key principle for us is that CPI Inflation is like Compound Interest; and so longer term cumulative views of the data should predominate.


Weaknesses of Traditional Reporting:  To us, the commonly used month over month results represent little more than noise since our focus is on cumulative results.

Even the very widely used year over year reporting is not particularly useful in this regard and is usually distracting at best (1 year effects are interesting but surely not the whole story) and misleading at worst as to the important long term inflation changes that are underway and the implications of those changes on real people's standards of living and quality of life.

A long standing general purpose admonition of ours is to Look at All the Data.   The traditional reporting on CPI violates this rule in spades with the focus on only headline CPI and core CPI .  CPI is an aggregate of aggregates of aggregates which averages out a huge degree of variation that is present in its sub-components.  To begin to understand the impact that cumulative price inflation has on Consumers (real people) we surely need to disaggregate the average CPI values and look individually at a good number of the key sub-components (e.g.  Food, Energy, Health Care, Rent, Transportation, Tuition) to see their cumulative price inflation over longer periods of time.   With few exceptions such as the excellent chart we posted earlier (A Unique View of CPI from Doug Short), almost all reportage we have found on CPI focuses on headline or core cpi.

Now it is true that the official report from the BLS (BLS August 2011 CPI news release)  does provide an exceptionally high degree of disaggregation in its tables by sub-categories and sub-sub-categories.  But there are no sub-component charts accompanying the BLS report, so looking at sub-component increases in cpi is left as a time-consuming exercise for the reader - an option that few have likely taken.    Furthermore, the data presented only covers the most recent 1 year period, so if you want to explore sub-category cumulative changes over periods of 2 to 20 years, you have to look somewhere else (e.g FRED or the BLS data base).

There is a lot of fruitful potential in this disaggregation, but I have not been able to find anyone so far other than Doug Short and our recent post (Drilling down into CPI 20 year trends)  tapping into this for more insights into understanding the cumulative impact of inflation on real people.  We will keep on looking and would love to hear about other work in progress studying the sub-components.

Core CPI interferes with clear thinking.  Finally, looking at widely discussed Core CPI and its many variants,  we see Core CPI as a serious impediment keeping us from thinking more clearly about the cumulative impact of price inflation on real people who all must eat and consume energy.  Rather than stripping food and energy out of the totals to create so-called CORE CPI, it would be much more useful in our estimation to include Food and include Energy as individual sub-categories - to look at the cumulative impact of Food price inflation and the cumulative impact of Energy price inflation as we have started to in our Drilling down into CPI 20 year trends  post.  And rather than lump all the rest of the categories together to create the "core", we would prefer to disaggregate these as well and look at them individually so we can discover patterns currently hidden..

A lot has been made about why Energy and Food are so volatile as the reason for going with Core CPI.  Our investigation shows that the mostly irrelevant month to month variations smooth out incredibly well for FOOD once we look at cumulative totals.  For ENERGY, there is clear volatility, but rather than hiding it, it's better to bring it out into the open.  And when we look at energy using a cumulative long term 5, 10, 15, 20 year approach, there is a lot less volatility than you might expect.  We'll have more on this later.

And the headline CPI number shows very low variability in trend when plotted using a long term cumulative view as we have shown in our early post ( Drilling down into CPI 20 year trends ).

There are further weaknesses we see in traditional CPI reporting regarding the limited number of different classes of consumers currently available) but we will save these for another day.

Now that we have some context on why we want to look at CPI, our plan is to revisit the CPI posts from earlier this past week.  We plan to update them by showing our thinking regarding what we see as the story that might be told for each chart in the context of the ideas we have explored in this post - real people, cumulative impact over longer time spans, disaggregation.  . 

Update 20 September 2011:  We have now updated all previous CPI posts with our further thoughts on what each chart is telling us.  Please check them out.


Friday, August 19, 2011

Common Weaknesses in Online Visualization & Storytelling

Looking around the web we notice a growing number of examples of online visualization and storytelling.

Some of these are brilliant and incisive and easily accessible and digestible by their intended audience.

Too many however (including I am sure many of my own) exemplify one or more of a common set of weaknesses that make them harder to understand and that diminish their usefulness and value.

Here is a list of the shortcomings that appear most regularly. Once you can recognize them, all of these are correctable, often with only a modest effort that will pay big dividends. We've talked about many of these in previous posts and will no doubt return to them again to describe the particular details.

Can you think of any others? Which ones do you think are most important to correct?

Please share your thinking in the comments. Thanks.

Online Visualization and Storytelling Weaknesses and Shortcomings
  1. Too short a time period shown
  2. Too few metrics shown (sometimes only one or two out of thousands) and often only a single independent view of the key story telling metrics
  3. No story presented - figuring out the story is left as an exercise for the audience/reader/viewer. This often goes hand in hand with visualizations that require a substantial time investment by the audience in order to discover messages that are not obvious at first glance. Or worse to spend time and not be able to figure out why that particular graphic was chosen from amongst all the choices available to the analyst/storyteller
  4. Data set used to create the graphics is not readily available for further analysis by interested audience members
  5. The larger data set used by the analyst/visualizer/storyteller is not available and not even defined or listed. Consequently the viewer has no idea of how much effort the storyteller put into the analysis before deciding to display a particular choice of graphical elements.
  6. A standard template is re-used without any new or fresh thinking and without any sign of building on what's already been learned from previous analyses
  7. Presenting only a single point in time for many metrics that change over time without providing the relevant time line view
  8. Comparing just the most recent and the previous value of a particular metric without taking earlier values into account. This goes hand in hand with over use of graphs and tables showing month over month change.
  9. When showing month over month change, failing to normalize the values to yearly percentages
  10. Explaining time series behavior in dense text that is hard to parse and understand even for expert data analysts when a simple time series graphic would have done the job in seconds
  11. Limited opportunities for further collaboration between the audience and those who created the visualizations and story line.
  12. Too few data points in the time series
  13. Use of large unsorted lists where some simple sorts and application of some variant of the 80/20 rule would have conveyed much more meaning in a much shorter time
  14. Too may metrics all mushed together into a single indecipherable graphic. Such charts typically are ones that have no story line associated with them. What does the chart mean? You go figure it out!
  15. Burying the lead (the potentially most interesting story element) so only audience members who invest significant time will ever have a chance to stumble across it. Everyone focuses on some headline number while the action is just a little bit below the surface and eager to see the light of day
  16. Absence of comparisons of the result to useful baseline values
  17. Working exclusively with the raw metrics as they arrive from their providers and missing out on opportunities to combine metrics to create calculated values that enhance the storytelling potential
  18. Heavy emphasis on working with aggregated metrics (e.g headline numbers) and not showing whether the same patterns hold up under a variety of disaggregation approaches
  19. Using widely varying raw metric values when a carefully selected simple moving average would have revealed greater insight
  20. Overly tiny graphics that fail to take advantage of the full screen real estate available and make key elements more difficult to read and understand
What weaknesses would you love to see corrected?