Frequency of Reporting

In the previous post, we brought up the concept of frequency of reporting on a given metric. We haven't talked much about this aspect of trend analysis so far, despite the fact that it can be quite important. And it interacts with another important topic we brought up in some previous posts: near real time reporting.

We'll take a look briefly at frequency in this post, and give a brief look at near real time reporting in the next.

Error on the Side of Too Often.
Each important factor that you wish to measure will prove most useful in the end if you make a conscious choice to consider its frequency of measurement separately on the merits. My general operational rule is wherever possible to err on the side of measuring too often rather than not often enough.

Measurement vs. Reporting Frequency. When you measure at a relatively high frequency, it's possible later to report on the data at either the high collection frequency, or at a reduced frequency in order to smooth out the trend line pattern. On the other hand, if you don't measure frequently enough, you may miss out on short-lived quantum shifts in behavior that will be lost when reporting at longer intervals.

It's unfortunately not uncommon for vital indicators that are already being collected to have their frequency set so that measurements are too far apart.

Measuring Daily is Often a Good Starting Point. Let's take a specific example: the percentage of eligible children who attend primary school. Ideally, in my opinion, a metric such as this would be best if it were measured daily. There will be times when you will want to look at the day to day trend and see if any disruptive multi-day even has shown up. You may also want to understand any patterns that might be dependent on the day of the week. There will be other times when week by week reporting will be just right and still others when month by month will give the clearest view. Before you actually have the data, you cannot really know which reporting interval will prove most revealing.

My recommendations for the important Iraq metrics of the kind we have been scraping from the recent news reporting is that we make every effort to measure them with a frequency of once per day wherever that is possible.

When is Higher Frequency Helpful? There are some indicators where it would be advantageous to measure them even more frequently than once a day. For example, electricity delivery in Baghdad is a factor that I would like to measure minute by minute during the day and to do this both for the city as a whole as well as disaggregating these results by districts in the city and/or by individual power stations. Unless I get a very high resolution frequency, I won't be able to understand the frequency of or length or breadth of districts impacted by each outage. Such diagnostic capability is likely vital to figuring out where the est opportunities are for achieving higher levels of service and moving closer to fulfilling the desired demand.

Obstacles to High Frequency. On the other hand, there will be metrics such as the percentage of Iraqi members of Parliament who are living abroad where the obstacles to measuring may mean that we only get a read out on this indicator once a month. It might be nice to see how this varies day to day, but prove to be logistically impractical to get it.

Adjusting as you go along. There is no perfect frequency and there can likely be differences of opinion as to which frequency is best. The good news is that frequency can be adjusted as needed as we go along. The really important part is identifying all the important metrics and beginning the process to measure and record their values as they change over time.