Averaging Ratios And The Perils Of Aggregation
|Special note: We are responding to reader requests concerning the development of a solid foundation for understanding, in very practical terms, statistical analysis. |
Many people suffer from acute statistical trauma. We are introducing a continuing series of articles that, we hope, provides readers with the basics required for proper usage of statistical tools and methods.
The first four "back-to-basics articles" should be read in the following order—namely: 1) Preface To Performance Measurement: Understanding Ratios, 2) Ratios: Abuses and Misuses, 3) Averaging Ratios and The Perils of Aggregation and 4) A Painless Look At Using Statistical Techniques To Find The Root Cause of a Problem.
For example, if you were buying lumber to repair your deck, and purchased 80 ft. of lumber at $2 a running foot; somewhat later purchased an additional 40 ft. at $2.50 a foot; and still later purchased an additional 24 ft. at $3 a foot, you should be able to calculate your cost per ft. for the entire project.
Obviously, you know that the average of the three purchases of $2, $2.50 and $3 is not $2.50, because you purchased different quantities at three different prices. The average purchase price per foot is easily found by calculating total dollars paid and dividing by the total feet obtained.
- $2 per foot x 80 feet= $160,
- $2.50 per foot x 40 feet. = $100,
- $3 per foot x 24 feet= $72.00,
- 144 feet= $332.00
The average purchase price of $2.3555 can be viewed as a weighted average of the three purchase prices. This average is closer to $2 than $3 because more lumber was bought at the lower prices than the higher prices.
The majority of performance metrics are ratios of one kind or another. The bulk of statistical tools used in Lean Six Sigma diagnostics and control involve weighted averages.
The problem of combination in space ratios (i.e. ratios of totals from different categories) is mostly a function of knowing when to use and when not to use the denominator as a weighting factor.
Suppose there were only three hospitals in a particular city. Hospital A had a surgical mortality rate of 6 percent during 2008. Hospital B's surgical mortality was 8 percent and Hospital C's was 10 percent during 2008.
If all surgery was performed in these three hospitals, what was the surgical mortality rate for the entire city? In all likelihood, it is not the average of the three ratios, or 8 percent. Why?
It is only 8 percent if the number of operations in each hospital was the same. One would have to weight each mortality ratio (number of deaths/number of operations performed) by the denominator of the ratio. What is the denominator? The number of operations performed!
In calculating a weighted average, the ratios to be averaged are first multiplied by their respective weights, that is, by the denominator of the ratio...the products are then totaled...and, finally, the total is divided by the sum of the weights.
The Following Example Demonstrates This Procedure
Assume that 1,000 operations were performed in the city. One hundred in Hospital A, 100 in Hospital B and 800 in Hospital C; then the surgical mortality rate for the entire city would have been 9.4 percent. Thus:
|Hospital||Ratio||Weight||Ratio Times Weight|
Weighted Ratio = 94/1000 x100 = 9.4 percent
However, if A had 800 operations, B 100 and C 100, then the surgical mortality rate for the city would have been 7.8 percent (we leave the computation to you). In short, the surgical mortality rate for the entire city could be anywhere between 6 and 10 percent depending upon their relative number of operations performed among the three hospitals.
The change in a weighted average can be as much a result of changes in the weights as it is of changes in the values being averaged. An extreme case of what can happen is called Simpson's paradox, apparently after E. H. Simpson, who called attention to the possibility in an article in The Journal of the Royal Statistical Society.
Is it possible for the average surgical mortality rate for the city to increase in 2008, but show an overall decline from the previous year even though the surgical mortality rate in each hospital increased? (This exercise is left to the reader.)
Hint: it's possible. Indeed, that's Simpson's paradox. Each rate can increase...but if the weights change, with more weight being put at the lower mortality rate, the weighted ratio could significantly decrease.
Warning: A Misuse of Ratios in the Making
The term "aggregation of data" refers to the level of detail provided in the numbers presented. To help consumers make informed decisions about health care, the government releases data about patient outcomes in hospitals.
If you were so inclined, you may want to compare surgical outcomes between, say, Hospital A and Hospital B. The evidence seems clear: Hospital A has 6 percent mortality rate... and Hospital B has an 8 percent mortality rate.
But we know we made you a skeptic. Of course, we assume you've read the article on misuses of ratios. The term "patient mortality" is an example of the perils of aggregation. (Please read Misuse of Ratios and other articles on the importance of finding relevant subgroups before analyzing data).
David S. Moore and George P. McCabe in their book Introduction To The Practice Of Statistics (New York: W. H. Freeman and company, 2000) provide an excellent example of what can happen if the data is disaggregated. Patients can be classified as being in either "poor" or "good" condition.
The patient's condition is labeled "a lurking the variable." If we ignore the lurking variable, Hospital A seems safer. If, for example, Hospital B is a medical center that attracts seriously ill patients... and Hospital A selects only "good condition" patients, then it is reasonable to conclude that we are comparing ratios refined to different degrees.
Moore and McCabe present an illustration that subdivides patients into two categories—namely, poor and good condition. You can probably guess the result. The hospital that initially looked like the better hospital turned out to be the inferior hospital in relationship to both categories of patients.
Just keeping you on your toes. Hopefully, you're beginning to feel comfortable with terms such as data aggregation, lurking variables, weighted averages and the like. Ratios or performance metrics must be constantly questioned.