Opinion

The Use of Statistics in Healthcare Quality Improvement

Joseph L Hagan*

Assistant Professor, Department of Pediatrics, Baylor College of Medicine Research Statistician, Texas Children’s Hospital, USA

Submission: July 23, 2018; Published: July 3, 2018

*Corresponding author: Joseph L Hagan, Sc.D., M.S.P.H, Assistant Professor, Department of Pediatrics, Baylor College of Medicine Research Statistician, Texas Children’s Hospital, USA, Tel: 832-824-3090; Email: jlhagan@texaschildrens.org

How to cite this article: Joseph L Hagan. The Use of Statistics in Healthcare Quality Improvement. JOJ Nurse Health Care. 2018; 8(5): 555746. DOI: 10.19080/JOJNHC.2018.08.555746.

Opinion

Although statistical process control (SPC) charts were introduced by Walter Shewhart in 1924 and have been widely used in industry ever since, they have only recently been employed in healthcare settings. SPC charts provide a way to visualize a process metric over time with rules for identifying a signal of assignable cause variation in the process (i.e., an unusually high or low value of the metric that is unlikely to occur simply due to random chance). Nowadays staff at many hospitals use SPC charts to monitor nurse sensitive indicators and adverse events such as rates of falls, central line associated blood stream infections, catheter associated urinary tract infections, ventilator acquired pneumonia, pressure ulcers, surgical site infections, readmission within 30 days, etc. Not only are SPC charts used to monitor these events over time so as to quickly identify an unwanted increase in their rate of occurrence, but SPC charts can also be used to evaluate the effectiveness of interventions designed to prevent the harmful events from occurring. Utilization of SPC charts can extend beyond monitoring events rates to other types of measures such as the average length of hospital stay or average cost of treating patients with a certain diagnosis. In fact, the potential uses of SPC charts for monitoring outcomes are virtually unlimited. However, in practice the application and interpretation of SPC charts is often not as rigorous as it should be.

Firstly, rules for identifying signals of assignable cause variation in these outcomes should be specified a priori, with an action plan for how to address such signals. Almost always, a deep dive to investigate the reason for the unusual observation will be warranted. Such investigations should be conducted as soon as possible in order to quickly remedy any modifiable causes of harm. Next, care should be taken to correctly apply SPC charts in order to yield results for which meaningful interpretations can be made. Application of SPC charts involves two phases. In the first phase, the process should be brought into a state of statistical control where the metric is stable (i.e., all values fall within the control limits defined as ±3 standard errors, or “3-sigma”, from the center line). Once the process is stable, at least 20 subgroups (where subgroups are defined by some unit of time such as weeks or months) of data should be used to estimate the center line (e.g., the process mean) and control limits of the SPC chart Montgomery [1]. Then in the second phase, the process is monitored over time for signals of assignable cause variation. Any subgroup value outside the 3-sigma control limits is considered such a signal, but additional “sensitizing rules” can be used to generate other types of signals of assignable cause variation (e.g., two out of three successive subgroups more than two standard errors from the center line on the same side of the center line, eight subgroups in a row on the same side of the center line representing a ‘drift” in the process, six subgroups in a row either increasing or decreasing representing a “trend”, etc.). Which of these sensitizing rules will be used should be agreed upon before data collection. There are many other important aspects of control charts that need to be considered, such as the number of observations per subgroup so as to yield control limits of a reasonable width, the frequency with which the event of interest (e.g., the nurse sensitive indicator) occurs which can determine the appropriate type of SPC chart (e.g., a P chart for measures of relatively common events such the percent of patient satisfaction scores in the top box versus a G chart for rare events such as the number of cases between surgical site infections). Appropriate use of SPC charts requires some degree of knowledge and, despite its importance, a thorough treatment of SPC charts is beyond the scope of this article. For more information, the reader is referred to other resources such as Provost and Murray [2], which provides a practical introduction to the use of SPC charts for healthcare quality improvement that is accessible to non-statisticians.

Although SPC charts are commonly employed, the use of classical statistical methods for hypothesis testing is sometimes discouraged in healthcare quality improvement settings e.g., Provost and Murray [2] based on the rationale that quality improvement data are collected for an analytic study “in which action will be taken on the process or cause-system that produced the frame studied, the aim being to improve practice in the future”, as contrasted with an enumerative study “in which action will be taken on the material in the frame studied” as is the case in most traditional research studies Deming [3]. The distinction is that for analytic studies used for quality improvement, there is not a well-defined population that serves as the sampling frame because conditions (other than the presence of the intervention being evaluated) change over time, so in theory the data collected in the future to test for improvement did not arise from the same population as the baseline data collected prior to implementation of the intervention. Classical statistical methods assume the sample of data to be analyzed is selected at random from a well-defined population. Using classical methods for statistical inference, the hypothesis that patients who receive an intervention have a better outcome than patients who do not is tested by computing the probability that differences (for example, in the intervention verses non-intervention group) in outcomes as large as those seen in the sample of data obtained for the study would be seen due to random chance (i.e., random sampling variation) if there is really no difference in the population from which the sample was obtained. This probability is called the p-value. The assumption here is that the intervention is the only difference between the two groups being compared. Deming argues that classical statistical methods are not valid in quality improvement because baseline data is different from the future data in ways other than the intervention. This is a controversial topic among statisticians. Woodall [4] provides some views from statisticians on different sides of the argument about the use of SPC charts and hypothesis testing in quality improvement.

The case could be made that the distinction between analytical and enumerative studies is somewhat theoretical, and classical statistical methods can still be of practical use in quality improvement. Indeed, industrial engineering textbooks that teach the use of statistics for quality improvement cover the classical statistical methods without mention of analytical versus enumerative studies e.g., Montgomery [1]. Quasi-experimental study designs where the intervention being studied is not randomized are typically employed for quality improvement. For example, pre-intervention versus post-intervention comparisons are commonly made in the context of quality improvement. When interpreting the results from pre- versus post-intervention studies, the caveat should be kept in mind that conditions other than the intervention might have changed and influenced the outcomes. Consider a hypothetical example where staff in a neonatal intensive care unit (NICU) introduce an intervention (5,000 IU intramuscular vitamin A administered 3 times per week) in the hopes of reducing the incidence of bronchopulmonary dysplasia (BPD) in preterm infants. BPD is a form of chronic lung disease defined using oxygen supplementation at 36 weeks corrected gestational age (CGA). For this quality improvement project, several years of data are collected before and after the intervention. When the data are analyzed, the hospital staff are disappointed that the proportions of preterm infants who experience BPD before versus after the intervention are not significantly different according to the chisquare test p-value. After giving the matter a great deal of thought, one of the neonatologists suggested that over the years during which the data were collected, technology and other aspects of caring for preterm infants in the NICU have improved. Therefore, more recently, extremely low birthweight (ELBW) infants who would not have survived to 36 weeks CGA just a few years ago were able to be kept alive. However, these ELBW infants who are disproportionally represented in the post-intervention data are at higher risk for BPD. Since the average birthweight was lower after the intervention, and lower birthweight is a risk factor for BPD, birthweight represents a potential confounder in the relationship between the vitamin A intervention and BPD incidence in this quality improvement project. After this realization, data were re-analyzed using logistic regression which showed that, after controlling for birthweight, the odds of BPD were significantly lower after the intervention compared to before the intervention. Because the results showed a benefit, the NICU staff continued to administer vitamin A to preterm infants.

This example illustrates how comparisons in quality improvement can be susceptible to confounding. But instead of shunning classical statistical methods for analyzing quality improvement data, the full arsenal of statistical methods should be used to appropriately analyze the data in order to obtain valid results. More sophisticated methods of analysis that control for potential confounders require putting thought into data collection beforehand and collecting patient level data on all relevant covariates will be more arduous than using less granular data summaries. And communication between knowledgeable statisticians and clinicians is needed to plan the appropriate data collection and analysis strategy. But these efforts will be rewarded by analyses that yield meaningful results to guide effective quality improvement.

References

1. Montgomery DC (2013) Introduction to Statistical Quality Control. (Montgomery DC) Introduction to Statistical Quality Control (7th edn.), John Wiley & Sons, Hoboken, N: Virginia, USA.
2. Provost LP, Murray SK (2011) The Healthcare Data Guide: Learning from Data for Improvement. Jossey-Bass: San Francisco, CA, pp. 480.
3. Deming WE (1975) On Probability as a Basis for Action. The American Statistician 29(4): 146-152.
4. Woodall W (2000) Controversies and Contradictions in Statistical Process Control. Journal of Quality Technology 32(4): 341-350.