Prevention of Severe Adverse Drug Reaction Outcomes – The Need for Clinical Decision Making Process for Identification of High Risk Patients in Large and Diverse Populations
Ash Genaidy*, George Tran and Alan Baldwin
Health Management, Anthem Inc, USA
Submission: February 26, 2018; Published: March 28, 2018
*Corresponding author: Ash Genaidy, Health Management, Anthem Inc, USA
How to cite this article: Ash Genaidy, George T, Alan B.Prevention of Severe Adverse Drug Reaction Outcomes – The Need for Clinical Decision Making Process for Identification of High Risk Patients in Large and Diverse Populations . JOJ Int Med. 2019; 1(1): 555552.
Keywords: Morbidity Clinical Predominantly Metrics Medicare Pharmacologic
Abbrevations: ADRs: Adverse Drug Reactions; NSAID: Non-Steroidal Anti-Inflammatory Drugs; ACEI : Angiotensin Converting Enzyme Inhibitors; RB: Receptor Blocker; ER: Emergency Room
Health Management, Anthem Inc
Although severe adverse drug reactions (ADRs) are a major contributor to morbidity/ mortality rates/costs, up to 50% of these adverse events can be prevented. To date, efforts of ADR prevention have relied upon the identification of high risk patients via models used to predict the likelihood of ADR outcomes without being specific about outcome type or the development of clinical rules geared at a limited set of ADRs, predominantly bleeding outcomes.
Collectively, these efforts lack the methodology to address prevention in large and diverse populations. This short report showcases a process designed to identify the desired volume of patients at high risk of severe ADRs for large/diverse populations as a function of multiple criteria including traditional statistical metrics (e.g., sensitivity), economic (e.g., cost-savings) and best practices (e.g., % of desired prevention intervention improvement) with the dual goals to improve the quality and cost of care. The methodology is applied to major and non-major bleeding, and acute kidney injury outcomes for a wide spectrum of the US population representing the Commercial (working population and dependents) and Government programs (Medicare – individuals who are 65+ years; Medicaid – socially disadvantaged individuals with limited income). Practical challenges are discussed to further improve efforts designed to advance the science of ADR prevention.
Introduction
Pharmacologic treatments are an essential strategy in the care delivery journey for chronic patients worldwide. Whilethey carry tremendous beneficial effects in care management interventions, medications may cause adverse drug events such as adverse drug reactions (ADRs) due to drug usage at normal doses [1-4]. The area of suspected severe ADRs (i.e., adverse outcomes occurring at any moment in time after post-drug marketing [2]) is of particular importance because, although they are a major cause of morbidity and mortality [5-9], about 45% of inpatient visits and 52% of outpatient visits induced by these ADRs can be prevented [10]. Consequently, the prevention of severe ADRs becomes a good value proposition and an important strategy to improve both the cost and quality of care for pharmacologic treatments.
To combat severe ADRs, two schools of thoughts emerged. First, the identification of patients at high risk for ADRs have relied on predictive models using the event likelihood as the outcome of interest [11-13]. In these efforts, severe ADRs have not been examined distinctively (e.g., major bleeding). This prompted some authors to express the need for explicit specification of ADR outcomes (e.g., major bleeding, acute kidney injury) so as to:
a) develop customized actionable interventions to prevent specific ADR outcomes
b) potentially improve the performance of developed predictive models [14]
The second set of efforts relied on the development of clinical rules designed for distinctive severe ADRs and primarily aimed at specialty care providers. These efforts have concentrated onmajor (e.g., intracranial bleeding) and non-major bleeding (e.g., gastrointestinal bleeding) [15-21], with emphasis on the elderly population and in some instances these efforts were based on small sample sizes. In addition, limited attempts made use of predictive models to predict the risk of major and non-major bleeding in a national UK population [22]. Collectively, the efforts have emphasized the identification of high risk individuals without giving adequate guidance for the translation of research findings to practice for ADR prevention particularly for large and diverse populations.
The objective of this short report is to advance the science of ADR prevention by showcasing a process designed to identify patients at high risk of ADRs within the context of large and diverse US populations including Commercial (working population and dependents) and Government (Medicare – individuals who are 65+ years; Medicaid – socially disadvantaged individuals with limited income) lines of business in managed care organizations. The specific aims are
a. to assess severe ADR outcomes in terms of annual incidence rates and healthcare costs for diverse US populations and
b. to present a process designed to identify patients at high risk of ADRs in large and diverse US populations as a function of validation parameters for ADR predictive models, economic measures utilizing cost-savings calculations, and best practices in managed care organizations
Methods
Program background
A program has been developed at Anthem Blue Cross and Blue Shield in order to monitor and prevent severe ADRs across a wide spectrum of the US populations exposed to different medications and spanning the Commercial, Medicare and Medicaid lines of business. Initially, the ADR monitoring and prevention program has been generated for eleven severe ADRs: major bleeding, non-major bleeding, acute kidney injury, hypotension, severe hypotension, acute liver injury, hyponatremia, uncontrolled hyperglycemia, hyperkalemia, and hypokalemia. This short report present examples of this work for major bleeding, nonmajor bleeding and acute kidney injury.
Outcome and exposure definition
The definitions of major bleeding, non-major bleeding, and acute kidney injury are provided in Table 1. Major and non-major bleeding may arise from exposure to anti-coagulants and antiplatelets. Anti-coagulants include warfarin and oral agents such as dagibatran, apixaban, betrixaban, edoxaban, and rivaroxaban. Furthermore, anti-platelets consist of aspirin, dipyridamole, abciximab, eptifibatide, tirofiban, cilostazol, vorapaxar, anagrelide, clopidogrel, prasagrel, ticlopidine, cangrelor, aspirin-dipyridamole. Acute kidney injury may be induced from individual or interactions of angiotensin converting enzyme inhibitors (ACEI) and receptor blocker (ARB) (e.g., benazeprilcaptopril, enalapril), diuretics (e.g., bumetanide, furosemide, torsemide), and non-steroidal anti-inflammatory drugs (NSAID) (e.g., diclofenac, etodofac, fenoprofen).
Calculation of incidence rates and healthcare costs
The target population for each of the aforementioned lines of business (Commercial, Medicare, Medicaid) were those exposed to the medications described above. All ADR outcomes were identified using ICD-10 codes documented in Table 1.
The medical databases were used as the data sources for the calculation of incidence rates and healthcare costs incurred during the 2016 fiscal year (Jan 1-Dec 31). An incidence was identified as the first event occurring in 2016 for a given individual. The incidence rate was calculated as the number of new events per 1000 of exposed population. The total healthcare expenditures associated with the ICD-10 in question for a given individual were computed for all emergency room (ER) and inpatient stay visits. Average costs are reported. All analyses were conducted using the Statistical Analysis Software [23].
Process for identification of patients at high risk of severe ADRs
The identification of patients at high risk of severe ADRs is a complex process requiring multiple criteria designed to reach this important clinical outcome in large and diverse segments of the population. The parameters and algorithm used the process are described in the following sections.
Validation parameters for predictive models. In this short report, we document the validation metrics for the ADR predictive models for major bleeding, non-major bleeding and acute kidney injury outcomes. These models were trained and tested as part of the tracking and prevention program of severe ADR outcomes as previously explained. The best models for acute kidney injury and non-major bleeding were obtained using the gradient boosting techniques (areas under the curve being 0.89 and 0.75, respectively for both models) and the random forest technique produced the best model for major bleeding (area under the curve=0.91). A technical comparison of random forest and gradient boosting techniques is provided elsewhere [24]. The input variables represent data derived from claims databases over a 24-month period and the outcome variables include ER and hospital admission events for the respective ADR defined in terms of ICD-10 codes over a 3-month period.
The validation metrics are described in terms of two sets of outcomes [25].
The first set of outcomes includes:
I. true positive (likelihood of a patient predicted by the model as high risk when he/she has the outcome of interest)
II. false positive (likelihood of a patient predicted by the model as high risk when he/she does not have the outcome of interest)
III. true negative (likelihood of a patient predicted by themodel as low risk when he/she does not have the outcome of interest)
IV. false negative (likelihood of a patient predicted by the model as low risk when he/she has the outcome of interest).
The second set of outcomes consists of:
i. sensitivity (proportion of the population who experiences the outcome of interest that the prediction model successfully identifies)
ii. positive predicted value (proportion of patients who are identified by the model as high risk that will truly experience the outcome being predicted)
iii. specificity (proportion of the population who does not experience the outcome of interest that the prediction model successfully identifies)
iv. negative predicted value (proportion of patients who are identified by the model as low risk that will not truly experience the outcome being predicted)
v. accuracy (the proportion of patients who are predicted correctly by the model as high risk and low risk) for three ADR models (major bleeding, non-major bleeding, acute kidney injury).
Economic criteria and best practices. With the advancements of information technologies, provider messaging alerts are a potential strategy for actionable ADR prevention interventions delivered via customized information sent from managed care organizations and obtained from claims data processing at the patient level. These message alerts are usually prepared for high risk patients identified via predictive modeling. For each patient, a list of the most important risk factors is summarized together with a recommendation for drug therapy changes.
With the above in mind, a major question arises as to how to calculate the optimum volume of high risk patients identified by the predictive model in order to prevent severe ADR from cost effectiveness standpoint. Based on our experience, the volume of high risk patients depends on multiple criteria including the traditional statistical validation metrics described in the previous section, cost-savings calculations together with rules of thumb based on best practices. Similar to the validation metrics, the costs, savings, saving/cost ratios were computed for the top population percentiles (from 1 to 50%). That is, the population percentiles were rank ordered according to the probability of risk from higher to lower.
The costs included the expenditures attributed to the interventions designed for messaging the information to the providers including mailing and information technology infrastructure costs. The savings were calculated in terms of the improvement achieved in terms of emergency room and inpatient visit prevention for the ADR outcomes of interest (major bleeding, non-major bleeding, and acute kidney injury).
Different scenarios were generated for different costs (calculated per patient in the targeted volume for messaging to the providers) and effectiveness levels of ADR prevention (i.e., % of savings achieved in terms of emergency room and inpatient stay visits). A detailed discussion can be found elsewhere based on cost-savings calculations for medical events [26].
In addition to the above, a discussion of different stakeholders brought into light the need to incorporate best corporate practices typically implemented in clinical programs. One such a practice involves the incorporation of a cost-saving ratio of 3:1 to achieve satisfactory results. Another practice is the determination of a conservative estimate for the level of improvement required for prevention purposes. It was established that a 15% prevention improvement would be a good start for such efforts. In addition, a savings level of at least $200K per ADR outcome is required to offset indirect costs not typically included in the cost of interventions.
Algorithm integrating multiple criteria for clinical decision making process. The algorithm relies on optimizing the above mentioned multiple criteria by determining the minimum values for inclusion in the algorithmic process. As such, a decision is reached to distinguish high risk from low risk patients with the desired target volume for provider messaging.
Results
Incidence rates and healthcare costs for severe ADR outcomes
The incidence rates and healthcare expenditures for ER and inpatient stay visits are summarized for bleeding and acute kidney injury (Table 1). Bleeding has been identified as major (e.g., intracranial) and non-major (e.g., gastrointestinal) and is induced as adverse events of anticoagulants and antiplatelet agents. In general, major bleeding is a rarer event (1 for ER and 2.13 for hospital admissions per 1000 population) relative to non-major bleeding (15.69 for ER and 9.04 for hospital admissions per 1000 population) and is much costlier in terms of healthcare system consumption and usage ($56,133 for major bleeding inpatient stay relative to $ 13,393 for non-major bleeding).
Acute kidney injury is also a rarer event than non-major bleeding in terms of ER and inpatient stay incidence rates. On the other hand, the average cost for an inpatient stay is comparable for both ADR outcomes. While the incidence rates for acute kidney injury are higher than those for major bleeding, the average cost of inpatient stay is slightly less than 25% of that for major bleeding.
In general, the annual ER or inpatient incidence rates for the Medicare population are the highest followed by those for the Medicaid population, then the Commercial cohorts. On the other hand, the average healthcare expenditure per person in the Medicare population is the lowest, followed by the Medicaid,
then the Commercial populations. It should be pointed out that the Medicare program has strict government contracting rules, thereby, explaining the lower medical costs despite the high incidence rates among all populations. The demographic characteristics of the populations exposed to medications are documented in Table 1.
![Click here to view Large Table 1a Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T001a.png)
![Click here to view Large Table 1b Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T001b.png)
![Click here to view Large Table 1c Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T001c.png)
Process for patient identification at high risk of ADRs
Validation metrics for severe ADR predictive models. To prevent severe ADR outcomes in scaled up diverse populations, the question becomes what is the method used in determining the volume of targeted patients for prevention purposes. This question is answered with the information provided in this section and the following one. Traditionally, the first step in this process is to determine the probability thresholds corresponding to acceptable levels of sensitivity, specificity, and positive predictive value. The validation metric for the three ADR predictive models are provided in Table 2.
![Click here to view Large Table 2 Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T002.png)
The results are shown for different top percentiles of the population (up to 50%). Furthermore, the threshold for detecting high risk patients as a function of sensitivity and specificity is reasonable (e.g., 65% or more for both measures). The positive predicted value (PPV), however, for the three severe ADR outcomes were in the low range with the highest values being around 3% for major bleeding, 10% for non-major bleeding, and 11% for acute kidney injury. Traditionally, the PPV values are in excess of 35% for valid models as measured in terms of area under the curve. The low values obtained for PPV are explained to a large part by the low incidence rates for severe ADRs, that is, rare medical events.
This in turn results in high false positives as shown in (Table 2). Therefore, this is a major issue which needs to be addressed for the proper prevention of severe ADRs. On the basis of the aforementioned results, one can conclude that traditional statistical measures of sensitivity, specificity and area under the curve cannot be solely relied on to make clinical decisions. Both true and false positives should also be taken into account for clinical decision-making for targeting volumes of high risk patients [27].
Economic criteria and best practices. The volume of targeted high risk patients usually depends in large part on cost-savings analyses as shown in Table 3. In this scenario, the cost per patient is estimated at $5.10 and the intervention is effective at preventing 15% of the ADR outcomes. Although cost-savings calculation is an essential ingredient in determining the optimum level of targeted high risk patients for prevention purposes, additional parameters should be taken into account. In essence, this process depends on multiple criteria including
![Click here to view Large Table 3a Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T003a.png)
![Click here to view Large Table 3b Insights in Mining Science & Technology](images/JOJIM.MS.ID.555552.T003b.png)
a. The levels of sensitivity, specificity and positive predictive values
b. Savings to cost ratio (e.g., 3 to 1 ratio)
c. Overall level of savings across all ADR outcomes (e.g., $200K).
Algorithm integrating multiple criteria. Table 3 provides a final summary of recommended volume of targeted patients which satisfies a number of criteria discussed among a number of stakeholders in the organization:
a) Reasonable sensitivity and specificity levels (e.g., > 65%)
b) Minimum levels of 3 to 1 savings/cost ratio
c) Overall savings in excess of $200,000 per ADR.
Due to the high rate of false positives, an inflation ratio of 20% was also implemented in order to safeguard against not achieving the best business practices particularly the amount of total savings per ADR outcome. It should be noted that the percentage of false positives is excessively high for severe ADR outcomes. This is a potential barrier which needs to be addressed in future research so as to improve the effectiveness of actionable interventions.
Discussion
Although severe ADR outcomes are life-threatening and have a major toll upon healthcare systems in terms of ER visits and hospital admissions, up to 50% of these ADR-related healthcarevisits can be prevented. Therefore, it is paramount to address this issue as part of the larger issue of increased healthcare expenditures by advancing the science of ADR prevention. Traditionally, attention has been given only to identifying atrial fibrillation patients who are at high risk of bleeding via exposure to anticoagulants and antiplatelet agents [14-20]. These efforts have mostly relied on probability thresholds ingrained in statistical criteria or clinical scores to determine the high risk patients. In dealing with large and diverse volumes of patient populations, there is a need to develop a process which integrates scientific methods and criteria based on best business practices. The goal of this short report was to fill in this gap with the dual goals of improving the total cost and quality of care. Based on a review of the literature, it is very seldom that researchers report detailed model performance metrics (e.g., sensitivity, positive predicted value, accuracy, positive likelihood ratio, area under the curve) such as those documented in Piovella et al. [21].
This is necessary information to build cost-effective business cases for deploying actionable interventions in addition to the cost-savings criteria [26]. In the process, we have discovered that best business practices should be implemented in order to reach optimal clinical decisions for the ultimate prevention of severe ADRs such as the selection of cost-savings ratio, minimum savings required per ADR outcome, as well as, the % of prevention intervention a stakeholder is willing to adopt in order to determine the optimum volume of targeted patients in scaled-up and diverse populations.
One should note that severe ADR outcomes are rare medical events the prediction of which may yield a high percentage offalse positive despite good valid metrics for the prediction model at hand. Potential solutions can be the development of clinical rules with customized information (e.g., detailed lab marker data) about the patient for post processing purposes in which the number of false positives is greatly reduced
Other alternative would be the refinement of medication exposure profiles with an intent to increase the prevalence of ADR outcome in question [28]. Further research is warranted to address these barriers. Future efforts will report a clinical utility assessment of the provider messaging intervention destined to prevent severe ADRs. Collectively, there is a growing importance for the conduct of post-drug marketing studies to enrich the scientific literature about the adverse events and prevention of ADRs as advocated by regulatory agencies such as the US FDA [29-32].
Acknowledgments
The authors have no conflict of interest to declare and would like to thank the Anthem Data Science team (Tao Chen, Joon-Ku Im, Zhen Hu, James Allison) for building the models described in this work.
References
- Bouvy JC, De Bruin MN, Koopmanschap MA (2015) Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf 38(5): 437-453.
- VA Center for Medication Safety and VHA Pharmacy benefits Management Strategic Healthcare Group and the Medical Advisory Panel. Adverse drug events, adverse drug reactions and medication errors. Frequently asked questions.
- Nebeker JR, Barach P, Samore MH (2004) Clarifying adverse drug events: a clinician’s guide to terminology, documentation, and reporting. Ann Intern Med 140(10): 795-801.
- Edwards IR, Aronson JK (2000) Adverse drug reactions: definitions, diagnosis, and management. Lancet 356(9237): 1255-1259.
- Budnitz DA, Lovergrove MC, Shehab N, Richards CL (2011) Emergency hospitalizations for adverse drug events in older Americans. N England J Med pp. 365.
- Angamo MT, Chalmers L, Curtain CM, Bereznicki LR (2016) Adverse-drug-reaction-related hospitalisations in developed and developed countries: a review of prevalence and contributing factors. Drug Saf 39(9): 847-857.
- Sultana J, Cutroneo P, Trifiro G (2013) Clinical and economic burden of adverse drug reactions. J Pharmacol Pharmacother 4(1): S73-S77.
- Wu C, Bell CM, Wodchis WP (2012) Incidence and economic burden of adverse drug reactions among elderly patients in Ontario emergency departments. A retrospective study. Drug Saf 35(9): 769-781.
- Hug BL, Keohane C, Seger DL, Yoon C, Bates DW (2012) The costs of adverse drug events in community hospitals. Jt Comm J Qual J Qual Pat Saf 38(3): 120-126.
- Hakkarainen KM, Hedna K, Petzold M, Hagg S (2012) Percentage of patients with preventable adverse drug reactions and preventability of adverse drug reactions-a meta-analysis. Plos one 7(3): e33236.
- Falconer N, Barras M, Cottrell N (2018) Systematic review of predictive risk models for adverse drug events in hospitalized patients. Br J Clin Pharmacol 84(5): 846-864.
- Nair NP, Chalmers L, Peterson GM, Bereznicki BJ, Castelino RL, et al. (2016) Hospitalization in older patients due to adverse drug reactions-the need for a prediction tool. Clin Intervent Aging 11: 497-505.
- Stevenson JM, Williams JL, Burnham TG, Prevost AT, Schiff R, et al. (2014) Predicting adverse drug reactions in older adults; a systematic review of the risk prediction models. Clin Intervent Aging 9: 1581-1593.
- Iuga A, Genaidy A (2018) Comment on Falconer et al (2018) the need for specific adverse drug reaction outcomes. Br J Clin Pharmacol 84(8): 1856.
- Gage BF, Yan Y, Milligan PE, Waterman AD, Culverhouse R, et al. (2006) Clinical classification schemes for predicting hemorrhage: results from the National Registry of Atrial Fibrillation (NRAF). Am Heart J 151(3): 713-719.
- Shireman TI, Mahnken JD, Howard PA, Kresowik TF, Hou Q, et al. (2006) Development of a contemporary bleeding risk model for elderly warfarin recipients. Chest 130(5): 1390-1396.
- Ruiz-Gimenez N, Suarez C, Gonzalez R, Todolí JA, Samperiz AL, et al. (2008) Predictive variables for major bleeding events in patients presenting with documented acute thromboembolism. Findings from the RIETE registry. Thromb Haemost 100(1): 26-31.
- Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJGM, et al. (2010) A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation. The Euro Heart Survey. Chest 138(5): 1093-1100.
- Fang MC, Go AS, Chang Y, Borowsky LH, Pomernacki NK, et al. (2011) A new risk scheme to predict warfarin-associated hemorrhage. J Am College Cardiol 58(4): 395-401.
- O’Brien EC, Simon DN, Thomas LE, Hylek EM, Gersh BJ, et al. (2015) The ORBIT bleeding score: a simple bedside score to assess bleeding risk in atrial fibrillation. Eur Heart J 36(46): 3258-3264.
- Piovella C, Valle FD, Trujillo-Santos J, Pesavento R, Lopez L, et al. (2014) Comparison of four scores to predict major bleeding in patients receiving anticoagulation for venous thromboembolism: findings from the RIETE registry. Intern Emerg Med 9(8): 847-852.
- Hippisley-Cox J, Coupland C (2014) Predicting risk of upper gastrointestinal bleed and intracranial bleed with anticoagulants: cohort study to derive abd validate QBleed scores. Br Med J 349: g4606.
- (2013) Statistical Analysis Software. Base SAS 9.4 procedures guide: statistical procedures, Second Edition. SAS: Cary, NC, USA.
- Ogutu JO, Piepho HP, Schulz-Streeck T (2011) A comparison of random forests, boosting and support vector machines for genomic selection. BMC Proceedings 5(3): S11.
- Lewis G, Curry N, Bardsley M (2011) Choosing a predictive risk model: a guide for commissioners England.
- Billings J, Dixon J, Mijanovich T, Wennberg D (2006) Case finding for patients at risk of readmission to hospital: development of algorithm to identify high risk patients. Br Med J 333(7563): 327.
- Vickers AJ, Van Calster B, Steyberg EW (2016) Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. Br Med J 352: i6.
- Abrahamowicz A, Beauchamp ME, Sylvestre MP(2012) Comparison of alternative models for linking drug exposure with adverse effects. Stat Med 31(11-12): 1014-1030.
- Corrigan Curay J, Sacks L, Woodcock J (2018) Real-world evidence and real-world data for evaluating drug safety and effectiveness. J Am Med Assoc 320(9): 867-868.
- Decousus H, Tapson VF, Bergmann JF, Chong BH, Froehlich JB, et al. (2011) Factors at admission associated with bleeding risk in medical patients: findings from the IMPROVE investigators. Chest 139(1): 69-79.
- (2018) National Institute for Health and Care Excellence. Acute kidney injury: prevention, detection and management.
- Dreischulte T (2014) Preventing drug-induced acute kidney injury in the community.