Predictive validation of existing bleeding and thromboembolic scores in elderly patients with comorbid atrial fibrillation and acute coronary syndrome

Hong-Hong ZHANG, Qi LIU, Hai-Jing ZHAO, Ya-Ni YU, Liu-Yang TIAN, Ying-Yue ZHANG, Zi-Hao FU, Li ZHENG, Yue ZHU, Yu-Han MA, Shuang LI, Yang-Yang MA, Yu-Qi LIU,,

1.Medical School of Chinese PLA, Beijing, China; 2.Department of Cardiology & National Clinical Research Center of Geriatrics Disease; Beijing Key Laboratory of Chronic Heart Failure Precision Medicine; National Key Laboratory of Kidney Diseases, Chinese PLA General Hospital, Beijing, China; 3.Department of Cardiology, the Sixth Medical Center,Chinese PLA General Hospital, Beijing, China; 4.Department of Information, Chinese PLA General Hospital, Beijing,China

ABSTRACT BACKGROUND The validation of various risk scores in elderly patients with comorbid atrial fibrillation (AF) and acute coronary syndrome (ACS) has not been reported.The present study compared the predictive performance of existing risk scores in these patients.METHODS A total of 1252 elderly patients with AF and ACS comorbidities (≥ 65 years old) were consecutively enrolled from January 2015 to December 2019.All patients were followed up for one year.The predictive performance of risk scores in predicting bleeding and thromboembolic events was calculated and compared.RESULTS During the 1-year follow-up, 183 (14.6%) patients had thromboembolic events, 198 (15.8%) patients had BARC class ≥2 bleeding events, and 61 (4.9%) patients had BARC class ≥ 3 bleeding events.For the BARC class ≥ 3 bleeding events, discrimination of the existing risk scores was low to moderate, PRECISE-DAPT (C-statistic: 0.638, 95% CI: 0.611-0.665), ATRIA (C-statistic:0.615, 95% CI: 0.587-0.642), PARIS-MB (C-statistic: 0.612, 95% CI: 0.584-0.639), HAS-BLED (C-statistic: 0.597, 95% CI: 0.569-0.624)and CRUSADE (C-statistic: 0.595, 95% CI: 0.567-0.622).However, the calibration was good.PRECISE-DAPT showed a higher integrated discrimination improvement (IDI) than PARIS-MB, HAS-BLED, ATRIA, and CRUSADE (P ＜ 0.05) and the best decision curve analysis (DCA).For thromboembolic events, the discrimination of GRACE (C-statistic: 0.636, 95% CI: 0.608-0.662) was higher than CHA2DS2-VASc (C-statistic: 0.612, 95% CI: 0.584-0.639), OPT-CAD (C-statistic: 0.602, 95% CI: 0.574-0.629) and PARIS-CTE(C-statistic: 0.595, 95% CI: 0.567-0.622).The calibration was good.Compared to OPT-CAD and PARIS-CTE, the IDI of the GRACE score slightly improved (P ＜ 0.05).However, NRI analysis showed no significant difference.DCA showed that the clinical practicability of thromboembolic risk scores was similar.CONCLUSIONS The discrimination and calibration of existing risk scores in predicting 1-year thromboembolic and bleeding events were unsatisfactory in elderly patients with comorbid AF and ACS.PRECISE-DAPT showed higher IDI and DCA than other risk scores in predicting BARC class ≥ 3 bleeding events.The GRACE score showed a slight advantage in predicting thrombotic events.

Atrial fibrillation (AF) is one of the most common arrhythmias in patients with acute coronary syndrome (ACS).Previous studies showed that 6%-21%[1]of patients with coronary artery disease (CAD) also had AF, and the proportion of concomitant CAD in patients with AF is 25%-30%.[2]Patients with AF and ACS are commonly associated with a higher risk of ischemia and bleeding, which also seems to apply to AF after cardioversion.[3]Due to pathological characteristics, comor-bidities, polypharmacy, and poor medication adherence, elderly patients face a higher risk of adverse events.Doctors tend to first consider the safety of antithrombotic therapy in real-world practice.This intuitive assessment and conservative treatment lead to a lack of standardized antithrombotic therapy in this population.[4]

Accurate risk assessment is key to determining antithrombotic therapy dose, combination, and duration.Current guidelines recommend optimal antithrombotic regimens based on existing thromboembolic and bleeding risk scores.[5-7]The default strategy for patients with comorbid AF and ACS after discharge is dual antithrombotic therapy, which includes an oral anticoagulant and P2Y12 inhibitor.[7]For patients with AF or ACS, the following scoring systems have been established for risk assessment:(1) thromboembolic event (TE), including GRACE,[8]CHA2DS2-VASc,[9]PARIS-CTE,[10]and OPT-CAD,[11]and (2) bleeding events, including HAS-BLED,[12]ATRIA,[13]PRECISE-DAPT,[14]PARIS-MB,[10]and CRUSADE.[15]HAS-BLED and CHA2DS2-VASc scores are currently used to evaluate the risk of bleeding and thrombosis in patients with AF.[6,7]GRACE and CRUSADE scores are generally used to evaluate the risk of ischemic and bleeding events in patients with ACS, respectively.Elderly patients with AF and ACS have a high risk of thrombosis and bleeding.Few studies evaluated the predictive validation of existing risk scores for patients with comorbid ACS and AF.Therefore, the present study externally validated and compared existing scores, including HAS-BLED, ATRIA, PRECISE-DAPT, PARIS,CRUSADE, GRACE, CHA2DS2-VASc, and OPT-CAD,based on their predictive validation of ischemic/thrombotic or bleeding events in this population.

METHODS

Study Population

From January 2015 to December 2019, elderly patients with AF and ACS in Chinese PLA General Hospital were consecutively enrolled in this retrospective study.The following inclusion criteria were used:(1) age 65 years or older; (2) diagnosis of AF at discharge, including paroxysmal, persistent, or permanent AF at admission or medical history of AF;and (3) diagnosis of ACS at discharge, including unstable angina (UA), non-ST-segment elevation myocardial infarction (NSTEMI), and ST-segment elevation myocardial infarction (STEMI).The following exclusion criteria were used: (1) valvular AF secondary to severe mitral stenosis or artificial heart valve history; (2) allergies or contraindications to aspirin, clopidogrel, ticagrelor, dabigatran, rivaroxaban, or warfarin; (3) psychosocial diseases, alcohol or drug abuse; (4) in-hospital death or noncardiac death; (5) reversible causes of AF; (6) incomplete or missing records of key variables; and (7) loss of follow-up after discharge.The study complied with the Declaration of Helsinki and was approved by the Ethics Committee of Chinese PLA General Hospital (NO.S2022-485-02) and registered as ChiCTR 2200067185.

Data Collection

Clinical baseline data such as patient demographics, medical history, physical examination, laboratory tests, and treatment regimens were obtained from an electronic data capture system.HAS-BLED, ATRIA, PRECISE-DAPT, PARIS, CRUSADE, GRACE,CHA2DS2-VASc, and OPT-CAD scores were calculated for each patient using online calculators or corresponding scoring criteria.[8-15]

Follow-up and Outcomes

The primary endpoints were: (1) thromboembolic events, including myocardial infarction, ischemic stroke, noncentral nervous system embolism, definite or probable stent thrombosis, and target vessel revascularization, and (2) bleeding events.Bleeding events were defined as major bleeding (BARC 3 or 5 bleeding) and any bleeding (BARC 2, 3 or 5 bleeding) in accordance with the Bleeding Academic Research Consortium (BARC) definitions.[16]All eligible patients were followed up for 12 months by trained cardiovascular physicians through repeated outpatient visits, telephone calls, or medical record reviews.When first reporting suspected adverse events,investigators verified the raw data and traced the adverse events.Some of the follow-up results were randomly selected for verification.The independent Clinical Events Committee adjudicated all of the endpoint events of the study, and these members were blinded to baseline information.

Statistical Analysis

Continuous variables are expressed as means ±SD or medians [interquartile range (IQR)], and were compared using Student'st-test or the Mann-WhitneyUtest.Categorical variables are expressed in numbers and percentages and were compared using Pearson’sχ2test or Fisher’s exact test.Risk scores were entered into Cox regression analyses as continuous and categorical variables to evaluate their contribution to 1-year endpoints.Receiver operating characteristic (ROC) curves were generated to assess the discrimination performance of the risk scores in predicting thromboembolic and bleeding events.We calculated the different areas under two ROC curves using DeLong’s method, and integrated discrimination improvement (IDI) and net classification improvement (NRI), which represent the improvement of discrimination.[17]To evaluate the calibration of risk scores, we performed calibration curve analysis.The net clinical practicability of risk scores was quantified using decision curve analysis (DCA).[18]Survival curves are depicted using the Kaplan-Meier method and were compared using the log-rank test.P＜ 0.05 was considered statistically significant.Statistical analyses were performed using R 4.2.1 (R Core Team, Vienna, Austria) and MedCalc 20.0.2.2(MedCalc Software, Belgium).

RESULTS

Baseline Characteristics of the Study

From January 2015 to December 2019, the lost to follow-up rate was 7.2%, and 1252 elderly patients with AF and ACS were ultimately enrolled in the analysis (Supplementary Figure S1).The baseline characteristics of the patients based on the incidence of bleeding and TE are summarized in Table 1 and Supplementary Table S1.The median age was 77 years (IQR 71-83), and 790 patients (63.1%) were male.A total of 37.1% of patients were aged 80 years or older.Patients with bleeding events were older and had lower body mass index (BMI), estimated glomerular filtration rates, and hemoglobin concentration but higher rates of chronic renal insufficiency,heart failure, prior bleeding, admission diagnosis of persistent and permanent AF, stent implantation and NT-proBNP level.They tended to use clopidogrel, warfarin, and triple antithrombotic therapy at discharge (allP ＜0.05).Patients with TE were older and more likely to have higher NT-proBNP levels,and white blood cell counts.They had a higher prevalence of comorbidities, such as heart failure, dyslipidemia, diabetes mellitus, renal insufficiency, peripheral arterial disease, prior coronary artery bypass grafting, and prior myocardial infarction (allP ＜0.05).Patients with the primary endpoint were more likely to be admitted for myocardial infarction.Furthermore, Cox regression analysis was performed to test the prognostic significance of antithrombotic strategies for BARC type 3 or 5 bleeding and thromboembolic events.As shown in Figure S2, triple therapy significantly correlated with incidences of BARC type 3 or 5 bleeding (HR: 3.297, 95% CI: 1.815-5.988,P ＜0.001), and single antiplatelet treatment was independently associated with an increased risk of TE(HR: 1.852, 95% CI: 1.015-1.852,P= 0.040).

Table 1 Baseline characteristics of complete studycohort.

Clinical Outcomes

During the follow-up, 183 (14.6%) patients had TE, 198 (15.8%) patients had BARC class ≥ 2 bleeding,and 61 (4.9%) had BARC class ≥ 3 bleeding events.Based on Cox proportional hazard regressions, GRACE,PARIS-CTE, and OPT-CAD scores significantly correlated with the incidence of TE (allP ＜0.05) (Supplementary Table S2).Regardless of variable type,all bleeding risk scores were independent prognostic factors of primary or any bleeding.The HAS-BLED score was statistically significant for BARC class ≥ 3 bleeding as a continuous variable but not a categorical variable score (Supplementary Table S3).Kaplan-Meier curves showed that for PRECISE-DAPT,PARIS, and CRUSADE scores, there were significant differences in the cumulative BARC type 3 or 5 bleeding risk by log-rank test (allP＜0.05).However,there was some overlap in these curves, with only the highest-risk stratum showing an increased risk of major bleeding (Figure S3).GRACE, PARIS-CTE,and OPT-CAD risk stratification exhibited an increasing cumulative CTE risk (log-rankP＜0.001)(Figure S4).

Predictive Validation of Risk Scores

For BARC class ≥3 bleeding, ROC curve analysis of CRUSADE (C-statistic: 0.595, 95% CI: 0.567-0.622)and HAS-BLED (C-statistic: 0.597, 95% CI: 0.569-0.624) showed threshold discriminative performance whereas PRECISE-DAPT (C-statistic: 0.638,95% CI: 0.611-0.665), ATRIA (C-statistic: 0.615, 95%CI: 0.587-0.642) and PARIS-MB (C-statistic: 0.612,95% CI: 0.584-0.639) showed moderate performance (Figure 1A).Compared with other risk scores,the C-statistic: of the PRECISE-DAPT score for BARC class ≥ 3 bleeding showed no superiority using the Delong test (Table 2).The PRECISE-DAPT score showed a higher IDI than the other risk scores (P＜0.05) and a higher NRI than the CRUSADE score (P=0.001, Table 2).The calibration of the above risk scores was good (Supplementary Figure S5).

Figure 1 Receiver operating characteristic curves of different risk scores.(A): BARC type 3 or 5 bleeding; and (B): thromboembolic events.

Table 2 The discrimination andcalibration of thePRECISE-DAPTscore in predicting the1-year incidenceof BARC class≥3bleedingcomparedtothe PARIS-MB, HASBLED, ATRIAand CRUSADEscores.

For TE events, the C-statistic: of PARIS-CTE was threshold (C-statistic: 0.595, 95% CI: 0.567-0.622), while GRACE (C-statistic: 0.636, 95% CI: 0.608-0.662),CHA2DS2-VASc (C-statistic: 0.612, 95% CI: 0.584-0.639), and OPT-CAD (C-statistic: 0.602, 95% CI:0.574-0.629) scores had moderate discriminative capacities (Figure 1B).Briefly, the predictive performance of risk scores for TE events remained low to moderate, and the GRACE score displayed higher discrimination compared to the OPT-CAD score(ΔC-statistic = 0.033, 95% CI: 0.002-0.064,P=0.035).Moreover, GRACE showed higher IDI than OPTCAD (ΔC-statistic: 0.009, 95% CI: 0.002-0.016,P=0.014) and PARIS-CTE (ΔC-statistic: 0.013, 95% CI:0.002-0.023,P=0.024).In addition, the GRACE score showed no significant difference compared with the CHA2DS2-VASc score (Table 3).As shown in the calibrated curves, the CHA2DS2-VASc score was relatively better than the GRACE score (Supplementary Figure S6).

Table 3 The discrimination andcalibration of theGRACE scoreinpredictingthe 1-year incidenceof thromboembolic events compared to theCHA2DS2-VASc, OPT-CAD,and PARIS-CTEscores.

Decision Curve Analysis of Risk Scores

When there are multiple predictive scores, each score has discrimination and calibration, DCA is a proper statistical method to assess the applicability of risk scores in clinical decision-making.[19]From the decision curves shown in Figure 2A, the threshold range of the PRECISE-DAPT score was the widest,and the net benefit (NB) within the most reasonable threshold probability ranges was the highest in the five curves.Therefore, it was the best optimal score.Figure 2B shows that, these risk scores were beneficial in a reasonable range of 10%-30%, which suggests their practicality in clinical application.The curves crossed in the middle of the rational range.Therefore, the NB of each risk score was similar in the critical range of the thresholds.There was no significant difference between the DCA curves of existing thrombosis risk scores.

Figure 2 Decision curves analysis of different risk scores.(A): BARC type 3 or 5 bleeding; and (B): thromboembolic events.

DISCUSSION

To the best of our knowledge, the present study is the first report to externally verify the predictive ability of risk scores for bleeding or TE risk in elderly patients with AF and ACS comorbidities.The results demonstrated that IDI analyses and DCA favored the PRECISE-DAPT score in predicting BARC class ≥ 3 bleeding.When considering the TE end-point, the predictive ability of GRACE and CHA2DS2-VASc scores was comparable.However, none of the risk scores satisfactorily predicted bleeding or TE events for elderly patients with the comorbid of AF and ACS.

The baseline characteristics significantly differed from the derived cohorts of risk scores described above.Patients in this study tended to be older.Compared to the PRECISE-DAPT score,[14]our cohort included a significantly higher percentage of females and patients with histories of previous bleeding events, peripheral vascular disease, diagnosis of unstable angina,lower WBC counts, and history of prior myocardial infarction.Triple antithrombotic therapy was more common at discharge in this cohort, and dyslipidemia was more common in the PARIS study.[10]Fewer patients in the HAS-BLED registry[12]study had low BMI and heart failure.The ATRIA-derived cohort[13]included fewer patients with diabetes mellitus and hypertension.Compared to the CRUSADE cohort,[15]the male prevalence and previous percutaneous coronary intervention (PCI) in the current study were significantly higher.Compared to the GRACE scoring cohort, the prevalence of past or current smokers in our cohort was lower, and more patients received aspirin treatment.[8]The baseline characteristics of CHA2DS2-VASc[9]and OPT-CAD[11]differed from our cohort, with higher rates of antiplatelet drugs and oral anticoagulant treatment, respectively.Notably, differences in baseline characteristics between the current study and the previously derived cohorts may significantly affect cardiovascular burden and clinical outcomes, which affect the predictive performance of the risk scores.These discrepancies partially explain the failure of the aforementioned risk scores to achieve similar predictive performance in our cohort as the derived studies for thrombotic and bleeding endpoints.We also found that the proportion of high-risk patients and the incidence of bleeding and thrombotic outcomes in this study were higher than the risk score-derived cohorts.Previous studies of patients with AF or ACS showed similar results, and the proportion of elderly individuals, comorbidities, antithrombotic therapy, and ethnicity may contribute to this discrepancy.[20-24]

Current guidelines recommend integrating risk assessment scores to determine an individual’s risk of thrombosis and bleeding,[5-7,25,26]and guide drug selection and the duration of antithrombotic therapy.Although multiple predictive scores have been developed for patients with AF or ACS, none of these scores is specific to patients with a comorbidity of AF and ACS.Limited studies assessed and compared the performance of the available predictive scores for postdischarge events and have reached controversial conclusions.Puurunen,et al.[27]found that an increased CHA2DS2-VASc score was the best predictor of thrombotic events in patients with AF referred to PCI, but only with a moderate predictive value(C-statistic: 0.57, 95% CI: 0.52-0.61), and the HASBLED score was useless in predicting bleeding events(C-statistic: 0.51, 95% CI: 0.44-0.57).A a retrospective study of 302 patients taking an oral anticoagulant undergoing PCI found that HAS-BLED and PRECISE-DAPT scores were better than other scores in predicting the risk of major bleeding based on the Thrombolysis in Myocardial Infarction criteria(TIMI).The PRECISE-DAPT score best improved the stratification of the risk of BARC class ≥ 3 bleeding complications.[24]However, this study included patients with oral anticoagulant indications other than AF and did not assess the incidence of TE.Guo,et al.[28]demonstrated that the C-statistic: of the GRACE score for ischemic stroke reached 0.715(95% CI: 0.574-0.856) in patients with AF who underwent PCI, but the CHA2DS2-VASc score (0.580,95% CI: 0.439-0.721) was relatively low.However,the relatively low endpoint events limited the interpretation of the results, and no antithrombotic therapy was recorded in the study.Our work focused on the existing bleeding scores and the thromboembolic risk scores in elderly patients with comorbid AF and ACS.Our study comprehensively verified multiple risk scores in the same external population, provided complete medication therapy and follow-up information, and had a larger sample size than most previous studies on this comorbidity population.

The incidence of bleeding or thromboembolism in the current study increased with the grade or value of the risk scores besides the HAS-BLED score.However, existing scores may be relatively limited to identifying patients with AF or ACS who tend to develop thromboembolism or bleeding.The HASBLED score cannot predict major bleeding.The present study suggested that the prediction performance of risk scores for endpoint events was lower than expected (all C-statistics ＜ 0.7).The methodological limitations of the original derivation study and the differences in baseline characteristics, treatment,and endpoints of the study population may explain the significant decline in discrimination.On the one hand, these risk scores were initially developed for patients with ACS or AF, which resulted in a relatively low proportion of comorbidity in the cohorts.On the other hand, although bleeding and thrombosis seriously affected prognosis, the elderly population was not fully included in previous cohorts of risk score derivation.The IFFANIAM registry[29]enrolled 208 patients aged ≥ 75 years with ST-segment elevation myocardial infarction.A total of 92.6% of the patients with a PRECISE-DAPT score ≥25 suggested that the recommended cut-off point should be adjusted for predicting bleeding risk in elderly individuals, and the scoring criteria for different age subgroups should be optimized.Concerning methodological limitations, 25% of patients in the HAS-BLED-derived cohort lacked information, and patients who could not take anticoagulants were excluded from the HAS-BLED[12]and CRUSADE studies,[15]which may underestimate the actual bleeding risk.The computerized database of the ATRIA score lacked potential key covariates,such as blood pressure and antiplatelet drugs.In addition, the previously developed risk scores did not use a unified definition of endpoint events.For example, PARIS and PRECISE-DAPT scores were initially used to predict bleeding complications defined by BARC or TIMI,[30]respectively.We used the BARC criteria which are considered the standard bleeding definition in the present study.Different definitions of endpoint events may also interfere with the prediction efficiency of the prediction scores.

Our study results indicated that the PRECISE-DAPT score had a higher discrimination value in identifying patients with a high risk of bleeding complications and the best clinical practicability of DCA analysis, which is roughly consistent with previous studies.In our study, 47.2% of patients had a PRECISEDAPT score ≥ 25.In the RE-DUAL PCI trial, 37.9%of patients had high bleeding risk (HBR) according to the PRECISE-DAPT score, which helped identify HBR patients and determine the intensity of the antithrombotic regimen and the optimal benefit-risk ratio for these individuals.[31]For TE scores, the GRACE and CHA2DS2-VASc scores were similar.The former score had better discrimination, and the latter score had better calibration, which may be due to the inclusion of renal function parameters in the GRACE score.Previous studies suggested that it would be beneficial to incorporate CHA2DS2-VASc and GRACE scores in ACS patients for risk stratification.[32]All scores had limited performance in predicting TE (C-statistics ＜ 0.7), which supports the need for hybrid tools to integrate clinical factors and biomarkers such as serological or imaging indicators, and the subtype of atrial fibrillation or the duration of atrial fibrillation load.[33]

Limitations

The present study had the following limitations.First, this study was a single-center retrospective study, so it was difficult to trace the information on the patient’s fragility, disability, cognitive decline,and other geriatric syndromes at the time of admission.However, a multicenter prospective study[34]showed that the impact of fragility and disability on bleeding events was low.Physicians only made decisions on the selection of antithrombotic therapy,thus may have a relatively homogeneous patient popu-lation and treatment mode.Second, the antithrombotic regimen was primarily dual antiplatelet therapy in our population.The proportion of anticoagulant and antiplatelet combination therapy consistent with the guidelines was relatively low but comparable to previous studies,[35-38]which may reflect the current situation of antithrombotic therapy in the real world for elderly patients with comorbidities of AF and ACS.[39,40]Finally, this observational study showed that more accurate evaluation scores based on multicenter data need to be established and verified in the future.

Conclusions

Our study is the first study to compare the prediction efficacy of multiple risk scores for bleeding or TE complications in elderly patients with AF and ACS comorbidities.However, the predictive performance of the risk scores in the real world was not robust in predicting the risk of bleeding and thromboembolism in these elderly patients.Predictive scores for thrombosis and bleeding in elderly patients with comorbidities of ACS and AF based on a large sample of multiple centers will be particularly valuable in the future.

ACKNOWLEDGEMENTS

This work was supported by the National Clinical Research Center for Geriatric Diseases (No.NCRCGPLAGH-20190003) and the Chinese Cardiovascular Health Alliance-Advanced Fund (No.2019-CCAACCESS-054).