Validation and modification of existing mortality prediction models for lower gastrointestinal bleeding: a retrospective study

Abstract

Background

Lower gastrointestinal bleeding (LGIB) often subsides without medical intervention; however, in some cases, the bleeding does not stop and the patient’s condition worsens. Therefore, predicting severe LGIB in advance can aid treatment. This study aimed to evaluate variables related to mortality from LGIB and propose a scoring system.

Methods

In this retrospective study, we reviewed the medical records of patients who visited the emergency room with hematochezia between January 2016 and December 2020. Through regression analysis of comorbidities, medications, vital signs, laboratory investigations, and duration of hospital stay, variables related to LGIB-related mortality were evaluated. A scoring system was developed and the appropriateness with an area under the receiver operating characteristics curve (AUROC) was evaluated and compared with other existing models.

Results

A total of 932 patients were hospitalized for LGIB. Variables associated with LGIB-related mortality were the presence of cancer, heart rate > 100 beats/min, blood urea nitrogen level ≥ 30 mg/dL, an international normalized ratio > 1.50, and albumin level ≤ 3.0 g/dL. The AUROCs of the models CNUH-4 and CNUH-5 were 0.890 (p < 0.001; cutoff, 2.5; 95% confidence interval, 0.0851–0.929) and 0.901 (p < 0.001; cutoff, 3.5; 95% confidence interval, 0.869–0.933), respectively.

Conclusions

The model developed for predicting the risk of LGIB-related mortality is simple and easy to apply clinically. The AUROC of the model was better than that of the existing models.

Background

Gastrointestinal bleeding (GIB) is anatomically classified based on the ligament of Treitz. Upper gastrointestinal bleeding (UGIB) is defined as bleeding above the ligament of Treitz, while lower gastrointestinal bleeding (LGIB) is defined as bleeding below the ligament of Treitz. Recently, with the introduction of small intestine endoscopy, small intestine bleeding is referred to as mid-gut bleeding, while colon and rectal bleeding below the ileocecal valve is called LGIB [1]. LGIB accounts for 20–40% of all GIB [1,2,3]. Compared with past data, the incidence of UGIB has reduced, whereas that of LGIB has increased. In the United States, the incidence of UGIB was 87 per 100,000 individuals in 1996 but decreased to 47 per 100,000 individuals in 2005. In contrast, the incidence of LGIB was 20 per 100,000 individuals in 1996 but increased to 33 per 100,000 individuals in 2005 [4, 5].

Various studies have evaluated the risk factors and mortality related to UGIB. Evaluation methods, such as AIMS65 [6] and Protetto Nazionale Emorragia Digestiva [7], which can predict UGIB-related mortality, and the Glasgow Blatchford Score (GBS) [8], which predicts the need for intervention after hospitalization, have been developed and applied in clinical practice. However, studies on LGIB are fewer than those on UGIB. This is because the incidence of LGIB is less than that of UGIB, and 80–85% of LGIB cases improve spontaneously without requiring hospitalization [9]. However, various studies are currently being undertaken to evaluate the mortality, severity, and risk factors of LGIB.

Compared with UGIB, no specific method has been established for evaluating the severity of LGIB. Hence, AIMS65, which is used for UGIB evaluation, is also used for LGIB evaluation. Recently, the age, blood tests, and comorbidities (ABC) score has been successfully developed for predicting LGIB-related mortality [10]. Thus, in this study, we aimed to evaluate variables related to mortality from LGIB and propose a scoring system. Additionally, this scoring system will be compared to AIMS65 and ABC.

Methods

Study design and population

In this retrospective study, all patient information was obtained through electronic medical records. Patients with hematochezia aged > 19 years who visited the emergency room (ER) of Chungnam National University Hospital from January 1, 2016 to December 31, 2020 were included. Among these, patients discharged from the hospital on the same day after treatment in the ER were excluded and only patients who received inpatient treatment were included. Causes of LGIB were classified based on investigational results such as duodenoscopy, colonoscopy, angiography, and abdominal hemorrhagic computed tomography (CT) among hospitalized patients.

Selection of predictive variables

Using previous studies as a reference, variables that are expected to be associated with LGIB-related mortality were searched using electronic medical records. Basic patient-related information included age, sex, comorbidities (cancer, heart failure [HF], renal failure [RF], liver cirrhosis [LC], chronic obstructive pulmonary disease [COPD]), and medication history (anticoagulant, antiplatelet, and non-steroidal anti-inflammatory drugs [NSAIDs]). Data on vital signs and blood test (hemoglobin level, hematocrit percentage, platelet count, albumin level, blood urea nitrogen [BUN] level, creatinine level, and international normalized ratio [INR]) at the ER, hospitalization duration, intensive care unit treatment duration, and death were also collected. The American Society of Anesthesiologists (ASA) scores are required to calculate the ABC score.

Primary outcome and definitions

LGIB was defined as colon and rectal bleeding below the ileocecal valve confirmed through duodenoscopy, colonoscopy, angiography, and hemorrhagic CT. The primary outcome was to investigate variables related to all-cause mortality of patients hospitalized with LGIB. The secondary outcome was to evaluate the area under the receiver operating characteristics curve (AUROC) values of the primary outcomes. The newly introduced AUROC in this study will be called CNUH, which stands for Chungnam National University Hospital. The CNUH was compared with those of existing models (ABC and AIMS65).

Statistical analyses

For continuous variables, the mean value was used when the variable values were relatively evenly distributed, and the median value was used otherwise. Categorical variables are represented as numbers (percentages). Univariate logistic regression analysis was used to investigate variables associated with all-cause mortality, and the Hosmer–Lemeshow test was conducted to determine the suitability of the results. Multivariable logistic regression analysis was used to determine the actual role of each factor by correcting the influences of the variables with meaningful results. While selecting statistically meaningful variables in the univariable logistic regression analysis, to reduce the error of selecting confounding and suppressor variables, the p-value was set at < 0.1. For the secondary outcome, four to five variables most related to all-cause mortality in the primary outcome were selected, and then each AUROC value was compared. SPSS version 22 (IBM Corp., Armonk, NY, USA) was used for the statistical analyses.

Results

Patient characteristics

From January 2016 to December 2020, 3181 patients with hematochezia aged ≥ 19 years older were admitted to the ER. Of them, 2086 were discharged on the same day after treatment and 1095 were hospitalized for LGIB. Finally, 932 patients were diagnosed with LGIB (Fig. 1).

Fig. 1

Study flow chart

Among patients who underwent inpatient treatment, 57.3% were men, and 42.7% were women. The median age was 68 years. Underlying comorbidities included cancer 18.8%, HF 12.0%, RF 12.4%), COPD 2.9%, and LC 10.8%. ASA scores were as follows: 1 point 58.9%; 2 points 13.8%; and 3 points 27.2%. In the ER, 1045 patients were alert and 50 had altered mental status. During hospitalization, the concomitant medications were antiplatelet agents 18.8%; anticoagulants 11.1%; and NSAIDs 27.9%. The median length of hospital stay was 6 days. Moreover, 51 patients received intensive care treatment and the median duration of intensive care unit treatment was 3 days (Table 1).

Table 1 Baseline characteristics of the patients

Vital signs, laboratory results, and mortality

Among patients who underwent inpatient treatment for LGIB, 40 (3.7%) died; among these, 23 (2.1%) died from LGIB and 17 (1.6%) died from causes other than LGIB (Table 1). The median heart rate was 100 beats/min, the median systolic blood pressure was 100 mmHg, and the median diastolic blood pressure was 53 mmHg (Table 2). The mean hemoglobin level and hematocrit percentage were 9.24 g/dL and 27.22%, respectively. The median platelet count, BUN level, creatinine level, albumin level, and INR were 161,000/µL, 22.3 mg/dL, 0.95 mg/dL, 3.0 g/dL, and 1.12, respectively (Table 2).

Table 2 Vital signs and laboratory test results at the emergency room

Endoscopic results

Of 1095 patients admitted for LGIB, 850 underwent endoscopy. Among these, 144 patients underwent endoscopic hemostasis, 24 underwent transarterial embolization and one underwent both procedures (Table 3). In endoscopy, the most common cause of LGIB was ischemic colitis 16.3%, followed by diverticular bleeding 16.0%, ulcer bleeding 6.8%, cancer bleeding 6.2%, and post-polypectomy bleeding 5.6%. The cause was not found in 6 patinets complaining hematochezia and 93 patients showed no sign of bleeding on endoscopy(Table 4).

Table 3 The proportion of patients who underwent endoscopy and intervention

Table 4 Endoscopic results

Correlation with all-cause mortality

In the univariate analysis, the following factors were associated with all-cause mortality: age ≥ 75 years, presence of cancer, heart rate > 100 beats/min, systolic blood pressure < 100 mmHg, diastolic blood pressure < 60 mmHg, hemoglobin ≤ 10.0 mg/dL, hematocrit ≤ 30.0%, platelet ≤ 100,000/µL, BUN ≥ 30 mg/dL, creatinine lev > 1.5 mg/dL, albumin ≤ 3.0 g/dL, and INR > 1.50 (Table 5). The P-value of the Hosmer–Lemeshow test conducted to determine suitability was 0.996.

Table 5 Logistic regression analyses results for risk factors of mortality from lower gastrointestinal bleeding

In the multivariable logistic regression analysis, significant (p < 0.05) results were obtained with the following variables: heart rate > 100 beats/min, BUN ≥ 30 mg/dL and INR > 1.50 (Table 5).

CNUH model for the prediction of mortality

To determine the predictability of all-cause mortality from the variables, the AUROC was evaluated. Three variables with meaningful results in multivariable analysis were selected. In addition, presence of cancer and albumin ≤ 3.0 g/dL, which were not statistically meaningful but were thought to have a relative effect on all-cause mortality, were selected.

Two models were compared. The first one was called a CNUH-4 model that incorporated the following variables: cancer, heart rate > 100 beats/min, BUN ≥ 30 mg/dL, and INR > 1.50. The AUROC was drawn by assigning points from 0 to 4. The second model (CNUH-5) was created by adding an albumin ≤ 3.0 g/dL to the CNUH-4 model. The AUROC was drawn by assigning points from 0 to 5.

The score distribution in the CNUH-4 model was as follows: 0 points, 310 (33.5%) patients; 1 point, 318 (34.3%) patients; 2 points, 183 (19.8%) patients; 3 points, 91 (9.8%) patients; and 4 points, 24 (2.6%) patients. The AUROC was 0.890 (p < 0.001; cutoff, 2.5; 95% CI, 0.0851–0.929) (Fig. 2). The score distribution in the CNUH-4 model was as follows: 0 points, 223 (24.1%) patients; 1 point, 254 (27.4%) patients; 2 points, 190 (20.5%) patients; 3 points, 148 (16.0%) patients; 4 points, 88 (9.5%) patients; and 5 points, 23 (2.5%) patients. The AUROC was 0.901 (p < 0.001; cutoff, 3.5; 95% CI, 0.869–0.933), higher than that of CNUH-4 (Fig. 2).

Fig. 2

Comparison of the area under the receiver operating characteristics curve (AUROC) between CNUH-4 and CNUH-5. The AUROC of CNUH-4 (left) was 0.890 (p < 0.001; cutoff, 2.5; 95% CI, 0.0851–0.929). The AUROC of CNUH-5 (right) was 0.901 (p = 0.000; cutoff, 3.5; 95% CI, 0.869–0.933). x = 1 − sensitivity, y = sensitivity

The results of CNUH-4 and CNUH-5 were compared with those of ABC and AIMS65 using data from the Chungnam National University Hospital in 2020. The AUROC of ABC was 0.881 (p < 0.001; 95% CI, 0.817–0.945), the AUROC of AIMS65 was 0.861 (p < 0.001; 95% CI, 0.771–0.951), the AUROC of CNHU-4 was 0.893 (p < 0.001; 95% CI, 0.826–0.960), and the AUROC of CNUH-5 was 0.896 (p < 0.001; 95% CI, 0.836–0.956). When compared with data in 2020, the AUROC of CNUH-5 was the highest (Fig. 3).

Fig. 3

Comparison of various models using patient data in 2020. The area under the receiver operating characteristics curve (AUROC) of ABC (left upper) was 0.881 (p < 0.001; 95% CI, 0.817–0.945), the AUROC of AIMS65 (right upper) was 0.861 (p < 0.001; 95% CI, 0.771–0.951), the AUROC of CNHU-4 (left lower) was 0.893 (p < 0.001; 95% CI, 0.826–0.960), and the AUC of CNUH-5 (right lower) was 0.896 (p < 0.001; 95% CI, 0.836–0.956). x = 1 − sensitivity, y = sensitivity

Discussion

Several studies have introduced indicators to predict the severity and mortality of UGIB [6,7,8]. Similar to UGIB, a scoring system that can predict the prognosis of LGIB using various predictors at the time of ER visit will be useful. Although the prognosis of LGIB is better than that of UGIB, it is very important to develop a scoring system to assess the mortality risk in advance. Therefore, we re-evaluated the risk factors using the existing scoring system, including only patients with LGIB and undergoing inpatient treatment, and developed a new scoring system for clinical use.

Prior to this study, other studies have evaluated the mortality and severity of LGIB. In these studies, heart rate > 100 beats/min, systolic blood pressure < 115 mmHg, aspirin use, ≥ 2 comorbidities [11], hematocrit ≤ 35.0%, systolic blood pressure < 100 mmHg or heart rate > 100/min [12], transfusion, re-bleeding, 20% reduction of hematocrit [13], prothrombin time > 1.2-times the control, erratic mental status, unstable comorbidities [14], age ≥ 75 years, creatinine level > 150 µmol/L, and albumin level ≤ 30 g/L [9] were found to be associated with LGIB-induced severe morbidity and mortality.

Similar to previous studies, our univariable logistic regression analysis showed that specific vital signs, age, and laboratory test results were associated with LGIB-induced mortality. However, in the multivariable logistic regression analysis, variables associated with all-cause mortality were the presence of cancer, heart rate > 100 beats/min, BUN ≥ 30 mg/dL, albumin level ≤ 3.0 g/dL, and INR > 1.50. When comparing the two models, CNUH-5 had a higher AUROC value than CNUH-4 (0.901 vs. 0.890).

Two previous validation and derivation studies on LGIB severity have reported that the AUROC was 0.754 for the validation study and 0.761 for the derivation study [11, 15]. In a study comparing the relationship between eight variables and 30-day mortality, the AUROC was 0.72 [16]. In a study that performed external validation with the existing NOBLADS score, the in-hospital mortality rate was > 5 points (AUROC, 0.83) [17]. Compared with these studies, the AUROC of the CNUH model was found to be higher.

Few studies have evaluated the prognosis of LGIB. AIMS65, GBS, and Oakland scores have been found to be useful in assessing LGIB, and AIMS65 is known to be highly correlated with LGIB-related mortality [14, 18]. In this study, we compared AIMS65, which has the highest mortality prediction ability, with the recently developed ABC. Albumin level, INR, mental status, systolic blood pressure, and age are required to calculate the AIMS65 score, while age, BUN, albumin level, creatinine level, mental status, LC, and malignancy are required to calculate the ABC score. However, the CNUH model does not include mental status, ASA score, and age; thus, a direct comparison was impossible. Therefore, we used the CNUH model as the comparator and validated the ABC and AIMS65 models.

The AUROCs of ABC, AIMS65, CNUH-4, and CNUH-5 with patient data in 2020 were 0.881, 0.861, 0.893, and 0.896, respectively. A slight difference was observed between the AUROCs of CNUH-4 and CNUH-5 when all enrolled patients were included. It is meaningful that the model presented in this study had a relatively higher AUROC than the two existing ones, although the patient’s information or score calculation method was simpler with the CNUH model.

This study has some limitations. In this retrospective study, all information was obtained from electronic medical records. This method does not provide adequate information on the change of conditions from the time of admission to death or discharge. LGIB often improves spontaneously and has a low mortality rate [19,20,21,22]. Because of the low all-cause mortality rate, the OR values of some variables, such as heart rate, BUN, albumin, and INR, might be overestimated. In addition, there might be a selection bias in targeting only patients hospitalized for LGIB. However, the scoring system was exclusively developed for patients who visited the hospital with hematochezia and in whom LGIB was clearly identified. Finally, this study was based on a single tertiary-care institution; our results need to be validated in other settings in a larger cohort.

Conclusions

We developed a model for predicting the risk of LGIB-related mortality, and five variables related to all-cause mortality were identified. Our model is simpler than the existing model. thus, it can be quickly applied when evaluating patients in the ER. Further studies are required to validate our results in a larger cohort.