Development and validation of a model to predict rebleeding within three days after endoscopic hemostasis for high-risk peptic ulcer bleeding
BMC Gastroenterology volume 22, Article number: 64 (2022)
Peptic ulcer bleeding remains a typical medical emergency with significant morbidity and mortality. Peptic ulcer rebleeding often occurs within three days after emergent endoscopic hemostasis. Our study aims to develop a nomogram to predict rebleeding within three days after emergent endoscopic hemostasis for high-risk peptic ulcer bleeding.
We retrospectively reviewed the data of 386 patients with bleeding ulcers and high-risk stigmata who underwent emergent endoscopic hemostasis between March 2014 and October 2018. The least absolute shrinkage and selection operator method was used to identify predictors. The model was displayed as a nomogram. Internal validation was carried out using bootstrapping. The model was evaluated using the calibration plot, decision-curve analyses, and clinical impact curve.
Overall, 386 patients meeting the inclusion criteria were enrolled, with 48 patients developed rebleeding within three days after initial endoscopic hemostasis. Predictors contained in the nomogram included albumin, prothrombin time, shock, haematemesis/melena and Forrest classification. The model showed good discrimination and good calibration with a C-index of 0.854 (C-index: 0.830 via bootstrapping validation). Decision-curve analyses and clinical impact curve also demonstrated that it was clinically valuable.
This study presents a nomogram that incorporates clinical, laboratory, and endoscopic features, effectively predicting rebleeding within three days after emergent endoscopic hemostasis and identifying high-risk rebleeding patients with peptic ulcer bleeding.
Trial registration This clinical trial has been registered in the ClinicalTrials.gov (ID: NCT04895904) approved by the International Committee of Medical Journal Editors (ICMJE).
Peptic ulcer bleeding (PUB) is the major cause of acute non-variceal upper gastrointestinal bleeding (NVUGIB), which remains an urgent medical problem with significant morbidity and mortality [1,2,3,4]. Although the prognosis of patients with PUB has improved with advances in proton pump inhibitors (PPIs) therapy and endoscopic treatment, it remains a potentially life-threatening gastrointestinal emergency. Many patients died every year due to peptic ulcer rebleeding [5,6,7]. Through our clinical observation, we found that most of the rebleeding occurred within three days after endoscopic hemostasis, and the death basically occurred in the people rebleeding within three days (see Additional file 1). Therefore, exploring efficient factors and developing tools for early identifying patients with a high risk of rebleeding after emergent endoscopic hemostasis is an adequate precaution to improve the prognosis of PUB.
Previous studies developed several scoring systems to estimate the prognosis of patients with upper gastrointestinal bleeding (UGIB), including the Glasgow-Blatchford score (GBS), Rockall score (RS), and the AIMS65 score [8,9,10]. However, the RS and the AIMS65 score were mainly developed to estimate the mortality risk of patients [11, 12]. As for the GBS, by summarizing much research, the guideline suggested using GBS ≤ 1 to identify patients at low risk for rebleeding or mortality [4, 5, 13]. Besides, the predictive ability of GBS in patients with high-risk ulcers after endoscopic hemostasis is unsatisfactory [11, 13,14,15]. In addition, these scoring systems’ complexity has limited their application in routine clinical situations.
Prior researches have been carried out to determine predictors of rebleeding in patients with PUB, and some have been incorporated into predictive models. Factors found to be predictors of rebleeding include Forrest classification, use of omeprazole, liver cirrhosis, recent surgery, systolic blood pressure below 100 mmHg, heart rate above 100 bpm, hematemesis, large ulcer size and ulcer site [16,17,18,19,20]. However, study predicting rebleeding within three days after emergency endoscopic hemostasis is rare. Besides, the predictive ability of models mentioned above was unsatisfactory. Thus, the study aimed to establish a novel prediction model displayed as a nomogram to predict rebleeding after emergency endoscopic hemostasis for PUB.
Patients and study design
This was a retrospective study. Patients who underwent emergent endoscopic for peptic ulcers with high-risk stigmata (Forrest Ia–Forrest IIb. Patients with more than one ulcer were classified as the most severe ulcer.) and performed endoscopic hemostasis at the Department of Gastroenterology, the First Affiliated Hospital of Nanchang University between March 2014 and October 2018 were enrolled. The exclusion criteria for this study were as follows: (1) Patients with Forrest IIc and III peptic ulcers, which did not require endoscopic therapy; (2) demographic data was incompleted. Then we collected patients’ information, including demographic information, physical examinations, clinical characteristics, auxiliary examination findings, the GBS, the RS, the AMIS65 score and clinical outcomes. The study protocol was approved by the review boards of The First Affiliated Hospital of Nanchang University center (No: 2021058).
The study outcome was rebleeding within three days of the initially successful therapeutic endoscopy. Rebleeding was defined as recurrent hematemesis or melena with a decrease in hemoglobin by at least 2 g/dL within three days after the initial endoscopic treatment , and all patients with rebleeding were confirmed with a second look of endoscopy. Shock was defined as shock index (pulse rate/systolic blood pressure) > 1.0. We classified patients with only hematemesis or both hematemesis and melena as hematemesis group, and classified patients with melena as melena group.
Endoscopic evaluation and pharmacologic therapy
Experienced endoscopists performed all emergent endoscopies within 12 h of hospital admission. A single-channel endoscope (GIF-XQ290, Olympus Optical Co., Ltd., Japan) was used during the procedure. The endoscopic hemostasis methods included injection therapy, thermal coagulation, mechanical therapy and combined therapy. After successfully hemostasis, the patients would receive high-dose intravenous proton pump inhibitors (80 mg of intravenous injection, then continuous infusion of 8 mg per hour for 72 h.). Then, the patients would receive 40 mg esomeprazole once daily for 30 days.
For normally distributed data, continuous variables were presented as the mean ± standard deviation (SD) and the differences between the rebleeding and no-rebleeding groups were compared using Student’s t-test. For non-normal distributed data, continuous variables were presented as the median and interquartile range (IQR) and the Mann–Whitney rank-sum test was used to analyze the difference between the two groups. Categorical variables were presented as proportions, and the chi-square test or Fisher’s exact test was used accordingly.
The least absolute shrinkage and selection operator (LASSO) method, which is suitable for the regression of high-dimensional data , was used to select the most useful predictive features from the primary data set. And we used the 1 standard error of the minimum criteria (the 1-SE criteria) value as cutoff. Albumin, prothrombin time (PT), shock, haematemesis/melena and Forrest classification were used to construct a nomogram.
Next, calibration curves and the concordance index (C-index) were calculated to evaluate the performance of the model in predicting prognosis. The values of C index of 0.5 and 1.0 respectively represent the random chance and good ability of the model to predict rebleeding. Besides, decision-curve analysis (DCA) and clinical impact curve were also used to determine the clinical net benefit associated with the use of the model . Finally, the model was internally validated via bootstrapping resampling of the construction data set (with 1000 bootstrap samples per model) to obtain optimism corrected discrimination via the C-index for rebleeding . What’s more, we also compared the Area Under Curve (AUC) for the models vs. three clinical risk scores (GBS, RS, and the AIMS65). P < 0.05 was considered to be statistically significant. All the statistical analyses were performed by R statistical software 4.1.0 (www.r-project.org).
A total of 386 patients with PUB who underwent emergency endoscopic hemostasis during the study period at our center were enrolled (Fig. 1). Among these patients, 48 had rebleeding within three days after initial endoscopic hemostasis. The included patients’ median age (IQR) was 56 (43–65) years old, and 313 (81.1%) of these patients were male. The enrolled patients’ baseline characteristics are shown in Tables 1, 2 and 3. Compared to patients who did not rebleed, patients who rebled were more likely to present haematemesis and shock at the time of admission. What is more, patients in the rebleeding group seemed to have a faster heart rate, higher AIMS65 score, higher white cell count, lower platelet, lower albumin, prolonged PT, prolonged activated partial thromboplastin time (APPT) and international normalized ratio (INR) (all P < 0.05). While no differences were observed between the rebleeding group and non-rebleeding group concerning age, sex, alcohol use, smoking, medication use, PU bleeding history, hypertension, diabetes mellitus, systolic blood pressure, diastolic blood pressure, GBS, Rockall score, haemoglobin level on admission, blood urea nitrogen, creatinine, ulcer location, ulcer size ≥ 2 cm and methods of endoscopic hemostasis.
Feature selection based on LASSO method
Rebleeding within three days after emergency endoscopic hemostasis was chosen as the study outcome. In order to reduce the dimensionality and screen out the most representative risk factors for PU rebleeding within three days after endoscopic hemostasis, LASSO regression analysis was performed on the 48 collected variables using the 1-SE criteria value as the cutoff. And as a result, five variables that predict rebleeding within three days after the initially successful therapeutic endoscopy for PU screened out, including albumin, PT, shock, haematemesis/melena, Forrest classification (Fig. 2A, B).
Development and assessment of the nomogram
To predict rebleeding within three days after emergency endoscopic hemostasis for PUB, we conducted multivariable logistic regression analysis using the five predictors selected by the LASSO method. And construct an accurate and stable nomogram (Fig. 3). The equation built for model was LogitP = − 0.148–0.21 * albumin (ALB ≤ 30 = 1) + 1.763 * PT (PT ≥ 14 = 1) + 1.873 * shock (shock = 1) − 1.363 * haematemesis/melena (haematemesis = 1/melena = 2) − 0.281 * Forrest classification (Forrest Ia = 1, Forrest Ib = 2, Forrest Iia = 3, and Forrest IIb = 4). (Table 4).
The calibration curve of the predictive model showed a good fit between the prediction and observation in the primary cohort (Fig. 4). The Hosmer–Lemeshow test yielded a nonsignificant statistic (P = 0.716), showing that the model worked well. The C-index for the predictive model was 0.854, which suggested the model had a good predictive ability.
The DCA demonstrated that this model improved patient outcomes compared with either treat-all or treat-none strategies by helping assess the risk of rebleeding in patients and informing interventions (Fig. 5A). The DCA showed more benefit in the current study with a threshold probability > 0.0% using the nomogram. Besides, the clinical impact curve for the model was also visually indicated that nomogram conferred high clinical net benefit and confirmed the clinical value of this model (Fig. 5B). And the model also performed better than GBS, RS, and AIMS65 (Fig. 6).
Finally, this model was internally validated using bootstrapping resampling of the construction data set (with 1000 bootstrap samples per model). Moreover, the C-index for this nomogram was 0.830, which suggested high accuracy.
PUB is the most common cause of acute non-variceal upper gastrointestinal bleeding. Although the development of endoscopic technology has effectively improved the prognosis of PUB, rebleeding is still one of the common complications . Therefore, it is imperative for clinicians to identify high-risk rebleeding patients after emergent endoscopic therapy, for which clinicians can give more powerful measures. According to our clinical observation, high-risk peptic ulcer rebleeding often occurs within three days after emergent endoscopic hemostasis. However, to our knowledge, there were few studies focused on rebleeding within three days. Anne C Travis et al. developed a model to predict rebleeding for NVUGIB. However, this model's study outcome was rebleeding within 30 days of the initially successful therapeutic endoscopy, and the predictive ability of this model was barely satisfactory (AUC = 0.752) . Zhiyu Dong et al. established a new scoring system to predict poor clinical outcomes for NVUGIB, while this study had a small sample size and the predictive ability is still low . There were also other clinical trials carried out to build models to predict rebleeding for PUB. However, these studies either did not focused on rebleeding within three days after emergent endoscopic hemostasis or had the poor predictive ability [18, 26,27,28]. Hence, in the present study, we developed a novel model to predict rebleeding within three days after emergent endoscopic hemostasis for high-risk peptic ulcer bleeding, and the model showed an excellent discriminatory ability (C-index: 0.854).
In the present study, five predictors identified by the LASSO method were incorporated into the nomogram, and the model was proven to be of excellent performance in internal validation. The Forrest classification is mainly used to stratify ulcer bleeding patients and guide management decisions, including endoscopic and pharmacological therapy. Moreover, many studies proved that the Forrest classification had excellent predictive value for rebleeding peptic ulcers [4, 18, 26]. Thus the Forrest classification is a stronger predictor for rebleeding in PUB. Hypoalbuminemia is a risk factor of mortality in certain diseases, and the correlation between hypoalbuminemia and the prognosis of PUB has been reported. Hsiu-Chi Cheng et al. indicated that hypoalbuminemia in patients with peptic ulcer bleeding could be an alarm indicator of recurrent bleeding . In our study, albumin ≤ 30 g/L scored more in the nomogram than albumin > 30 g/L, which is similar to the previous study. Patients admitted to the hospital with hematemesis and shock heralded more dangerous gastrointestinal bleeding and heralded a worse prognosis . Hence, hematemesis and shock scored more in the nomogram. Prolonged PT indicates deranged coagulation function and was another valuable predictor for rebleeding in patients presenting with PUB . For patients with prolonged PT, the doctor should pay more attention to or take an extra intervention.
There were several advantages in the present study. First, this is the first study to construct a model that incorporated variables from clinical, laboratory, endoscopic features for predicting rebleeding within three days after initial endoscopic therapy for high-risk PUB. Second, informative variables were identified using the LASSO method, which can avoid the statistical defects of overfitting compared with using univariate analysis. Third, our model had an excellent predictive ability. What is more, the model was displayed as a nomogram which was intuitive and easy to use in clinical practice.
However, there were some limitations in the present study. First, the present study was a single-centre retrospective study. Second, our study only had internal validation. However, the model had a good performance in predictive ability (C-index: 0.854), and the internal verification performance was also good (C-index via bootstrapping validation: 0.830). This model needs to be prospectively validated on a distinct group of patients in the future.
In conclusion, we established and internally validated a nomogram to predict rebleeding within three days after emergent endoscopic hemostasis. This nomogram incorporated variables from clinical, laboratory and endoscopic features and can be conveniently used to identify high-risk patients after emergent endoscopic hemostasis, which can help doctors pay more attention to or give extra intervention.
Availability of data and materials
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.
The International Committee of Medical Journal Editors
Peptic ulcer bleeding
Non-variceal upper gastrointestinal bleeding
Proton pump inhibitors
Upper gastrointestinal bleeding
The least absolute shrinkage and selection operator
1 Standard error
Area under curve
International normalized ratio
Prolonged activated partial thromboplastin time
Brullet E, Garcia-Iglesias P, Calvet X, Papo M, Planella M, Pardo A, et al. Endoscopist’s judgment is as useful as risk scores for predicting outcome in peptic ulcer bleeding: a multicenter study. J Clin Med. 2020;9(2):408.
Wilkins T, Wheeler B, Carpenter M. Upper gastrointestinal bleeding in adults: evaluation and management. Am Fam Physician. 2020;101(5):294–300.
Mullady DK, Wang AY, Waschke KA. AGA clinical practice update on endoscopic therapies for non-variceal upper gastrointestinal bleeding: expert review. Gastroenterology. 2020;159(3):1120–8.
Barkun AN, Almadi M, Kuipers EJ, Laine L, Sung J, Tse F, et al. Management of nonvariceal upper gastrointestinal bleeding: guideline recommendations from the international consensus group. Ann Intern Med. 2019;171(11):805–22.
Stanley AJ, Laine L. Management of acute upper gastrointestinal bleeding. BMJ. 2019;364:l536.
Gralnek IM, Dumonceau JM, Kuipers EJ, Lanas A, Sanders DS, Kurien M, et al. Diagnosis and management of non-variceal upper gastrointestinal hemorrhage: European Society of Gastrointestinal Endoscopy (ESGE) Guideline. Endoscopy. 2015;47(10):a1-46.
Lau JY, Barkun A, Fan DM, Kuipers EJ, Yang YS, Chan FK. Challenges in the management of acute peptic ulcer bleeding. Lancet. 2013;381(9882):2033–43.
Rockall TA, Logan RF, Devlin HB, Northfield TC. Risk assessment after acute upper gastrointestinal haemorrhage. Gut. 1996;38(3):316–21.
Blatchford O, Murray WR, Blatchford M. A risk score to predict need for treatment for upper-gastrointestinal haemorrhage. Lancet. 2000;356(9238):1318–21.
Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding. Gastrointest Endosc. 2011;74(6):1215–24.
Robertson M, Majumdar A, Boyapati R, Chung W, Worland T, Terbah R, et al. Risk stratification in acute upper GI bleeding: comparison of the AIMS65 score with the Glasgow-Blatchford and Rockall scoring systems. Gastrointest Endosc. 2016;83(6):1151–60.
Lu X, Zhang X, Chen H. Comparison of the AIMS65 score with the Glasgow-Blatchford and Rockall scoring systems for the prediction of the risk of in-hospital death among patients with upper gastrointestinal bleeding. Rev Esp Enferm Dig. 2020;112(6):467–73.
Stanley AJ, Laine L, Dalton HR, Ngu JH, Schultz M, Abazi R, et al. Comparison of risk scoring systems for patients presenting with upper gastrointestinal bleeding: international multicentre prospective study. BMJ. 2017;356:i6432.
Kim MS, Choi J, Shin WC. AIMS65 scoring system is comparable to Glasgow-Blatchford score or Rockall score for prediction of clinical outcomes for non-variceal upper gastrointestinal bleeding. BMC Gastroenterol. 2019;19(1):136.
Hyett BH, Abougergi MS, Charpentier JP, Kumar NL, Brozovic S, Claggett BL, et al. The AIMS65 score compared with the Glasgow-Blatchford score in predicting outcomes in upper GI bleeding. Gastrointest Endosc. 2013;77(4):551–7.
Chung IK, Kim EJ, Lee MS, Kim HS, Park SH, Lee MH, et al. Endoscopic factors predisposing to rebleeding following endoscopic hemostasis in bleeding peptic ulcers. Endoscopy. 2001;33(11):969–75.
Lin HJ, Tseng GY, Lo WC, Lee FY, Perng CL, Chang FY, et al. Predictive factors for rebleeding in patients with peptic ulcer bleeding after multipolar electrocoagulation: a retrospective analysis. J Clin Gastroenterol. 1998;26(2):113–6.
Guglielmi A, Ruzzenente A, Sandri M, Kind R, Lombardo F, Rodella L, et al. Risk assessment and prediction of rebleeding in bleeding gastroduodenal ulcer. Endoscopy. 2002;34(10):778–86.
Al-Akeely MH, Alam MK, Al-Salamah SM, Abdu MA, Al-Teimi IN, Mohammed AA. Initial factors predicting rebleeding and death in bleeding peptic ulcer disease. Saudi Med J. 2004;25(5):642–7.
Chiu PW, Joeng HK, Choi CL, Kwong KH, Ng EK, Lam SH. Predictors of peptic ulcer rebleeding after scheduled second endoscopy: clinical or endoscopic factors? Endoscopy. 2006;38(7):726–9.
Fernández-Delgado M, Sirsat MS, Cernadas E, Alawadi S, Barro S, Febrero-Bande M. An extensive experimental survey of regression methods. Neural Netw. 2019;111:11–34.
Van Calster B, Wynants L, Verbeek JFM, Verbakel JY, Christodoulou E, Vickers AJ, et al. Reporting and interpreting decision curve analysis: a guide for investigators. Eur Urol. 2018;74(6):796–804.
Henderson AR. The bootstrap: a technique for data-driven statistics. Using computer-intensive analyses to explore experimental data. Clin Chim Acta. 2005;359(1–2):1–26.
Travis AC, Wasan SK, Saltzman JR. Model to predict rebleeding following endoscopic therapy for non-variceal upper gastrointestinal hemorrhage. J Gastroenterol Hepatol. 2008;23(10):1505–10.
Dong Z, Wang J, Zhan T, Zhang H, Yi L, Xu S. A new scoring system to predict poor clinical outcomes in acute nonvariceal upper gastrointestinal bleeding patients with high-risk stigmata. Gastroenterol Res Pract. 2018;2018:5032657.
de Groot NL, van Oijen MG, Kessels K, Hemmink M, Hazen WL, Lelyveld NV, et al. Reassessment of the predictive value of the Forrest classification for peptic ulcer rebleeding and mortality: can classification be simplified? Endoscopy. 2014;46(1):46–52.
Laursen SB. Treatment and prognosis in peptic ulcer bleeding. Dan Med J. 2014;61(1):B4797.
Wong GL, Ma AJ, Deng H, Ching JY, Wong VWS, Tse YK, et al. Machine learning model to predict recurrent ulcer bleeding in patients with history of idiopathic gastroduodenal ulcer bleeding. Aliment Pharmacol Ther. 2019;49(7):912–8.
Cheng HC, Yang EH, Wu CT, Wang WL, Chen PJ, Lin MY, et al. Hypoalbuminemia is a predictor of mortality and rebleeding in peptic ulcer bleeding under proton pump inhibitor use. J Formos Med Assoc. 2018;117(4):316–25.
Lee S, Kim SW, Moon JC, Jang JW, Kim HYI, Park WS, et al. Early angiographic embolization is more effective than delayed angiographic embolization in patients with duodenal ulcer bleeding. J Gastroenterol Hepatol. 2012;27(11):1670–4.
Ethics approval and consent to participate
The study was approved by the ethics committee of The First Affiliated Hospital of Nanchang University (No. 2021058). And all methods were performed in accordance with the relevant guidelines and regulations. Informed consent by verbal was also obtained from all participants approved by the ethics committee of The First Affiliated Hospital of Nanchang University.
Consent for publication
All authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Lai, Y., Xu, Y., Zhu, Z. et al. Development and validation of a model to predict rebleeding within three days after endoscopic hemostasis for high-risk peptic ulcer bleeding. BMC Gastroenterol 22, 64 (2022). https://doi.org/10.1186/s12876-022-02145-9
- Peptic ulcer bleeding
- Emergency endoscopic hemostasis