Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Development and validation of a simple-to-use nomogram to predict liver metastasis in patients with pancreatic neuroendocrine neoplasms: a large cohort study

BMC Gastroenterology volume 21, Article number: 101 (2021) Cite this article

Abstract

Background

Liver metastasis is an important prognostic factor for pancreatic neuroendocrine neoplasms (pNENs), but the relationship between the clinical features of patients with pNEN and liver metastasis remains undetermined. The aim of this study was to establish and validate an easy-to-use nomogram to predict liver-metastasis in patients with pNEN.

Methods

We obtained the clinicopathologic data of 2960 patients with pancreatic neuroendocrine neoplasms from the Surveillance, Epidemiology and End Results (SEER) database between 2010 and 2016. Univariate and multivariate logistic regression were done to screen out independent influencing factors to establish the nomogram. The calibration plots and the area under the receiver operating characteristic curve (AUC) were used to evaluate the performance of nomogram. Decision curve analysis (DCA) was applied to compare the novel model with the conventional predictive methods.

Results

A total of 2960 patients with pancreatic neuroendocrine neoplasms were included in the study. Among these, 1974 patients were assigned to the training group and 986 patients to the validation group. Multivariate logistic regression identified, tumor size, grade, other site metastasis, T stage and N stage as independent risk factors. The calibration plot showed good discriminative ability in the training and validation groups, with C-indexes of 0.850 for the training cohort and 0.846 for the validation cohort. The AUC values were 0.850 (95% CI 0.830–0.869) and 0.839 (95% CI 0.812–0.866), respectively. The nomogram total points (NTP) had the potential to stratify patients into low risk, medium risk and high risk (P < 0.001). Finally, comparing the nomogram with traditional prediction methods, the DCA curve showed that the nomogram had better net benefit.

Conclusions

Our nomogram has a good ability to predict liver metastasis of pancreatic neuroendocrine neoplasms, and it can guide clinicians to provide suitable prevention and treatment measures for patients with medium- and high-risk liver metastasis.

Peer Review reports

Background

Pancreatic neuroendocrine neoplasms (pNENs) are relatively rare, with an estimated annual incidence of approximately 3.65/10,000 people per year [1, 2]. The natural disease progression of pancreatic neuroendocrine tumors can lead to local lymph node, liver, lung, and bone metastases. Among these, liver metastases are the most common. It is reported that more than 60% of patients with pNEN have liver metastases [3]. Studies have found that liver metastasis is an important risk factor for prognosis [4]. The treatment strategy and prognosis of pNEN largely depend on whether there is liver metastasis. Therefore, early diagnosis and treatment of pNEN patients with liver metastases can significantly improve the quality of life and prognosis. Due to the lack of typical clinical manifestations of nonfunctional pNEN in the early stage, 20% to 30% of pNEN patients have liver metastases when diagnosed, which seriously affects their quality of life and long-term survival [5, 6]. Therefore, it is critical that clinicians accurately identify the risk of liver metastases in patients with pNEN for optimal treatment strategies.

The routine examination for excluding liver metastasis is a computed tomography (CT), but it has low sensitivity and specificity for microscopic liver metastasis [7]. Previous studies have shown that liver metastases from neuroendocrine tumors are correlated with a variety of clinicopathological factors, including histological type, primary site, tumor size, lymphatic invasion, and proliferative activity [8, 9]. However, the above studies are limited to some fragmentary risk factors and small sample sizes. It is essential to explore the relationship between clinicopathological factors and liver metastasis based on a large sample database and to develop a prediction model of the risk of liver metastasis in pNEN patients.

In this study, we constructed and validated a simple-to-use nomogram model. With this prediction model, clinicians can accurately identify patients with pNEN at medium and high-risk of liver metastasis patients with pNEN and provide patients with personalized prevention and treatment strategies.

Methods

Study population and data sources

The data were extracted from the Surveillance, Epidemiology, and End Results (SEER) database using SEER*Stat software Version 8.3.6. Data from patients with pNEN diagnosed in 2010–2016 who had complete information including age, sex, race, primary site, grade, marital status, T stage, N stage, tumor size, histology, and metastasis site, were included in the study. Pancreatic neuroendocrine neoplasms were selected on the basis of International Classification of Disease codes (ICD-O-3), including carcinoma (8150), malignant beta-cell tumor (8151), malignant alpha-cell tumor (8152), G-cell tumor (8153), VIPoma (8155), malignant somatostatinoma (8156), carcinoid tumor (8240), carcinoid tumor (8240) and atypical carcinoid tumor (8249). The exclusion criteria were as follows: (1) patients without definitive liver metastasis data; (2) patients with more than one primary cancer; and (3) patients without definitive grade and metastasis site information.

Construction and validation of the nomogram

We randomly assigned two-thirds of our patients to the training group and the rest of them were assigned to the validation group. The chi-square tests was used to compare the baseline characteristics of the two groups. In the training group, liver metastasis risk factors were determined through the univariate logistic regression. Variates with P values less than 0.05 were used in the multivariate logistic regression analysis. Based on the coefficients of the independent risk factors in the multivariate analysis, the prediction model was visualized in the form of the nomogram. To draw this nomogram, we needed to assign a score of 0–100 to each factor. The coefficients of the above multiple logistic regression results were transformed and are shown in the form of graphs. The nomogram's ruler for each indicator was based on the index with the most influence. The greater the influence of the risk factors, the higher the nomogram score [10]. The whole process was done in R 3.6.2 software. The details of building the nomogram and R codes are provided in Additional file 1: Supplement Method 1.

The concordance index (C-index), the receiver operating characteristic curve (ROC), and the area under the curve (AUC) were used to evaluate the predictive accuracy and discrimination of the nomogram. The decision curve (DCA) [11] was used to evaluate the clinical utility of the nomogram, and compare nomogram with conventional predictive risk factors including grade, T stage, and tumor size. The details of DCA curve building and R codes were provided in Additional file 1: Supplement Method 2.

Risk group stratification and statistical analysis

According to the characteristics of each patient's risk factors, a straight line was drawn to the "point" at the top of the model to obtain each factor score. The total score was obtained by summing the scores for all the factors. To further discriminate the risk groups of liver metastasis, the patients were categorized into low-, medium- and high-risk groups based on the nomogram total points (NTP) of every pNEN patients. The optimal two cut-off values for NTP were calculated by X-tile software. The cut-off value was then validated in the validation group. The chi-square test was used to compare all risk groups.

Statistical analysis was performed using SPSS software version 23 and R version 3.6.2 software. For all analyses, P values less than 0.05 were considered statistically significant.

Result

Baseline characteristics of the patients

There were 2960 eligible patients with pNEN who were included in this study. A total of 1974 patients were allocated to the training group and 986 cases were allocated to the validation group. The two groups had no significant difference in baseline characteristics (all P > 0.05) (Table 1). In the entire study group, the median age was 58 years. The majority of the patients were white (n = 2268, 76.6%) and married (n = 1814, 61.3%). The pancreatic tail was the most common site of pNEN tumors (n = 1058, 35.7%). The main pathological grade of neoplasms was G1 (n = 2068, 69.9%), followed by G2 (n = 577, 19.5%). During the whole follow-up, most of the patients were alive (81.9%) and only 535 (18.1%) patients died. There were 419 (21.2%) and 222 (22.5%) pNEN patients with liver metastases in the training group and validation group, respectively. Liver metastasis was found to be correlated with sex, primary site, grade, T stage, N stage, tumor size and other site metastasis in pNEN patients (Table 2).

Table 1 Baseline characteristics of the pNEN patients
Full size table
Table 2 The relationship of pNEN patients with liver metastases and clinicopathological factors in the training group and the validation group
Full size table

Independent risk factors and nomogram construction

Univariate regression analysis was used to screen the risk factors for liver metastasis. The significant risk variables were included in the multivariate regression analysis. The results of multivariate logistic regression analysis showed that grade, T stage, N stage, tumor size, and other site metastasis were independent risk factors for liver metastasis (Table 3). All the above variables were used to establish the nomogram model (Fig. 1). In this model, it was found that grade, T stage and tumor size had the greatest impact on liver metastasis, followed by N stage and other site metastasis. The probability of liver metastasis in each pNEN patient can be computed by adding up the corresponding scores of all the independent risk factors.

Table 3 Univariate and multivariate logistic analyses of liver metastasis in pNEN patients
Full size table
Fig. 1
figure 1

Nomogram for predicting the risk of liver metastasis in patients with pancreatic neuroendocrine neoplasms. Assign points to each risk factor by drawing a line up from the corresponding value to the point line. The total points of all risk factors are summed and are found on the total points line. A line is drawn down to read the corresponding prediction of liver metastasis risk in pNEN patients

Full size image

Nomogram validation and risk classification

The calibration plot showed good agreement in the training and validation group (Fig. 2A, B). The C-index of liver metastasis prediction was 0.850 and 0.846 in the training and validation group, respectively. When the ROC curves were plotted, the training group had an AUC of 0.850 (95% CI 0.830–0.869), which was verified in the validation group (AUC = 0.839, 95% CI 0.812–0.866) (Fig. 2C, D). Decision curve analysis (DCA) was done next (Fig. 3), which is a novel method that can evaluate the clinical practicality of models. The results showed that the nomogram had satisfactory net benefits among most of the threshold probabilities in both groups. Compared with conventional predictive methods, our nomogram was more exact in predicting liver metastasis.

Fig. 2
figure 2

The calibration plots and ROC curves in the training cohort (A and C, respectively) and the validation cohort (B and D, respectively)

Full size image
Fig. 3
figure 3

DCA for the nomogram and the conventional forecasting methods including grade, T-stage, and tumor size in the training (A) and validation groups (B). The x-axis shows the threshold probabilities. The y-axis measures the net benefit, which is calculated by adding the true positives and subtracting the false positives. The horizontal solid black line: assumes no liver metastasis will happen; the solid grey line: assumes all patients will experience tumor liver metastasis. In DCA, the nomogram yielded a superior clinical net benefit compared with the conventional forecasting methods across a range of threshold probabilities

Full size image

The training group was divided into three subgroups based on the two optimal NTP cut-off values. According to the X-tile calculation results, the optimal cut-off values were 105.5 and 156.0 respectively (Fig. 4A). The patients were divided into low-risk (NTP < 105.5, n = 1278 (64.7%)), medium-risk (105.5 ≤ NTP < 156.0, n = 368 (18.6%)) and high-risk subgroups (NTP ≥ 156.0, n = 328 (16.6%)). The same cut-off values were used for grouping in the validation group. Notably, the high-risk pNEN patients were more likely to have liver metastases in both groups (P < 0.05) (Fig. 4B, C).

Fig. 4
figure 4

X-tile software was used to find the best cut-off value (A). The risk-classification performance of the nomogram in the training (B) and validation groups (C)

Full size image

Discussion

Although the natural history of many pancreatic neuroendocrine tumors is characterized by slow progression and inertia, there are still patients with metastasis during the course of the disease, especially liver metastasis. For patients with resectable pNEN with liver metastases, active surgical resection of primary and liver metastases should be the preferred treatment. Previous studies have reported that surgical resection of primary and metastatic lesions could improve quality of life and prolong survival, with a 5-year survival rate of 60–80% [12,13,14,15,16]. However, due to the limited sensitivity of the current imaging modalities, early pNEN patients with liver metastasis have a high rate of missed diagnosis, which makes the patients lose their best chance of radical surgical resection when they are diagnosed. Liver biopsy has a high diagnosis rate, but it increases the risk of distant metastasis and leads to reduced survival time [17]. Therefore, a noninvasive and simple-to-use method is required for predicting the likelihood of liver metastasis in patients with pNEN. In our study, a novel nomogram was developed for predicting the probability of liver metastasis of pNEN based on a large database. The results demonstrated that the nomogram model is significantly discriminative and thus provides an individualized prediction of the probability of liver metastasis.

Our study mainly focussed on the clinical characteristics of pNEN patients with liver metastasis, and demonstrated that grade, T stage, N stage, tumor size, and other site metastasis were independent risk factors for liver metastasis. The G1-2 group had a higher percentage of pNEN patients with liver metastases (70.5%) than the other groups. This result is similar to that of Ruzzenente (81.9%) [18]. In addition, Spolverato [19] found that nonfunctional and moderate-to-poor tumors were more likely to have liver metastases. We speculate the reason that the G1-2 non-functional tumor easily neglected in the early stage due to the lack of obvious clinical symptoms, and the tumor is already in advanced stage when diagnosed. Previous studies have shown that the main cause of liver metastases is vascular invasion [20]. During hematogenous metastasis, the liver is the first filter for tumor cell invasion. In this study, we found that the size and T stage of the primary tumor were closely related to the infiltration of neuroendocrine tumor cells into the liver. The size of the tumor is directly related to the T stage. The larger the primary tumor size, the more aggressive it is towards surrounding organs or blood vessels. This study also confirmed that the larger the tumor and the higher the T stage, the greater the probability of liver metastasis.

Apart from the route of hematogenous metastasis, pancreatic neuroendocrine tumor may also metastasize to distant sites via lymphatic pathways. In our study, LN metastasis was identified as an independent risk factor in predicting liver metastasis. Positive lymph nodes are a common sign before distant metastasis, which has been demonstrated in other tumors [21, 22]. In our study, 47.3% of patients with liver metastases had positive lymph nodes. Therefore, more attention should be paid to the presence of metastasis in the liver and other sites in patients with positive lymph nodes. Besides liver metastasis, there were also other distant site metastases (bone, lung, brain). In this study, more than 72.2% of pNEN patients with other site metastases also had liver metastases. This result reveals that there are probably other metastases when liver metastases are found. This finding is consistent with other studies [23,24,25].

The advice given to the patient and the choice made among treatment options are based on the assessment of the individual's prognosis and risk [26]. Nomograms are graphical representations of statistical prediction models that predict the probability of an event occurring [27]. Thus, the variables contained in the nomogram should be easy to obtain and measure. In this study, we developed a nomogram to predict live metastasis in patients with pNEN. Our nomogram model has been shown to have good discernment with high C-indexes and AUCs, in both groups. Finally, DCA curves were generated to show that the nomogram could be used to obtain a better net benefit within the derived probabilities than traditional prediction methods [26].

There are some limitations to this study. The major limitation of our study is the lack of important variables, such as surgical margin, Ki-67 and other molecular biomarkers. The Ki-67 index and surgical margin play an important role in the prognosis of pNEN [28]. Unfortunately, the absence of Ki-67 and surgical margins in the SEER database made it impossible to assess its role in predicting liver metastasis of pNEN. Second, our nomogram has been verified to have excellent prediction capabilities, but further external validation based on a large multicenter data cohort is still required. Finally, since the SEER database is a retrospective database, selection bias cannot be completely avoided. Therefore, bootstrapping with 1000 resamples was performed in this study to minimize bias.

Conclusion

In conclusion, we successfully created and validated a simple-to-use nomogram for predicting the probability of liver metastasis in pNEN patients. This model has good predictive power and it is easy for the clinician to use. By assessing the risk of liver metastasis, clinicians could realize individualized treatment and take necessary preventive measures to reduce the risks borne by patients and improve their quality of life and prognosis.

Availability of data and materials

The datasets analyzed during the current study are available in the SEER repository (https://seer.cancer.gov/).

Abbreviations

pNEN:

Pancreatic neuroendocrine neoplasms

NTP:

Nomogram total points

ROC:

Receiver operating characteristic

ICD-O-3:

International classification of diseases for oncology, third edition

SEER:

Surveillance, epidemiology, and end results

LN:

Lymph node

AUC:

The area under the curve

DCA:

The decision curve

References

  1. Frilling A, Modlin IM, Kidd M, Russell C, Breitenstein S, Salem R, Kwekkeboom D, Lau WY, Klersy C, Vilgrain V, et al. Recommendations for management of patients with neuroendocrine liver metastases. Lancet Oncol. 2014;15(1):e8–21.

    Article  Google Scholar 

  2. Sandvik OM, Soreide K, Gudlaugsson E, Kvaloy JT, Soreide JA. Epidemiology and classification of gastroenteropancreatic neuroendocrine neoplasms using current coding criteria. Br J Surg. 2016;103(3):226–32.

    Article  CAS  Google Scholar 

  3. Bertani E, Fazio N, Radice D, Zardini C, Grana C, Bodei L, Funicelli L, Ferrari C, Spada F, Partelli S, et al. Resection of the primary tumor followed by peptide receptor radionuclide therapy as upfront strategy for the treatment of g1–g2 pancreatic neuroendocrine tumors with unresectable liver metastases. Ann Surg Oncol. 2016;23(Suppl 5):981–9.

    Article  Google Scholar 

  4. Yao KA, Talamonti MS, Nemcek A, Angelos P, Chrisman H, Skarda J, Benson AB, Rao S, Joehl RJ. Indications and results of liver resection and hepatic chemoembolization for metastatic gastrointestinal neuroendocrine tumors. Surgery. 2001;130(4):677–85.

    Article  CAS  Google Scholar 

  5. Ito T, Igarashi H, Nakamura K, Sasano H, Okusaka T, Takano K, Komoto I, Tanaka M, Imamura M, Jensen RT, et al. Epidemiological trends of pancreatic and gastrointestinal neuroendocrine tumors in Japan: a nationwide survey analysis. J Gastroenterol. 2015;50(1):58–64.

    Article  Google Scholar 

  6. Wang YH, Lin Y, Xue L, Wang JH, Chen MH, Chen J. Relationship between clinical characteristics and survival of gastroenteropancreatic neuroendocrine neoplasms: a single-institution analysis (1995–2012) in South China. BMC Endocr Disord. 2012;12:30.

    Article  Google Scholar 

  7. Rockall AG, Reznek RH. Imaging of neuroendocrine tumours (CT/MR/US). Best Pract Res Clin Endocrinol Metab. 2007;21(1):43–68.

    Article  CAS  Google Scholar 

  8. Maithel SK, Fong Y. Hepatic ablation for neuroendocrine tumor metastases. J Surg Oncol. 2009;100(8):635–8.

    Article  Google Scholar 

  9. Frilling A, Sotiropoulos GC, Li J, Kornasiewicz O, Plockinger U. Multimodal management of neuroendocrine liver metastases. HPB (Oxford). 2010;12(6):361–79.

    Article  Google Scholar 

  10. Iasonos A, Schrag D, Raj GV, et al. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. 2008;26:1364–70.

    Article  Google Scholar 

  11. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74.

    Article  Google Scholar 

  12. Chen H, Hardacre JM, Uzar A, Cameron JL, Choti MA. Isolated liver metastases from neuroendocrine tumors: does resection prolong survival? J Am Coll Surg. 1998;187(1):88–93.

    Article  CAS  Google Scholar 

  13. Chamberlain RS, Canes D, Brown KT, Saltz L, Jarnagin W, Fong Y, Blumgart LH. Hepatic neuroendocrine metastases: does intervention alter outcomes? J Am Coll Surg. 2000;190(4):432–45.

    Article  CAS  Google Scholar 

  14. Elias D, Lasser P, Ducreux M, Duvillard P, Ouellet JF, Dromain C, Schlumberger M, Pocard M, Boige V, Miquel C, et al. Liver resection (and associated extrahepatic resections) for metastatic well-differentiated endocrine tumors: a 15-year single center prospective study. Surgery. 2003;133(4):375–82.

    Article  Google Scholar 

  15. Sarmiento JM, Heywood G, Rubin J, Ilstrup DM, Nagorney DM, Que FG. Surgical treatment of neuroendocrine metastases to the liver: a plea for resection to increase survival. J Am Coll Surg. 2003;197(1):29–37.

    Article  Google Scholar 

  16. Mayo SC, de Jong MC, Bloomston M, Pulitano C, Clary BM, Reddy SK, Clark GT, Celinski SA, Kooby DA, Staley CA, et al. Surgery versus intra-arterial therapy for neuroendocrine liver metastasis: a multicenter international analysis. Ann Surg Oncol. 2011;18(13):3657–65.

    Article  Google Scholar 

  17. Zhang L, Xiang ZL, Zeng ZC, Fan J, Tang ZY, Zhao XM. A microRNA-based prediction model for lymph node metastasis in hepatocellular carcinoma. Oncotarget. 2016;7(3):3587–98.

    Article  Google Scholar 

  18. Ruzzenente A, Bagante F, Bertuzzo F, Aldrighetti L, Ercolani G, Giuliante F, Ferrero A, Torzilli G, Grazi GL, Ratti F, et al. A novel nomogram to predict the prognosis of patients undergoing liver resection for neuroendocrine liver metastasis: an analysis of the italian neuroendocrine liver metastasis database. J Gastrointest Surg. 2017;21(1):41–8.

    Article  Google Scholar 

  19. Spolverato G, Bagante F, Aldrighetti L, Poultsides G, Bauer TW, Field RC, Marques HP, Weiss M, Maithel SK, Pawlik TM. Neuroendocrine liver metastasis: prognostic implications of primary tumor site on patients undergoing curative intent liver surgery. J Gastrointest Surg. 2017;21(12):2039–47.

    Article  Google Scholar 

  20. Dong S, et al. Risk factors of liver metastasis from advanced pancreatic adenocarcinoma: a large multicenter cohort study. World J Surg Oncol. 2017;15(1):120.

    Article  Google Scholar 

  21. Wu SG, Zhang WW, Sun JY, Li FY, Lin Q, He ZY. Patterns of distant metastasis between histological types in esophageal cancer. Front Oncol. 2018;8:302.

    Article  Google Scholar 

  22. Gaitanidis A, Alevizakos M, Tsaroucha A, Tsalikidis C, Pitiakoudis M. Predictive nomograms for synchronous distant metastasis in rectal cancer. J Gastrointest Surg. 2018;22(7):1268–76.

    Article  Google Scholar 

  23. Oweira H, Petrausch U, Helbling D, Schmidt J, Mannhart M, Mehrabi A, Schob O, Giryes A, Decker M, Abdel-Rahman O. Prognostic value of site-specific metastases in pancreatic adenocarcinoma: a Surveillance Epidemiology and End Results database analysis. World J Gastroenterol. 2017;23(10):1872–80.

    Article  Google Scholar 

  24. Shiroyama T, Suzuki H, Tamiya M, Tamiya A, Tanaka A, Okamoto N, Nakahama K, Taniguchi Y, Isa SI, Inoue T, et al. Clinical characteristics of liver metastasis in nivolumab-treated patients with non-small cell lung cancer. Anticancer Res. 2018;38(8):4723–9.

    Article  CAS  Google Scholar 

  25. He C, Zhong L, Zhang Y, Cai Z, Lin X. Development and validation of a nomogram to predict liver metastasis in patients with pancreatic ductal adenocarcinoma: a large cohort study. Cancer Manag Res. 2019;11:3981–91.

    Article  Google Scholar 

  26. Balachandran VP, Gonen M, Smith JJ, DeMatteo RP. Nomograms in oncology: more than meets the eye. Lancet Oncol. 2015;16(4):e173–80.

    Article  Google Scholar 

  27. Bochner BH, Kattan MW, Vora KC. Postoperative nomogram predicting risk of recurrence after radical cystectomy for bladder cancer. J Clin Oncol. 2006;24(24):3967–72.

    Article  Google Scholar 

  28. Watzka FM, Meyer F, Staubitz JI, Fottner C, Schad A, Lang H, Musholt TJ. Prognostic assessment of non-functioning neuroendocrine pancreatic neoplasms as a basis for risk-adapted resection strategies. World J Surg. 2020;44(2):594–603.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors express their thanks to Xiaoting Wu at The First Affiliated Hospital of Fujian Medical University in the contribution of layout of pictures and forms.

Funding

This work was supported bythe National Natural Science Foundation of China [No. 82072074]. Startup Fund for scientific research, Fujian Medical University [No. 2019QH2017]. National Natural Science Foundation of China [No.82001895]; Joint Funds of Scientific and Technological Innovation Program of Fujian Province [No. 2018Y9039].

Author information

Author notes

  1. Maoen Pan and Yuanyuan Yang have contributed equally to this work

Authors and Affiliations

  1. Department of General Surgery, Fujian Medical University Union Hospital, No.29, Xinquan Road, Fuzhou, 350001, China

    Maoen Pan, Yuanyuan Yang, Tianhong Teng, Fengchun Lu, Yanchan Chen & Heguang Huang

Authors
  1. Maoen Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tianhong Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fengchun Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanchan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Heguang Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PME and HHG designed the study. YYY and TTH contributed to get access to data and statistical analysis. PME and YYY drafted the manuscript. HHG, LFC, CYC critically revised the paper. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Heguang Huang.

Ethics declarations

Ethics approval and consent to participate

As the data used was from SEER dataset (public), Ethics approval and consent to participate could be checked in SEER.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

The detail methods of nomogram and DCA curve construction.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pan, M., Yang, Y., Teng, T. et al. Development and validation of a simple-to-use nomogram to predict liver metastasis in patients with pancreatic neuroendocrine neoplasms: a large cohort study. BMC Gastroenterol 21, 101 (2021). https://doi.org/10.1186/s12876-021-01685-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12876-021-01685-w

Keywords

BMC Gastroenterology

ISSN: 1471-230X

Contact us