Development and validation of a simple-to-use nomogram to predict liver metastasis in patients with pancreatic neuroendocrine neoplasms: a large cohort study

Background Liver metastasis is an important prognostic factor for pancreatic neuroendocrine neoplasms (pNENs), but the relationship between the clinical features of patients with pNEN and liver metastasis remains undetermined. The aim of this study was to establish and validate an easy-to-use nomogram to predict liver-metastasis in patients with pNEN. Methods We obtained the clinicopathologic data of 2960 patients with pancreatic neuroendocrine neoplasms from the Surveillance, Epidemiology and End Results (SEER) database between 2010 and 2016. Univariate and multivariate logistic regression were done to screen out independent influencing factors to establish the nomogram. The calibration plots and the area under the receiver operating characteristic curve (AUC) were used to evaluate the performance of nomogram. Decision curve analysis (DCA) was applied to compare the novel model with the conventional predictive methods. Results A total of 2960 patients with pancreatic neuroendocrine neoplasms were included in the study. Among these, 1974 patients were assigned to the training group and 986 patients to the validation group. Multivariate logistic regression identified, tumor size, grade, other site metastasis, T stage and N stage as independent risk factors. The calibration plot showed good discriminative ability in the training and validation groups, with C-indexes of 0.850 for the training cohort and 0.846 for the validation cohort. The AUC values were 0.850 (95% CI 0.830–0.869) and 0.839 (95% CI 0.812–0.866), respectively. The nomogram total points (NTP) had the potential to stratify patients into low risk, medium risk and high risk (P < 0.001). Finally, comparing the nomogram with traditional prediction methods, the DCA curve showed that the nomogram had better net benefit. Conclusions Our nomogram has a good ability to predict liver metastasis of pancreatic neuroendocrine neoplasms, and it can guide clinicians to provide suitable prevention and treatment measures for patients with medium- and high-risk liver metastasis. Supplementary Information The online version contains supplementary material available at 10.1186/s12876-021-01685-w.

Background Pancreatic neuroendocrine neoplasms (pNENs) are relatively rare, with an estimated annual incidence of approximately 3.65/10,000 people per year [1,2]. The natural disease progression of pancreatic neuroendocrine tumors can lead to local lymph node, liver, lung, and bone metastases. Among these, liver metastases are the most common. It is reported that more than 60% of patients with pNEN have liver metastases [3]. Studies have found that liver metastasis is an important risk factor for prognosis [4]. The treatment strategy and prognosis of pNEN largely depend on whether there is liver metastasis. Therefore, early diagnosis and treatment of pNEN patients with liver metastases can significantly improve the quality of life and prognosis. Due to the lack of typical clinical manifestations of nonfunctional pNEN in the early stage, 20% to 30% of pNEN patients have liver metastases when diagnosed, which seriously affects their quality of life and long-term survival [5,6]. Therefore, it is critical that clinicians accurately identify the risk of liver metastases in patients with pNEN for optimal treatment strategies.
The routine examination for excluding liver metastasis is a computed tomography (CT), but it has low sensitivity and specificity for microscopic liver metastasis [7]. Previous studies have shown that liver metastases from neuroendocrine tumors are correlated with a variety of clinicopathological factors, including histological type, primary site, tumor size, lymphatic invasion, and proliferative activity [8,9]. However, the above studies are limited to some fragmentary risk factors and small sample sizes. It is essential to explore the relationship between clinicopathological factors and liver metastasis based on a large sample database and to develop a prediction model of the risk of liver metastasis in pNEN patients.
In this study, we constructed and validated a simpleto-use nomogram model. With this prediction model, clinicians can accurately identify patients with pNEN at medium and high-risk of liver metastasis patients with pNEN and provide patients with personalized prevention and treatment strategies.

Study population and data sources
The data were extracted from the Surveillance, Epidemiology, and End Results (SEER) database using SEER*Stat software Version 8.3.6. Data from patients with pNEN diagnosed in 2010-2016 who had complete information including age, sex, race, primary site, grade, marital status, T stage, N stage, tumor size, histology, and metastasis site, were included in the study. Pancreatic neuroendocrine neoplasms were selected on the basis of  (8240) and atypical carcinoid tumor (8249). The exclusion criteria were as follows: (1) patients without definitive liver metastasis data; (2) patients with more than one primary cancer; and (3) patients without definitive grade and metastasis site information.

Construction and validation of the nomogram
We randomly assigned two-thirds of our patients to the training group and the rest of them were assigned to the validation group. The chi-square tests was used to compare the baseline characteristics of the two groups. In the training group, liver metastasis risk factors were determined through the univariate logistic regression. Variates with P values less than 0.05 were used in the multivariate logistic regression analysis. Based on the coefficients of the independent risk factors in the multivariate analysis, the prediction model was visualized in the form of the nomogram. To draw this nomogram, we needed to assign a score of 0-100 to each factor. The coefficients of the above multiple logistic regression results were transformed and are shown in the form of graphs. The nomogram's ruler for each indicator was based on the index with the most influence. The greater the influence of the risk factors, the higher the nomogram score [10]. The whole process was done in R 3.6.2 software. The details of building the nomogram and R codes are provided in Additional file 1: Supplement Method 1.
The concordance index (C-index), the receiver operating characteristic curve (ROC), and the area under the curve (AUC) were used to evaluate the predictive accuracy and discrimination of the nomogram. The decision curve (DCA) [11] was used to evaluate the clinical utility of the nomogram, and compare nomogram with conventional predictive risk factors including grade, T stage, and tumor size. The details of DCA curve building and R codes were provided in Additional file 1: Supplement Method 2.

Risk group stratification and statistical analysis
According to the characteristics of each patient's risk factors, a straight line was drawn to the "point" at the top of the model to obtain each factor score. The total score was obtained by summing the scores for all the factors. To further discriminate the risk groups of liver metastasis, the patients were categorized into low-, medium-and high-risk groups based on the nomogram total points (NTP) of every pNEN patients. The optimal two cut-off values for NTP were calculated by X-tile software. The cut-off value was then validated in the validation group. The chi-square test was used to compare all risk groups.
Statistical analysis was performed using SPSS software version 23 and R version 3.6.2 software. For all analyses, P values less than 0.05 were considered statistically significant.

Baseline characteristics of the patients
There were 2960 eligible patients with pNEN who were included in this study. A total of 1974 patients were allocated to the training group and 986 cases were allocated to the validation group. The two groups had no significant difference in baseline characteristics (all P > 0.05) ( Table 1). In the entire study group, the median age was 58 years. The majority of the patients were white (n = 2268, 76.6%) and married (n = 1814, 61.3%). The pancreatic tail was the most common site of pNEN tumors (n = 1058, 35.7%). The main pathological grade of neoplasms was G1 (n = 2068, 69.9%), followed by G2 (n = 577, 19.5%). During the whole follow-up, most of the patients were alive (81.9%) and only 535 (18.1%) patients died. There were 419 (21.2%) and 222 (22.5%) pNEN patients with liver metastases in the training group and validation group, respectively. Liver metastasis was found to be correlated with sex, primary site, grade, T stage, N stage, tumor size and other site metastasis in pNEN patients ( Table 2).

Independent risk factors and nomogram construction
Univariate regression analysis was used to screen the risk factors for liver metastasis. The significant risk variables were included in the multivariate regression analysis. The results of multivariate logistic regression analysis showed that grade, T stage, N stage, tumor size, and other site metastasis were independent risk factors for liver metastasis (Table 3). All the above variables were used to establish the nomogram model ( Fig. 1). In this model, it was found that grade, T stage and tumor size had the greatest impact on liver metastasis, followed by N stage and other site metastasis. The probability of liver metastasis in each pNEN patient can be computed by adding up the corresponding scores of all the independent risk factors.

Nomogram validation and risk classification
The calibration plot showed good agreement in the training and validation group ( Fig. 2A, B). The C-index of liver metastasis prediction was 0.850 and 0.846 in the training and validation group, respectively. When the ROC curves were plotted, the training group had an AUC of 0.850 (95% CI 0.830-0.869), which was verified in the validation group (AUC = 0.839, 95% CI 0.812-0.866) (Fig. 2C,  D). Decision curve analysis (DCA) was done next (Fig. 3), which is a novel method that can evaluate the clinical practicality of models. The results showed that the nomogram had satisfactory net benefits among most of the threshold probabilities in both groups. Compared with conventional predictive methods, our nomogram was more exact in predicting liver metastasis.
The training group was divided into three subgroups based on the two optimal NTP cut-off values. According to the X-tile calculation results, the optimal cutoff values were 105.5 and 156.0 respectively (Fig. 4A). The patients were divided into low-risk (NTP < 105.5, n = 1278 (64.7%)), medium-risk (105.5 ≤ NTP < 156.0, n = 368 (18.6%)) and high-risk subgroups (NTP ≥ 156.0, n = 328 (16.6%)). The same cut-off values were used for grouping in the validation group. Notably, the high-risk pNEN patients were more likely to have liver metastases in both groups (P < 0.05) (Fig. 4B, C).

Discussion
Although the natural history of many pancreatic neuroendocrine tumors is characterized by slow progression and inertia, there are still patients with metastasis during the course of the disease, especially liver metastasis. For patients with resectable pNEN with liver metastases, active surgical resection of primary and liver metastases should be the preferred treatment. Previous studies have reported that surgical resection of primary and metastatic lesions could improve quality of life and prolong survival, with a 5-year survival rate of 60-80% [12][13][14][15][16]. However, due to the limited sensitivity of the current imaging modalities, early pNEN patients with liver metastasis have a high rate of missed diagnosis, which makes the patients lose their best chance of radical surgical resection when they are diagnosed. Liver biopsy has a high diagnosis rate, but it increases the risk of distant metastasis and leads to reduced survival time [17]. Therefore, a noninvasive and simple-to-use method is required for predicting the likelihood of liver metastasis in patients with pNEN. In our study, a novel nomogram was developed for predicting the probability of liver metastasis of pNEN based on a large database. The results demonstrated that the nomogram model is significantly discriminative and thus provides an individualized prediction of the probability of liver metastasis.
Our study mainly focussed on the clinical characteristics of pNEN patients with liver metastasis, and demonstrated that grade, T stage, N stage, tumor size, and other site metastasis were independent risk factors for liver metastasis. The G1-2 group had a higher percentage of pNEN patients with liver metastases (70.5%) than the other groups. This result is similar to that of Ruzzenente (81.9%) [18]. In addition, Spolverato [19] found that nonfunctional and moderate-to-poor tumors  were more likely to have liver metastases. We speculate the reason that the G1-2 non-functional tumor easily neglected in the early stage due to the lack of obvious clinical symptoms, and the tumor is already in advanced stage when diagnosed. Previous studies have shown that the main cause of liver metastases is vascular invasion [20]. During hematogenous metastasis, the liver is the first filter for tumor cell invasion. In this study, we found that the size and T stage of the primary tumor were closely related to the infiltration of neuroendocrine tumor cells into the liver. The size of the tumor is directly related to the T stage. The larger the primary  . The x-axis shows the threshold probabilities. The y-axis measures the net benefit, which is calculated by adding the true positives and subtracting the false positives. The horizontal solid black line: assumes no liver metastasis will happen; the solid grey line: assumes all patients will experience tumor liver metastasis. In DCA, the nomogram yielded a superior clinical net benefit compared with the conventional forecasting methods across a range of threshold probabilities tumor size, the more aggressive it is towards surrounding organs or blood vessels. This study also confirmed that the larger the tumor and the higher the T stage, the greater the probability of liver metastasis. Apart from the route of hematogenous metastasis, pancreatic neuroendocrine tumor may also metastasize to distant sites via lymphatic pathways. In our study, LN metastasis was identified as an independent risk factor in predicting liver metastasis. Positive lymph nodes are a common sign before distant metastasis, which has been demonstrated in other tumors [21,22]. In our study, 47.3% of patients with liver metastases had positive lymph nodes. Therefore, more attention should be paid to the presence of metastasis in the liver and other sites in patients with positive lymph nodes. Besides liver metastasis, there were also other distant site metastases (bone, lung, brain). In this study, more than 72.2% of pNEN patients with other site metastases also had liver metastases. This result reveals that there are probably other metastases when liver metastases are found. This finding is consistent with other studies [23][24][25].
The advice given to the patient and the choice made among treatment options are based on the assessment of the individual's prognosis and risk [26]. Nomograms are graphical representations of statistical prediction models that predict the probability of an event occurring [27]. Thus, the variables contained in the nomogram should be easy to obtain and measure. In this study, we developed a nomogram to predict live metastasis in patients with pNEN. Our nomogram model has been shown to have good discernment with high C-indexes and AUCs, in both groups. Finally, DCA curves were generated to show that the nomogram could be used to obtain a better net benefit within the derived probabilities than traditional prediction methods [26].
There are some limitations to this study. The major limitation of our study is the lack of important variables, such as surgical margin, Ki-67 and other molecular biomarkers. The Ki-67 index and surgical margin play an important role in the prognosis of pNEN [28]. Unfortunately, the absence of Ki-67 and surgical margins in the SEER database made it impossible to assess its role in predicting liver metastasis of pNEN. Second, our nomogram has been verified to have excellent prediction capabilities, but further external validation based on a large multicenter data cohort is still required. Finally, since the SEER database is a retrospective database, selection bias cannot be completely avoided. Therefore, bootstrapping with 1000 resamples was performed in this study to minimize bias.

Conclusion
In conclusion, we successfully created and validated a simple-to-use nomogram for predicting the probability of liver metastasis in pNEN patients. This model has good predictive power and it is easy for the clinician to use. By assessing the risk of liver metastasis, clinicians could realize individualized treatment and take necessary preventive measures to reduce the risks borne by patients and improve their quality of life and prognosis.