Introduction
Colorectal cancer is the second leading cause of cancer-related death worldwide [1]. Nearly half of these patients will develop liver metastases (colorectal liver metastases (CRLM)) with <10% surviving past 3 years [2,3]. Precise assessment of CRLM is vital for treatment decision-making including thermal percutaneous ablation, radiotherapy, surgical resection, arterial embolization, and systemic chemotherapy [4,5].
Abdominal contrast-enhanced CT (CECT) scans are central to this evaluation, providing detailed imaging for tumor burden assessment and staging [3,4].
Clinical challenges.
The appearance of tumors in CT scans is affected by variation in (1) image acquisition and reconstruction protocol (contrast agent, contrast timing, resolution, reconstruction kernels, etc), (2) tumor biology (mucinous content, fibrosis, necrosis, treatment effects), and (3) underlying liver disease (steatosis, steatohepatitis, sinuisoidal obstruction syndrome) [6]. Manual segmentation of liver tumors is labor-intensive, poorly reproducible, subjective, and requires specialized expertise, making it especially challenging in cases with complex metastatic patterns [6]. Response evaluation criteria in solid tumors (RECIST) require the measurement of target lesions, but for each organ only up to two lesions are measured in their maximal axial diameters [7], which underrepresents the total tumor burden.
In CRLM in particular, tumors are small and distributed throughout the liver, and are difficult to trend over time in treated patients. Fully automated liver tumor segmentation in CT images would bridge clinical gaps but remains a challenge [8].
Technical challenges.
Recent advances in deep learning have revolutionized medical image segmentation [9–11] and detection [12]. Frameworks like nnU-Net achieve state-of-the-art performance by automatically optimizing architectures and hyperparameters for specific datasets [13]. Incorporating iterative annotation workflows, where human-refined contours are fed to the segmentation engine, has proven particularly effective for abdominal CT segmentation tasks [14].
Despite these advances, clinical deployment is hindered by variations in imaging protocols (e.g., slice thickness, contrast timing) and scanner heterogeneity [15,16]. Even top-performing models fail catastrophically when encountering out-of-distribution data, highlighting the need for reliable failure detection [17]. Existing methods for segmentation reliability emphasize uncertainty estimation [18–21], out-of-distribution detection [22], and quality regression [23], with the Area Under the Risk Coverage Curve (AURC) emerging as a critical evaluation metric [24].
However, these approaches often neglect key factors such as slice thickness and tumor size, despite their significant impact on segmentation performance. Subcentimeter tumors (<1 cm), for example, are particularly prone to segmentation errors due to partial volume effects and boundary ambiguity. Therefore, for eventual clinical translation, a robust liver tumor segmentation method should be able to handle tumors of all sizes, including small ones found in patients with CRLM, while also providing reliable failure detection.
To address these challenges, we developed the liver cancer segmentator (LCS), a deep-learning system designed specifically for CRLM parenchyma and tumor segmentation in portal venous CT scans. In particular, small CRLMs are difficult to segment due to reduced contrast relative to the surrounding parenchyma, irregular lesion morphology, and heightened susceptibility to partial-volume effects [9].
Complementary advances in segmentation and clinical AI [25,26]—including a real-time segmentation framework [27], an XGBoost risk model for early tumor detection [28], attention-enhanced U-Net methods for lung nodule analysis [29], a hybrid FrCN–U-Net architecture for skin lesion segmentation [30], and volumetric/replicator networks for MRI brain tumor analysis [31]—offer additional methodological perspectives for model design and evaluation. Additionally, broader mathematical and reliability-oriented studies on graph-theoretic structures, fractional integro-differential formulations, reliability-based design optimization using differential–algebraic equations, and fuzzy-logic reliability allocation [32–38] provide conceptual grounding for robustness and risk quantification that can inform confidence-scoring strategies.
While previous benchmarks and nnU-Net based pipelines have established strong baselines for liver and tumor segmentation (e.g., LiTS [8]), they commonly evaluate on mixed hepatic disease cohorts and do not explicitly incorporate acquisition metadata into reliability estimates for clinical deployment. Recent surveys on uncertainty estimation [39] highlight that ensemble and Bayesian approaches improve failure detection but often ignore scan-level acquisition factors (slice thickness, reconstruction kernel) and lesion-level descriptors (tumor volume) that strongly influence segmentation reliability. Related liver tumor classification efforts [40,41] further demonstrate the growing interest in machine learning methods for hepatic disease analysis.
Our work addresses these gaps by (i) training and evaluating on a multi-institutional CRLM-specific cohort that includes small, diffuse metastases typical of colorectal disease, and (ii) introducing a metadata-enhanced confidence score that combines ensemble pairwise agreement with slice-thickness and per-lesion volume weighting to prioritize cases for expert review. These additions reduce the AURC substantially on our held-out test set while maintaining high segmentation accuracy.
Segmentation reliability is crucial for ensuring clinician confidence. To address the uncertainties inherent in segmentation models, we developed a confidence scoring methodology. This method quantifies the reliability of each prediction and identifies cases requiring further review.
By incorporating both model agreement [19] and domain-specific factors, our framework enhances the interpretability and robustness of CRLM segmentations, providing clinicians with a measure of prediction certainty. To assess the reliability of our confidence estimates, we evaluated the AURC [24], which quantifies the trade-off between risk and coverage, with lower values indicating better confidence calibration.
The proposed framework — referred to as the LCS — integrates segmentation and metadata-aware confidence scoring. The key contributions of our work are: The LCS, a deep learning model trained on 446 multi-institutional CT scans, achieving a Dice similarity coefficient (DSC) of 0.9707 (95% CI: 0.9663–0.9751) for liver parenchyma and 0.7695 (95% CI: 0.7166–0.8224) for tumors – comparable to radiologist performance.A metadata-enhanced confidence scoring framework incorporating tumor size thresholds (<1 cm) and slice thickness (>5 mm) into reliability estimates, reducing AURC from 16.7 to 10.3 – showing a clinician where the models fail.The public release of the segmentation model, confidence scoring framework, and source code to facilitate clinical translation.1
Materials and methods
The overall system, termed the LCS, consists of preprocessing, segmentation, iterative refinement, and metadata-guided confidence scoring. We developed a confidence-aware deep learning pipeline (Fig. 1) for automated CRLM segmentation in CT scans.
The workflow consists of preprocessing, model training, iterative refinement of segmentation outputs, and confidence scoring to ensure robustness and interpretability. Our approach incorporates data from CRLM patients across different disease stages. A confidence-based quality control step evaluates segmentation reliability: predictions exceeding a predefined confidence threshold (τ) are accepted, while lower-confidence segmentations are flagged for radiologist review.
This ensures that unreliable predictions are identified and mitigated before clinical application.
Datasets
This multi-institutional study analyzed 446 portal venous phase CT examinations from patients with CRLM. Scans were acquired from Memorial Sloan Kettering Cancer Center (MSK) and the University of Texas MD Anderson Cancer Center (MDA). The data set included publicly available ‘‘TCIA’’ data [42], retrospectively collected cohorts of patients at different cancer stages, and a prospectively collected cohort of patients at all stages.
By incorporating patients at different stages and images acquired under a variety of conditions, we aim to improve diversity of imaging data and generalizability of resulting models.
The dataset was composed of four cohorts: Resection cohort: This cohort includes patients prior to liver resection of CRLM queried from liver resection databases. Images were acquired prior to hepatic resection. Eligibility for liver resection is determined based on size, number, location and distribution of tumors, as well as maintaining sufficient future liver remnant [3].
In general, liver tumors are confined to a single lobe.Chemotherapy cohort: This cohort includes patients who received chemotherapy treatment queried from medical oncology databases. The images were acquired before and during treatment, so treatment effects are present in the images, such as decreases in enhancement (becoming darker on a contrast CT scan) and development of calcifications (bright dots on the image).All stages cohort: Patients in this cohort were prospectively enrolled in an imaging trial at the time of routine imaging. Images of patients at all stages and all types of treatment were therefore included, representing the full variability of patients seen at both institutions.TCIA cohort: This is a public dataset collected from MSK.
The cohort consists of patients undergoing resection, meeting the same criteria as those in the ‘‘Resection’’ cohort above.
Fig. 2(a) shows the challenges of CRLM segmentation in different stages and treatments of cancer. Fig.
2(b) shows the distribution of tumor counts across the patients in our study demonstrating the wide variability of the imaging data use for training and evaluation in our study.
In the combined dataset (n = 3959 tumors) (Fig. 3(f)), the median tumor volume was 0.579 cm3 (equivalent sphere diameter ~10.4 mm) with an interquartile range (IQR) of 3.1 cm3 (equivalent sphere diameter ~18.2 mm). All cohorts demonstrated right-skewed distributions, with the MDA cohort showing the most pronounced skewness and kurtosis, indicative of a high concentration of smaller tumor volumes.
These results underscore the importance of interpreting tumor volume distributions in a cohort-specific context, accounting for differences in imaging protocols.
CT series were randomly sampled across institutions to minimize selection bias. The dataset was further stratified by institution and divided into an 80% training/validation set (355 examinations) and a 20% independent test set (91 examinations). All ground truth tumor and liver segmentations were manually annotated using 3D Slicer software [43], following standardized protocols established by a panel of board-certified abdominal radiologists.
Final segmentations were reviewed and approved by a senior radiologist with over 10 years of post-fellowship experience.
Tables 1 and 2 summarize the dataset characteristics. Table 1 provides tumor volume statistics across cohorts, including mean and median tumor volume, diameter, and average tumor counts. Table 2 compares key parameters between the training and test sets, with values reported as median along with their interquartile ranges (IQR).
This includes measures of in-plane resolution, slice thickness, volume size, tumor counts, and both liver and tumor volumes, as well as the intensity of liver and tumor regions in Hounsfield Units (HU).
Imaging protocols
All abdominal CT scans were acquired during the portal venous phase following protocols aligned with institutional standard of care protocols. Imaging was conducted using multi-detector CT systems (predominantly GE Discovery, LightSpeed, Revolution series; GE Healthcare, Madison, WI, USA) with detector configurations spanning 4–128 rows, detector widths of 0.6–1.25 mm, and total collimation widths of 10–80 mm. A subset of MDA scans in the ‘‘Resection’’ and ‘‘Chemotherapy’’ cohorts was acquired using scanners from other vendors.
Tube voltage was 100–140 kVp, and most commonly 120 kVp. Tube current was either fixed (MSK: mean 272–340 mA, range 119–755 mA; MDA: mean 283–326 mA, range 148–720 mA) or dynamically adjusted via GE smart mA tube current modulation with variable noise index (NI) and current range (NI: 8–20 for MSK and 9–33 for MDA; tube current 220–380 mA for MSK, 275–650 mA for MDA). Gantry rotation times were most commonly 0.7–0.8 s, with pitch 0.984, although a rotation time of 0.5 s and pitch factor of 0.516 was also common at MDA.
Reconstruction utilized soft-tissue kernels across all studies. Section thicknesses varied from sub-millimeter (0.625 mm) to 7.5 mm, with contiguous slices. Adaptive statistical iterative reconstruction (ASiR) was applied in more recent scans, typically at a level of 20%.
Iterative training strategy
We employed a six-phase curriculum learning strategy for training the segmentation model. In each phase, additional cohorts of CRLM patients were introduced to the training set, progressively improving the model’s generalization capability. Starting with a smaller dataset in Phase 1, we incrementally incorporated new data from different institutions and disease stages.
The specific number of samples introduced at each training phase is detailed in Table 3. These steps form the training workflow of the LCS.
To further improve segmentation accuracy, the model was iteratively refined using expert corrections. After each training phase, model outputs were reviewed by an abdominal radiologist (>10 years of post-fellowship experience). Errors were manually corrected, and the refined annotations were reintroduced into the training set, allowing the model to progressively learn from challenging cases.
This iterative feedback loop, depicted in Fig. 1, ensured continuous performance improvements across diverse imaging protocols and clinical conditions.
Segmentation model
The segmentation framework is based on a 3D Residual Encoder U-Net (ResEncUNet) built on nnU-Net [13,44] and corresponds to the model trained during Phase 5, which yielded the best performance. The final 3D architecture comprises seven encoder–decoder stages with feature depths [32, 64, 128, 256, 320, 320, 320]. All convolutional layers use 3 × 3 × 3 kernels, instance normalization, and LeakyReLU activations (α = 0.01).
Downsampling is implemented via strided convolutions, with stride patterns chosen to preserve higher axial resolution in deeper stages. Input patches of 96 × 256 × 256 voxels were used to balance memory constraints and field-of-view requirements. This model configuration serves as the core segmentation engine of the LCS.
Preprocessing and planning followed nnU-Net conventions with adaptations for anisotropic CT data. Images were read with SimpleITK and normalized using the nnU-Net CT scheme (foreground 0.5–99.5th percentile ≈ 9–208 HU; mean ≈ 111 HU; SD ≈ 34.9 HU), followed by z-score standardization. Resampling used nnU-Net’s default routines (cubic interpolation for image data; linear/nearest for segmentations and probability maps) to a target spacing of [1.5, 0.8066, 0.8066] mm to preserve axial detail across scans with slice thickness ranging from 0.8–7.5 mm.
For completeness, the planner also produced a 2D configuration (patch size 512 × 512, batch size 35) and a 3D full-resolution configuration (patch size 96 × 256 × 256, batch size 2); the latter was employed as our final model.
Data splits and leakage prevention.
All dataset partitioning was performed at the examination (patient) level to prevent any image from the same patient being present in both training and test sets. The multi-institutional data were stratified by cohort and split into an 80% training/validation set (355 examinations) and a 20% independent held-out test set (91 examinations), as reported in Table 3. Preprocessing (resampling, intensity normalization) and any hyperparameter selection were performed strictly within training/validation folds; final model evaluation used only the held-out test set.
For robustness assessment we also conducted a 5-fold cross-validation on the combined MSK+MDA institutional cohorts (reported in Section 3.5). Ensemble models (N = 5) were trained using distinct training splits; ensemble averaging and pairwise agreement were computed only after independent model training. These procedures minimize the risk of data leakage and ensure the test set reflects an unseen sample distribution.
Confidence scoring methodology
As part of the LCS, we introduce a metadata-enhanced confidence scoring module designed to quantify prediction reliability. To quantify the reliability of each segmentation prediction, we developed a confidence scoring methodology that integrates both model agreement and domain-specific factors. This framework provides a measure of prediction certainty and allows for the identification of low-confidence predictions requiring further review.
Below, we describe the key components of our confidence scoring methodology, including pairwise agreement among ensemble models, domain-specific adjustments, and the evaluation of confidence scores using the AURC.
Pairwise agreement and confidence score
To detect potential failures, we computed pairwise Dice similarity per case, taking the mean Dice coefficient across all unique pairs of model outputs. This calculation used the full segmentation mask for each case. For parenchyma and tumor, the same procedure was applied while restricting the Dice computation to the respective voxel regions.
The baseline confidence score was derived from pairwise agreement between ensemble models [19].
For N = 5 models, we computed the DSC for each model pair as: (1)DSCfi,fj=2Sfi∩SfjSfi+Sfj, where Sfi and Sfj are segmentation masks from models fi and fj. The ensemble confidence score was then calculated as: (2)κpairwise=1(N2)∑i=1N∑j=i+1NDSCfi,fj.
Confidence adjustments
The baseline confidence score was adjusted using two domain-specific factors:
Slice Thickness Weighting: Scans with zspacing > 5.0 mm received reduced confidence due to thick-slice artifacts: (3)αthickness=1.0,zspacing≤5.0mm,5.0/zspacing,otherwise.
Tumor Volume Weighting: Small tumors (Vtumor ≤ Vlower_bound) were penalized using linear interpolation between 0.5 and 1.0: (4)αtumor=0.5,Vtumor≤Vlower_bound,1.0,Vtumor≥Vupper_bound,Vtumor-Vlower_boundVupper_bound-Vlower_bound,otherwise.
The final confidence score combined these components: (5)κfinal,i=κpairwise,i×αthickness,i×αtumor,i.
Implementation details and parameter choices.
All experiments used an ensemble of N = 5 models and the pairwise agreement baseline defined in Eq. (2). We applied two domain-specific multiplicative adjustments to produce a final confidence score per prediction (Eq.
(5)).
Slice thickness (α-thickness).
We empirically selected a z-spacing threshold of 5.0 mm. Scans with z_spacing ≤ 5.0 mm keep full weight (αthickness = 1.0). For thicker scans we applied the inverse linear scaling αthickness = 5.0/z-spacing (Eq.
(3)), which reduces confidence smoothly as slice thickness increases. This choice reflects the observed degradation in Dice for thicker reconstructions (Fig. 4(a)).
Tumor volume (α-tumor).
For tumor-level weighting we penalize very small lesions using the linear scheme in Eq. (4). In the experiments reported here we set Vlower = 0.51 cm3 (the 1st quartile cut-off in the combined dataset; cf.
Fig. 4(b)) and Vupper = 2.0 cm3 (a pragmatic upper bound beyond which no penalty is applied). Thus lesions with Vtumor ≤ 0.51 cm3 receive αtumor = 0.5, lesions with Vtumor ≥ 2.0 cm3 receive αtumor = 1.0, and intermediate volumes are interpolated linearly.
We chose these bounds to (i) penalize subcentimeter/sub-Q1 lesions strongly (which show high Dice variance) and (ii) avoid penalizing larger lesions whose segmentation was robust in our data (Fig. 4(b)).
Case/scan aggregation rule.
For scans with multiple tumors we computed αtumor based on the average tumor volume across all lesions in that scan. Using the mean lesion size provides a balanced per-scan weighting that reflects the overall lesion burden without allowing a single very small lesion to disproportionately dominate the confidence score.
AURC evaluation
To assess the reliability of our confidence estimates, we evaluated the AURC [24]. AURC quantifies the trade-off between risk and coverage, where lower values indicate better confidence calibration.
For each test case xi, we defined segmentation risk as: (6)ri=1-DSCyi,ygt, where yi is the predicted segmentation and ygt the ground truth. The risk-coverage curve was computed as: (7)Rs(τ)=∑i=1Nri⋅Iκfinal,i≥τ∑i=1NIκfinal,i≥τ,(8)C(τ)=1N∑i=1NIκfinal,i≥τ, where I is an indicator function.
AURC was computed following the approach in [45], adapted for segmentation tasks. The impact of metadata-based confidence adjustments on AURC is analyzed and reported.
Results
Tumor volume and equivalent diameter distributions across cohorts
The impact of metadata-based confidence adjustments on AURC is shown in Table 3. A comparative analysis of tumor volume distributions across experimental cohorts is illustrated in Fig. 3.
Tumor volumes are reported in cm3, and equivalent sphere diameters (calculated from the volume assuming a spherical shape) are provided in parentheses for reference. The “All stages-MSK” cohort (Fig. 3(b)) comprised patients with more advanced, unresectable disease, contributing to its larger median tumor volumes compared to the MDA cohort (Fig.
3(a)), which included a more diverse CRLM population.
The All stages-MDA cohort (n = 1457 tumors) exhibited the lowest median tumor volume (0.378 cm3, equivalent sphere diameter ~8.9 mm), while the MSK cohort had higher median volumes (0.648 cm3, equivalent sphere diameter ~10.7 mm). Significant heterogeneity was observed across the cohorts, reflecting differences in patient populations and clinical contexts. The “Chemotherapy-MSK” cohort (Fig.
3(d)) (n = 754 tumors) had patients with unresectable disease and the highest tumor burden, with the largest tumor volume (representing a confluence of lesions) reaching 2771 cm3 (equivalent sphere diameter ~163.5 mm).
The “MSK resection” cohort (Fig. 3(c)), with a smaller number of tumors (n = 170), had a median tumor volume of 0.395 cm3 (equivalent sphere diameter ~9.1 mm), while the “TCIA-MSK” cohort (Fig. 3(e)) demonstrated the widest tumor volume range, from 1 mm3 (equivalent sphere diameter ~1.2 mm) to 1099 cm3(equivalent sphere diameter ~128.0 mm), with a median of 1.5 cm3 (equivalent sphere diameter ~14.2 mm).
Additionally, the dataset exhibited heterogeneous imaging parameters, with slice thicknesses in the test set ranging from 0.8 mm to 7.5 mm (Fig. 4(a)).
Impact of tumor size on segmentation performance
To explicitly link size distribution to model performance, we examined segmentation quality across tumor-size strata (Fig. 4(b)). In the combined dataset (n = 3959 tumors; median volume 0.579 cm3, IQR 3.1 cm3), lesions in the smallest quartile (≤0.51 cm3) exhibited substantially lower and more variable Dice scores compared with larger quartiles (Fig.
4(b)). This decrease in performance for small lesions is consistent with partial-volume effects, reduced lesion-to-parenchyma contrast and boundary ambiguity — factors that are exacerbated in scans with thicker reconstructions (slice thickness up to 7.5 mm; Fig. 4(a)).
Quantitative quartile-level DSC statistics are shown in Fig. 4(b); a brief discussion of mitigation strategies is given in the Discussion.
Iterative training performance
We employed a six-phase iterative training strategy, progressively incorporating additional cohorts and refining model predictions using expert corrections (see Table 3). Model performance improved steadily across phases as more data and expert feedback were added. For liver parenchyma segmentation, the DSC increased from 0.9618 ± 0.0444 in Phase 1 to 0.9718 ± 0.0191 in Phase 6.
For tumor segmentation, the DSC increased from 0.7106 ± 0.2815 in Phase 1 to 0.7584 ± 0.2590 in Phase 5. Although tumor performance showed greater variability, the overall upward trend highlights the value of iterative refinement for more challenging structures. Overall, these results demonstrate that our multi-phase training strategy reduced the need for manual corrections and minimized radiologists’ involvement.
For all six training phases, we initially used nnU-Net v1. Based on the tumor DSC, Phase 5 showed the strongest performance for tumor segmentation. When the improved nnU-Net v2 became available, we applied its ResEncUNet architecture [44] specifically to Phase 5 to obtain better results while conserving time and computational resources.
Although nnU-Net v2 typically outperforms v1 across all phases, we limited its use to the best-performing phase to avoid re-running all previous experiments.
Table 3 reports the performance achieved with nnU-Net v1 across all phases. In contrast, the Phase 5 ResEncUNet model achieved a liver parenchyma DSC of 0.9707 (95% CI: 0.9663–0.9751). For tumor segmentation, it achieved a DSC of 0.7695 (95% CI: 0.7166–0.8224), representing an improvement over the nnU-Net v1 tumor DSC of 0.7584.
The iterative training approach, combined with the final ResEncUNet model in Phase 5, leveraged NVIDIA A100 GPUs (40 GB) to efficiently process the large datasets and integrate the latest modifications to the model architecture and annotations.
Segmentation performance across phases
The boxplots and swarmplots in Fig. 5 present a comparison of segmentation performance in Phase 1 and Phase 5 using the DSC and the Normalized Surface Distance (NSD, 3 mm tolerance; a metric quantifying the percentage of surface points with inter-surface distances ≤3, mm) for both liver and tumor structures.
For liver segmentation, the model demonstrated consistent DSC improvement across cohorts, with the “All” cohort increasing from 0.9618 ± 0.0446 in Phase 1 to 0.9707 ± 0.0213 in Phase 5. Similar improvements were observed in other cohorts, such as “All stages-MDA” (from 0.9361 ± 0.0732 to 0.9610 ± 0.0438) and “All stages-MSK” (from 0.9537 ± 0.0776 to 0.9756 ± 0.0134), suggesting that liver segmentation accuracy benefited from the incorporation of additional training data.
Tumor segmentation showed more modest improvements compared to liver segmentation. The “All” cohort displayed an increase in tumor DSC from 0.7233 ± 0.2831 in Phase 1 to 0.7695 ± 0.2573 in Phase 5. The “All stages-MDA” cohort, however, exhibited a more significant improvement, with the tumor DSC rising from 0.8175 ± 0.2007 to 0.9247 ± 0.0521, suggesting that the model refined its tumor segmentation capabilities in this cohort.
For NSD, liver segmentation also showed a clear improvement from Phase 1 to Phase 5, with the “All stages-MDA” cohort increasing from 0.9264 ± 0.0663 to 0.9680 ± 0.0359. In contrast, tumor segmentation results were more variable, with the “Resection” cohort showing a smaller increase in NSD from 0.5485 ± 0.4380 in Phase 1 to 0.6949 ± 0.3406 in Phase 5.
Overall, the results indicate that the model’s performance improved over the training phases, particularly in liver segmentation. Tumor segmentation, while showing measurable improvement, exhibited greater variability across cohorts and phases. This variability is expected given that tumor segmentation is inherently more challenging than liver segmentation, partly due to inter-radiologist variability in assessing tumor burden.
In fact, the model nearly achieved human-level accuracy for tumor segmentation, suggesting that simply adding more samples may yield diminishing returns. Further improvements in tumor segmentation may require alternative strategies beyond increasing the dataset size.
Impact of tumor size on segmentation accuracy
Fig. 4(b) illustrates a positive correlation between tumor size and segmentation accuracy. Mean DSC increased by 435% from the smallest tumor quartile (Q1) to the largest (Q4).
Standard deviation showed an inverse correlation with tumor size, decreasing from 0.30 in Q1 to 0.19 in Q4. While larger tumors (Q3–Q4) achieved clinically relevant performance (DSC > 0.65), smaller tumors (Q1–Q2) exhibited greater variability, particularly in Q2, where the standard deviation (0.38) was close to the mean value (0.43). These findings underscore the need for specialized strategies to improve segmentation of lesions smaller than 0.51 cm3.
Cross-validation and model complexity
To complement the held-out test evaluation, we performed a 5-fold cross-validation on the combined MSK and MDA institutional cohorts and report both overall stability and cohort-level behavior. Over the cross-validation folds the liver segmentation achieved a mean DSC of 0.9707 ± 0.0012 and tumor segmentation a mean DSC of 0.7703 ± 0.0120, indicating low variance across splits and stable aggregate performance.
To inspect cohort-specific agreement between validation and the independent test set, we aggregated per-cohort mean DSC from cross-validation and compared these to held-out test DSC using radar (spider) plots (Figs. 6(a) and 6(b)). The Liver radar (Fig.
6(a)) shows consistently high parenchyma DSC across cohorts (held-out mean ≈ 0.971; cross-validation mean ≈ 0.971), indicating that liver segmentation is robust to cohort- and site-level differences. By contrast, the Tumor radar (Fig. 6(b)) reveals larger between-cohort variability: for example, the Resection cohort has a substantially lower held-out tumor DSC (0.433) than its cross-validation average (0.547), whereas the All stages-MDA cohort attains very high tumor DSC on both held-out (0.931) and cross-validation (0.906) evaluations.
Overall, cross-validation means closely track held-out performance (overall tumor: held-out ≈ 0.758 vs. CV ≈ 0.770), but cohort-level discrepancies highlight the effect of differing lesion distributions and imaging conditions and motivate the metadata-aware confidence scoring and site-specific calibration described in Section 2.5.
We also quantified model complexity and runtime characteristics for practical deployment. The final 3D ResEncUNet contains 141,073,790 trainable parameters and requires roughly 7277.78 GMac per forward pass on a full-resolution input patch. Average inference for a complete test volume on our evaluation GPU was approximately 2 min.
These measurements provide a realistic estimate of computational requirements for prospective clinical use and can guide resource planning for deployment.
Improvement of failure detection through confidence scoring adjustments
Incorporating slice thickness and tumor size into confidence scoring significantly improved failure detection performance, reducing the AURC from 16.6 (baseline) to 13.3 (slice thickness scaling) and ultimately to 10.5 (combined scaling) (Fig. 7). This result aligns with the theoretical optimal AURC framework proposed by [24], where the ideal confidence score is defined as: (9)κoptimal=-risk=-(1-DSC).
Although the optimal AURC, which is theoretically near 0 (since a perfect segmentation with DSC = 1 would yield zero risk), serves as a lower bound, in practice we focus on the relative improvements in AURC across different weighting strategies rather than reporting the absolute optimal value, which is rarely achieved due to inherent segmentation uncertainties.
The AURC quantifies uncertainty estimation quality by integrating segmentation risk (1 – DSC) across confidence thresholds. Unlike traditional metrics such as Pearson (ρ) or Spearman (rs) correlations — which measure rank associations — AURC provides a clinically interpretable benchmark where lower values directly reflect improved failure detection [24].
Table 4 highlights the model performance improvements between Phase 1 (single-cohort training) and Phase 5 (multi-cohort training) across various CRLM patient cohorts. Table 4 shows the AURC, Spearman, and Pearson correlation coefficients, which are used to evaluate segmentation reliability and confidence. Lower AURC values reflect improved failure detection, with the combined tumor volume and slice thickness weighting strategy yielding the lowest AURC values, particularly in the “All stages-MDA” cohort.
The results demonstrate a substantial improvement from Phase 1 to Phase 5 in most cohorts, with AURC reductions such as from 8.39 to 4.06 in the “All stages-MDA” cohort and from 42.70 to 22.53 in the “Resection-MSK” cohort. These findings underscore the efficacy of multi-cohort training and metadata-driven confidence adjustments in enhancing segmentation robustness across diverse patient populations and imaging protocols.
Discussion
Key findings
Our proposed LCS demonstrated high segmentation accuracy. The model achieved a DSC of 0.9713 for liver parenchyma and 0.7584 for liver tumors, resulting in an average combined liver+tumor DSC of 0.8649. For context, a recent benchmarking study (“nnU-Net Revisited”) [46] reports average combined liver+tumor DSCs of 0.8234 for CNN-based methods, 0.7910 for Transformer-based methods, and 0.8040 for Mamba-based methods on the LiTS dataset [8].
Our combined DSC of 0.8649 therefore represents an absolute improvement of +0.0415 (≈4.15 percentage points) over the CNN-based benchmark, +0.0739 (≈7.39 percentage points) over the Transformer-based benchmark, and +0.0609 (≈6.09 percentage points) over the Mamba-based benchmark.
We emphasize that direct comparisons should be interpreted with caution. The benchmark results were obtained on LiTS, whereas our results are based on a larger, multi-institutional dataset with different imaging characteristics and a broader range of clinical cases; these differences can affect absolute performance. Nevertheless, the explicit gains reported above strengthen the claim that our iterative training and multi-institutional data contributed to improved and more robust segmentation performance.
Finally, the integration of an iterative training framework — progressively refining the training set with expert-guided corrections — reduced the need for manual corrections and helped minimize radiologists’ intervention in finalizing segmentations, supporting clinical applicability.
Metadata-enhanced confidence scoring
A key contribution of this study is the development of a metadata-enhanced confidence scoring framework that incorporates tumor size and slice thickness. Unlike traditional uncertainty-based methods, which often fail to account for clinically relevant imaging variabilities, our approach reduced the AURC from 16.6 (baseline) to 10.5 (combined scaling) by penalizing predictions on small tumors and thick-slice acquisitions. This reduction underscores the value of metadata-driven reliability estimation, particularly for clinical deployment where segmentation errors could significantly impact treatment decisions.
The confidence scoring framework also demonstrated strong correlations between tumor volume, slice thickness, and segmentation reliability. For a focused discussion of the particular challenges posed by small lesions (which show markedly lower and more variable DSC; see Fig. 4(b)), and our mitigation strategies, see the Limitations and Future Work subsection below.
Clinical implications
The ability to flag low-confidence segmentations using metadata-derived confidence scores provides a mechanism for targeted expert review, thereby enabling radiologists to focus on cases that require manual intervention. By prioritizing these cases, the framework has the potential to reduce radiologists’ workload while maintaining high segmentation accuracy. In our study, the observed reduction in manual corrections suggests that confidence-aware segmentation models could streamline radiological workflows and enhance overall efficiency.
Moreover, the public release of our trained models and source code supports reproducibility and encourages broader adoption of the model in both clinical and research settings.
In addition to these workflow benefits, it is important to recognize the clinical limitations and appropriate use-cases of the proposed model. Although the system achieves high liver accuracy and strong tumor performance for larger lesions, segmentation of subcentimeter tumors remains challenging, as reflected by the lower DSC observed in the smallest quartile (Fig. 4(b)).
Performance is also affected by imaging quality, particularly thicker reconstructions (up to 7.5 mm), which introduce partial-volume artifacts. For these reasons, the model is best used as an assistive tool that automatically accepts high-confidence cases while directing radiologists’ attention to low-confidence or small-lesion cases flagged by the metadata-enhanced confidence score. Prospective validation and site-specific calibration are recommended before deployment, and thin-slice acquisitions should be preferred when accurate assessment of small lesions is clinically required.
Clinically, our LCS can function as an assistive, triage-capable tool that automatically accepts high-confidence scans while flagging low-confidence cases for radiologist review, potentially reducing manual correction burden and improving measurement consistency (liver DSC = 0.9713, tumor DSC = 0.7584, combined = 0.8649; AURC reduced 16.6 → 10.5). However, subcentimeter lesions and thick-slice reconstructions remain important limitations — the model should therefore be used with human-in-the-loop review, site-specific calibration, and preference for thin-slice imaging when small-lesion assessment is critical.
Limitations and future work
Despite the strong segmentation performance, several limitations remain. Small tumors pose particular challenges for automated segmentation in portal-venous CT. In our data the lesion size distribution is heavily right-skewed and a large fraction of lesions are subcentimeter (per-lesion median ≈ 10.4 mm; several cohorts have median diameters <10–11 mm), and the smallest lesions in the TCIA subset approach 1 mm3 in volume.
Importantly, lesions in the smallest quartile (≤0.51 cm3) exhibit substantially lower and more variable DSC (Fig. 4(b)), which we attribute to partial-volume effects, reduced lesion-to-parenchyma contrast, and boundary ambiguity. These issues are exacerbated by thicker reconstructions (slice thickness up to 7.5 mm).
To mitigate these effects we (i) resampled to preserve axial detail during preprocessing, (ii) incorporated slice-thickness and tumor-volume weighting into our confidence score to flag likely failures, and (iii) used iterative human-in-the-loop corrections and an ensemble strategy to improve robustness. For clinical deployment we recommend — where feasible — use of thinner reconstructions or targeted high-resolution scanning for small-lesion workups, lesion-aware loss weighting or focal-loss formulations, and targeted augmentation or oversampling of subcentimeter lesions during training to reduce variability in the smallest quartiles.
While the proposed metadata-enhanced confidence score improves failure detection (AURC reduced from 16.6 to 10.5; Fig. 6), it has limitations that warrant explicit discussion. First, the score relies on a small set of heuristic adjustments (slice-thickness scaling and tumor-volume interpolation) whose thresholds (e.g., Z-spacing = 5.0 mm, Vlower = 0.51 cm3, Vupper = 2.0 cm3) were selected empirically; these values are reported in Section 2.5 and may require site-specific recalibration.
Second, our per-scan aggregation rule (mean lesion volume) provides a balanced estimate of lesion burden but can under-emphasize a single clinically important tiny lesion in scans with multiple larger lesions; alternative aggregation choices (minimum or median) offer different safety/performance trade-offs and should be considered based on clinical priorities. Third, ensemble pairwise agreement may be biased if the ensemble shares common failure modes or lacks architectural diversity, and the current score does not explicitly quantify epistemic uncertainty for out-of-distribution inputs (e.g., unusual reconstruction kernels or vendor-specific artifacts). Finally, annotation bias and class imbalance (a preponderance of small lesions) can influence both model predictions and the estimated confidence.
To reduce these risks we recommend the following next steps: carry out sensitivity analyses for threshold and aggregation choices, evaluate complementary uncertainty estimators (e.g., Bayesian/MC-dropout or ensembles with greater architectural diversity), perform prospective site-level calibration and validation, and retain a human-in-the-loop review step for low-confidence predictions. Future work will also explore advanced augmentation, super-resolution or targeted acquisition protocols to improve small-lesion performance and the integration of principled uncertainty quantification to better capture out-of-distribution risk. Prospective multi-center evaluation and clinical workflow integration remain prerequisites for safe deployment.
Conclusion
We presented an iterative, expert-in-the-loop training strategy combined with a metadata-enhanced confidence scoring framework that together materially improve automated segmentation of liver parenchyma and CRLM. On our multi-institutional dataset the system achieved strong segmentation accuracy (liver DSC ≈ 0.97; tumor DSC ≈ 0.76; combined DSC ≈ 0.865), improved reliability in failure detection (AURC reduced from 16.6 to 10.5 with combined scaling), and reduced the need for manual corrections during label curation—demonstrating both technical performance gains and practical workflow benefits.
These quantitative gains translate into two complementary clinical advantages: (1) reliable automatic acceptance of high-confidence cases, which can streamline routine volumetric reporting and reduce radiologist time spent on trivial edits; and (2) targeted triage of low-confidence cases to expert review, which concentrates human effort where it matters most. At the same time, important limitations temper immediate deployment: segmentation performance degrades for subcentimeter lesions (smallest quartile ≤ 0.51 cm3), thicker reconstructions increase partial-volume errors, the confidence score depends on empirically chosen thresholds that may require site-specific recalibration, and ensemble agreement can be susceptible to common failure modes and annotation bias.
To translate these results into safe clinical practice we recommend the following next steps: prospective, multi-center validation and site-level calibration of confidence thresholds; integration of principled uncertainty estimators (e.g., Bayesian/MC-dropout or more architecturally diverse ensembles); targeted strategies to improve small-lesion performance (high-resolution acquisitions, super-resolution, lesion-aware loss weighting, and focused augmentation/oversampling); evaluation of alternative per-scan aggregation rules (minimum/median) to prioritize clinically important tiny lesions; and user-interface and workflow studies to quantify radiologist time-savings and error modes in real-world use. Finally, continued public release of models, code, and evaluation scripts will facilitate external replication, safety auditing, and regulatory review—necessary steps before widespread clinical deployment.
In summary, our approach advances the reliability and clinical readiness of deep learning–based liver and tumor segmentation by combining improved accuracy with metadata-aware reliability estimation, while also identifying clear technical and validation milestones that must be met prior to routine clinical adoption.