key: cord-0806416-yrugc0zg
authors: Ansari, Mohammed Yusuf; Abdalla, Alhusain; Ansari, Mohammed Yaqoob; Ansari, Mohammed Ishaq; Malluhi, Byanne; Mohanty, Snigdha; Mishra, Subhashree; Singh, Sudhansu Sekhar; Abinahed, Julien; Al-Ansari, Abdulla; Balakrishnan, Shidin; Dakua, Sarada Prasad
title: Practical utility of liver segmentation methods in clinical surgeries and interventions
date: 2022-05-24
journal: BMC Med Imaging
DOI: 10.1186/s12880-022-00825-2
sha: a0ec40b9b0281f376bbd0aa9b306a5f29199f411
doc_id: 806416
cord_uid: yrugc0zg

Clinical imaging (e.g., magnetic resonance imaging and computed tomography) is a crucial adjunct for clinicians, aiding in the diagnosis of diseases and planning of appropriate interventions. This is especially true in malignant conditions such as hepatocellular carcinoma (HCC), where image segmentation (such as accurate delineation of liver and tumor) is the preliminary step taken by the clinicians to optimize diagnosis, staging, and treatment planning and intervention (e.g., transplantation, surgical resection, radiotherapy, PVE, embolization, etc). Thus, segmentation methods could potentially impact the diagnosis and treatment outcomes. This paper comprehensively reviews the literature (during the year 2012–2021) for relevant segmentation methods and proposes a broad categorization based on their clinical utility (i.e., surgical and radiological interventions) in HCC. The categorization is based on the parameters such as precision, accuracy, and automation.

The World Health Organization (WHO) has reported Hepatocellular Carcinoma (HCC) as the leading cause of cancer deaths worldwide. In 2020, liver cancer has resulted in 830,000 deaths, and HCC has accounted for about 80% of primary liver cancers [1] . Surgeons, radiologists, and oncologists study liver physiology, pathology and morphology using multiple tools to evaluate the disease state. For example, imaging modalities such as CT/ MRI images are utilized for computer-aided diagnoses (CAD) to evaluate pathologic liver conditions. Segmentation of CT/MRI liver images greatly augment clinical decision support by playing an essential role in existing CAD systems. They enable the surgeons to identify the lesions that have even similar gray-level intensities as the liver, and help them devise case-appropriate treatment pathways. Precision and reliability of segmentation methods are important for obtaining clinically relevant boundary and volumetric assessments in staging of liver tumors (e.g., Response Evaluation Criteria in Solid Tumor (RECIST) protocol) [2] .

Accurate delineation of liver and tumor helps in appropriate planning for HCC treatment [3, 4] . Thus, liver and tumor segmentation methods play a critical role in treatment approaches such as Radio-frequency Ablation (RFA), Percutaneous Ethanol Injection (PEI), Selective Radiation Therapy (SIRT), Transcatheter Arterial Chemoembolization (TACE), and the use of targeted agents [5] . Liver and tumor segmentation are also a prerequisite for surgical resection [6] . Segmentation is also crucial in post-interventional tracking of ablated/resected tissues of the liver; it also ensures appropriate negative tissue margins, allowing the clinician to evaluate the efficacy and success of the procedure. Liver and tumor segmentation Open Access *Correspondence: SDakua@hamad.qa thus play an important role in the diagnosis, treatment, and follow-up of HCC patients [5] .

The categorization of segmentation methods has been often subjective; they are generally classified based on their methodology or extent of human intervention. Methodology based categorizations include model-based (e.g., active contours [7] , statistical shape models [8] , and graph cuts [9] ), or intensity-based (e.g., region growing [10] ) approaches. Model-based approaches tend to attain better segmentation performance than intensitybased methods due to their accurate statistical and mathematical modeling for capturing the region of interest. However, model-based approaches may require parameter tuning and initialization, resulting in high computational time. Segmentation methods can also be classified as semi-automatic or automatic based on the extent of human intervention. The semi-automatic methods require the surgeon's assistance, thereby providing more control to the surgeons during the procedure. On the other hand, the automatic methods have minimal user errors due to a lack of external interventions. However, the automatic segmentation methods are biased towards the statistical distribution of the training data. Recently, deep learning-based (i.e., model-based) methods have gained popularity due to advances in accuracy, robustness, and generalization provided by the neural networks [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] .

Differences in liver morphologies and pathologies between people result in varying liver volumes and shapes in CT/MR imaging, thereby adding complexity to the development of accurate liver and tumor segmentation methods. Furthermore, usage of contrast media injections increases the complexity of the segmentation process, because it alters the grey level values of the liver, making its intensity similar to neighboring tissues/organs such as the stomach, spleen, and abdominal wall [21] . Figure 1 shows the challenge faced by the segmentation algorithms due to ambiguous anatomical liver boundaries. Liver lesion segmentation is also challenging due to variable contrast levels (i.e., hyper-/hypo-intense tumors) and broad-spectrum abnormalities (inconsistent size and shape of lesions) [5] .

Thus, in order to suggest utility within surgical hepatobiliary interventions, it would be useful to first compare and categorize different segmentation methods with respect to their accuracy, extent of automation, and segmentation performance. These metrics are expected to vary based on liver pathology and morphology, as well the staging and spread of malignancies such as HCC. Some efforts have already been made in this regard, for instance, Bilic et al. [5] assess over 24 state-of-the-art liver and tumor segmentation methods and conclude that a single segmentation algorithm might not always be the best fit for segmenting the liver and its tumors.

In this paper, we propose a categorization schema of segmentation methods with respect to their utility in diagnosing HCC and planning hepatobiliary surgical interventions. Specifically, we consider suggesting segmentation methods for the outcomes of an early-stage HCC treatment protocol (i.e., BCLC). The diagnosis of HCC is a complex procedure that may be impacted by existing liver morphology (due to diseases such as liver cirrhosis) [22] ) as well as tumor pathology (size [23] , intensity [24] , malignant or benign nature [25] ). There have been extensive publications in the literature, which classify and detect tumors [26] [27] [28] . In this work, we categorize the existing liver and tumor segmentation methods as per their suitability for appropriate therapeutic Fig. 1 Examples of challenges in liver segmentation: a ambiguous boundary between liver and stomach, b ambiguous boundary between liver and heart, c similar intensity of liver and tumor pathways. The scope of the paper is to propose a categorization schema that could serve as a knowledge base for the treating physician (surgeon/radiologist) for expediting their segmentation tool selection from existing choices. The primary focus of our paper is on segmentation methods of the liver and tumor. In order to accomplish this, We have extensively reviewed the literature (surgical and nonsurgical interventions for HCC treatment) and categorized them based on some pre-defined critical parameters such as accuracy, automation, and precision.

This paper is structured as follows; Section 2 provides an overview of the liver and tumor segmentation methods. Section 3 and 4 present the segmentation methods for surgical and radiological interventions, respectively. Section 5 discusses the technical and clinical challenges facing the segmentation algorithms and treatment of HCC. Section 6 concludes the paper and provides critical future directions related to the liver and lesion segmentation.

Liver segmentation in CT and MRI scans is challenging due to variability in liver dimensions and comparable gray-level intensity of its neighboring organs (e.g., heart and kidney). Furthermore, the blurry anatomical boundaries, poor contrast of the medical images, partial volume effects resulting from patient movement, spatial averaging, and reconstruction artifacts make the liver segmentation daunting. Figure 2 describes the application of segmentation for the other parts of liver.

Generally, the conventional image segmentation methods are either model-based (e.g., active contours, and snake algorithms) [29] [30] [31] [32] [33] or intensity-based (e.g., thresholding). Some algorithms utilize primitive image features, e.g., pixels intensities in region-growing/thresholding based approaches [30, [34] [35] [36] . Fuzzy segmentation methods have also been utilized for multi-channel image segmentation and extended for single-channel images [37, 38] . Zhang et al. [8] , and Nuzillard et al. [39] have shown that model-based statistical approaches achieve expected results relative to the conventional segmentation methods based on image intensities. Mahr et al. [40] evaluate several segmentation methods and conclude that model-based methods are the potential futures for liver segmentation. However, the statistical models are limited by constraints and require additional parameter tuning and initialization, resulting in high computational time. If the models are evaluated without the constraints, they have the potential to misclassify or under-segment the region of interests.

As discussed earlier, there is also a different categorization that is based on the extent of human intervention; some of them are automatic [41] [42] [43] [44] [45] [46] [47] [48] that do not require human intervention for the generation of segmentation masks and some are semi-automatic requiring human assistance, say for seed selection or segmentation mask refinement [49] [50] [51] [52] [53] . Linguraru et al. [42] suggest an automatic segmentation approach for liver segmentation based on an affine invariant shape formulation. The paper makes a point-to-point comparison of various 3D surface features in the affine parameter space. Another automatic method proposed by Seo et al. [54] follows a multi-stage approach by utilizing an optimal threshold value to segment liver, hepatic vessels, and tumors sequentially. Chartrand et al. [49] introduce a semi-automatic liver segmentation technique that generates an approximate liver model and deforms it by using a Laplacian mesh optimization to obtain accurate liver segmentation. Peng et al. [52] utilize semi-automatic level sets that integrate the likelihood energy and anatomical boundary information to segment the liver. Zhang et al. [50] propose a semi-automatic method based on Couinaud's theory to segment the liver with varying clinical conditions. Zhao et al. [55] report a semi-automatic region-growing method that avoids over-predictions of the surrounding tissues and organs using shape constraints. The primary shortcoming of semi-automatic implementations is user intervention that interrupts the segmentation process and results in subjective outcomes. From this standpoint, it is essential to note that automated segmentation methods are preferred in time-constrained clinical applications.

Segmentation of vessels, tumors, and bile duct also play an important in the diagnosis, treatment, and post-treatment evaluation of HCC [56, 57] . These liver components are relatively minute, thus their segmentation is certainly challenging. In addition, artifacts, poor contrast, and distortions in the standard image at the native scale/ resolution increase the segmentation complexity. Specific denoising and image enhancement methods improve the images overpowering the high noise levels and artifacts (e.g., Wavelet and Ridgelet transform). These methods transform the image to a different domain in order to segregate the image noise [40, 58] . Shang et al. [48] propose an active contour-based method that uses a Gaussian mixture model to segment major liver vessels. Then, a vascular vector field centerline segments thin vessels with lower visibility. Kirbas et al. [59] present a comprehensive review to understand the conventional vessel segmentation algorithms. Wang et al. [60] suggest a model-based algorithm to detect and segment bile duct carcinoma. Similar model-based approaches have been proposed for detecting and segmenting liver malignancies (e.g., HCC) [42, 43] .

In recent years, machine learning and deep learning have vigorously gained popularity in medical image segmentation. Specifically, the U-net architecture, proposed for biomedical image segmentation, has been modified for the segmentation of organs in CT/MRI images [61] . Although the deep learning algorithms provide acceptable reliability and accurate results, they require large datasets and dedicated hardware (i.e., GPU). The challenges related to limited data and computation cost have been mitigated using data augmentation and efficient network layers.

Alexey et al. [16] present a hybrid convolutional LSTM architecture that merges time distributed convolutions, bi-directional C-LSTM blocks, and pooling operations to work with partial input volumes. The resultant hybrid network is competitive in terms of computational power, memory consumption, and inference times for liver segmentation. Kavur et al. [12] perform an empirical study to evaluate the accuracy and repeatability of twelve semi-automatic and automatic methods. The authors have exhaustively evaluated the methods with segmentation metrics (e.g., VOE, average symmetrical surface distances, etc.) and compared their results with sliceby-slice evaluations and a scoring system. The results highlight that the automatic deep learning methods outperform the conventional semi-automatic methods on all segmentation metrics. Isensee [13] propose a neural network-based segmentation framework (nnU-Net) that automatically configures preprocessing, network architecture, training, and post-processing for biomedical image segmentation. The results indicate that the framework achieves state-of-the-art accuracy on many image segmentation tasks. Furthermore, the nnU-Net framework provides more robust baselines for abdominal organs and sub-structures (e.g., liver, tumor, and vessels). Following the nnU-Net, researchers have organized challenges that improve the practicality and usability of the segmentation methods. FLARE 21 1 encourages the development of abdominal organ segmentation architectures for CT scans that can be deployed in memorylimited and time-constrained environments. Zhang et al. [14] introduce an efficient context-aware architecture for meeting the objectives of the FLARE 21 challenge, thereby maximizing segmentation accuracy while minimizing inference time and GPU memory consumption. Complementary to FLARE 21, the CHAOS [15] challenge promotes deep learning architectures that generalize well for abdominal organs (e.g., liver) across imaging modalities (i.e., CT and MRI). Conze et al. [62] suggest a conditional generative adversarial network with a partially pre-trained generator to achieve high segmentation accuracy across CT and MRI imaging modalities in the CHAOS challenge. Organ-focused challenges such as KiTS 2 have been organized to overcome the pathological and imaging modality constraints associated with an organ (e.g., kidney). Chen et al. achieve high accuracy in KiTS 21 by employing a coarse to fine segmentation approach inspired by the nnU-Net framework with the surface loss function to maximize area overlap and minimize surface discrepancies. Altogether, there has been a rapid development of neural network-based solutions for abdominal organ segmentation. In this paper, we aim to suggest a few of the robust liver and tumor segmentation methods to assist in the outcomes (e.g., transplant, resection, and ablation) of the clinical protocols for HCC (e.g., Barcelona staging classification). Table 1 provides a summary of community-organized challenges and some of their publications.

HCC prognosis is dependant on tumor stage as well as residual hepatic dysfunction due to cirrhosis. Extent of symptomatic presentation and comorbidities also contribute to prognosis. Various HCC staging systems are used to clinically guide HCC treatment, such as Okuda system, Tumor, Node, Metastasis (TNM) staging, Cancer of the Liver Italian Program (CLIP) score, Barcelona staging classification (BCLC), Albumin-Bilirubin (ALBI) score, etc. [64] [65] [66] [67] [68] . Among these staging systems, BCLC system is preferably used by clinicians due to its holistic approach, as it accounts for the extent of hepatic lesion, vascular invasion, hepatic function status, and spread outside the liver [68] . Furthermore, several studies have also reported that the BCLC system outperforms other systems in predicting HCC prognosis due to a more holistic tumor staging [69, 70] . Thus, using BCLC, clinicians are likely to decide the most appropriate therapeutic intervention for patients suffering from HCC. Figure 3 showcases the staging and treatment recommendations according to the BCLC criteria. Studies have shown that patients diagnosed with large multifocal tumors or advanced stage HCC are less likely to benefit from transplantation, liver resection, and ablation therapy [71] [72] [73] [74] . Nevertheless, the patients diagnosed with the early or initial stage of HCC with no liver diseases can be treated by surgical liver resection [75] . Figure 4 summarizes the role of segmentation and volumetry in surgical and radiological interventions of the liver.

A liver transplant is recommended for treating patients with very early stages of HCC and increased portal pressure or bilirubin levels 3. However, clinicians need to ensure that the transplanted liver is well matched as this may impact the functional capabilities of the liver. In liver transplant surgeries, accurate segmentation and calculation of liver volume are critical, as the success of the donor and recipient operations depends heavily on the graft size [76] . The accepted standard for liver segmentation and volume calculation is manual delineation, which requires a highly experienced surgeon to manually trace the boundaries of the liver on CT/MRI images. However, manual tracing of organs is time-consuming and idiosyncratic. Therefore, reliable and automated segmentation methods with fast inference times can reduce the complexity of the liver transplantation procedure. It has been shown that structure-based, machine learning, and deep learning methods are suitable for liver delineation and volume estimation because of their robustness and ability to achieve high segmentation accuracy [77, 78] . These methods learn to identify variations of the liver shapes that may be missed by conventional segmentation algorithms, allowing for robust and consistent boundary delineation and volume calculation. 

Liver segmentation in 3-D medical scans is a crucial prerequisite for the calculation of liver volume and liver/tumor ratio (i.e., tumor burden) [84, 85] . Liver segmentation methods with high precision and accuracy (comparable to manual delineation) are highly desirable in clinical workflows. As discussed earlier, the deep learning-based approaches have recently provided robust and accurate liver segmentation methods that may assist in liver transplantation. Alirr 

Liver and lesion volumetry, provide valuable information to the surgeons contributing to the success of liver transplantation [5, 86] . Over the years, several automated segmentation methods have been proposed to segment the liver in CT and MRI imaging. Recently, Lu et al. [78] introduce a 3D convolutional neural network (CNN) with graph cut to delineate the liver and predict its volume. The model's evaluation on the MICCAI-Sliver07 and 3DIRCADb datasets result in a volumetric overlap error (VOE) of 5.9% and 9.36%, respectively. Wang et al. [79] present a computationally light 2D U-Net variant for liver segmentation and volumetry. The model is trained using 330 abdominal CT examinations in two stages, allowing coarse fine segmentation. Results show that the proposed model reaches over 95% agreement with the ground truth. In [80] , a comparison of Vivo hepatic automated volumetry with manual volumetry is performed to assess the effectiveness and margin of error for automated segmentation methods in liver transplantation scenarios. These neural network-based systems could provide viable information to the clinicians for deciding donor-patient compatibility based on liver volume estimation. Table 2 provides a summary of liver segmentation and volume estimation methods along with the datasets and performance. 

BCLC staging system recommends liver resection for patients suffering from early or initial stage HCC with a single tumor, normal portal pressure, and bilirubin levels.

For physicians, tumor information plays a critical role in surgical planning and image-guided interventions. Specifically, the exact volume, morphology, shape, and location of tumors must be accurately determined to carry out a successful resection procedure. In a conventional setting, surgeons manually delineate liver lesions by relying on their experience and observations, which results in biased outcomes that lack efficiency and robustness.

Therefore, automated liver tumor segmentation methods are considered as a crucial second-opinion for interventional radiologists and surgeons. However, performing automatic tumor segmentation is quite challenging due to the low intensity, poor contrast, and anatomical variation of liver and lesions between patients. Specifically, sometimes the tumors vary in shape, size, and location, making the algorithm challenging to generalize for a diverse patient population. In addition, unclear boundaries of some lesions (as in Fig. 5 ) make it difficult for edge-based algorithms to perform effectively. Furthermore, the variations in anisotropic dimensions of the medical scans (i.e., voxel space ranging from 0.45 mm to 6.0 mm) may cause loss of critical volumetric information. Nevertheless, several conventional methods, deformable models, and neural networks have been proposed to segment liver lesions [87] [88] [89] .

One significant challenge for the tumor segmentation algorithm is the inconsistency in tumor shapes and locations between patients. Deep learning models can overcome this challenge by using training data that contains a diverse patient population with tumors of different shapes, sizes, and locations. It has been shown in literature that neural networks' robustness and generalization capability increase with the quantity and diversity of the dataset. There have been a few high-accuracy liver tumor segmentation models based on deep learning: Zhang et al. [88] propose a level-set technique for CT-based liver tumor segmentation that incorporates an edge indicator and an automatically computed initial curve. The method employs a 2D-slice-based U-net to localize the liver, followed by a 3D patch-based FCN to refine the liver segmentation and locate the tumor. After segmenting the existing tumors in the liver, the surgeons and radiologists need to recognize its type to determine the extent of cancer spread and malignancy. This classification process can be automated and embedded within the segmentation algorithm, efficiently providing physicians valuable secondary information. Trivizakis et al. [26] train a 3-D CNN using 130 DW-MRI scans to classify the tumor type. The network results in an 83% classification accuracy as compared to 69.6% of a previously implemented 2-D CNN. Chen et al. [27] present a probabilistic neural network (PNN) that is trained using fractal information gray-level co-occurrence matrix to classify liver tumors into hepatoma and hemangioma. Balagourouchetty et al. [28] suggest an ensemble FCNet classifier trained using GoogLeNet features to classify six different classes of liver tumors.

The heterogeneous shape of tumors, and inconsistent background, creates high unpredictability between the liver and the lesions, thereby adding complexity to automatic tumor segmentation methods [77] . Most interactive or semi-automatic methods that involve input from a physician have shown better results and are used for critical hepatic operations like hepatic biopsies and hepatic therapeutic interventions [96] . Zhang et al. [97] present an interactive seed-selection strategy for liver tumor segmentation using support vector machines in CT scans. Lin et al. [92] propose an interactive implementation that places emphasis on region partition and boundary information. The tumor texture information and clear tumor boundary allow the model to segment tumors effectively. Moreover, the Lucas-Kanade algorithm selects the seed pixel for initiating model training, and user inputs are utilized to incorporate the data variations. The collaborative model obtains promising results and an average segmentation accuracy of 80%. On the other hand, fully automated methods lack performance because of the complex and volatile nature of surgeries and complications [98] . Nonetheless, complete automation is being consistently pursued to achieve performance that is comparable to semi-automatic methods. A fully automated deep learning approach based on Attention Hybrid Connection (AHC) Network architecture is implemented by [93] , giving decent results. The network is tested using 20 cases from the 3DIRCADb dataset and 117 cases from a clinical dataset, achieving a global Dice coefficient of 0.62 ± 0.07 in tumor segmentation. Seo et al. [94] proposes a modified U-Net (mU-Net), which combines object-dependent high-level features to improve liver-tumor and liver segmentation from CT scans. The model's evaluation on the (LiTS) dataset results in a Dice similarity coefficient (DSC) of 89.72 % for liver tumors. Vivanti et al. [95] present an automatic method for liver tumors segmentation in post-treatment CT studies that use a CNN to image patches. Next, a voxel classifier is employed to generate the refined tumor segmentation mask. The model's evaluation on a custom dataset results in an average of 16.05% VOE and a 2.05 mm average symmetric surface distance (ASSD), giving a success rate of 90.5%. Table 3 provides a summary of liver tumor segmentation methods along with the datasets and performance.

Interventional radiology has opened new avenues for the treatment of liver cancers. BCLC staging system recommends radiological interventions (e.g., ablation) for patients not suited for transplant or with livers with associated diseases. Radiological treatments can be performed by an endovascular approach or by direct transcapsular access [99] . Endovascular treatments include TACE, Stereotactic Body Radiation Therapy (SBRT), Transarterial Radioembolization (TARE), and portal vein embolization (PVE). Direct transcapsular access treatments involve microwave thermal ablation (MWA), RFA, and PEI [100] .

TARE and TACE block the hepatic artery to treat the liver cancer segment by cutting off its blood supply. TARE is a selective internal radiation therapy that requires an intra-arterial supply of microspheres packed with radioactive compounds such as ttrium90, iodine131, or rhenium188 [101] . In comparison, TACE is a type of chemoembolization that involves chemotherapy. CT or MRI imaging is used to predict whether or not extra-hepatic arteries augment tumors. All of the feeding arteries of a tumor, including any possible extra-hepatic arteries, are examined by angiographic images. TACE can treat liver tumors larger than 5 cm, but it may take 2 or 3 treatments [102] . Furthermore, CT scans must be taken 2 to 3 months after TACE to ensure treatment success [103] . Liver tumor segmentation methods may provide crucial secondary information to monitor treatment progress and success.

PVE increases the volume of the Future Liver Remnant (FLR) for extended hepatectomy by embolizing a portal vein region, resulting in hepatic regeneration. PVE is performed, when a large FLR is required for a post-operative liver recovery as determined by liver volumetry. Often this is due to the extent of the liver resection or the underlying liver disease [104] . Segmentation and volumetry of CT scans provide crucial pre-requisite information for the success of PVE. SBRT has been used in the treatment of primary HCC (with slight metastases) that require radiation in less than 25% [105] .

RFA and MWA use image guidance intervention, where a probe is utilized for heat generation, resulting in coagulation necrosis to destroy the cancer cells [99] . PEI is performed for tumors less than or equal to 3 cm. PEI injects highly concentrated alcohol using a thin needle, leading to complete ablation of up to 70% of lesions. Ultrasound guidance is generally utilized for performing ablation, and the treatment requires 4-6 sessions. Realtime segmentation of the captured images can assist the radiologist in carrying out the procedure and improve treatment success.

FLR, Total Liver Volume (TLV), and liver burden are all important volumetry metrics needed for radiological intervention treatment planning [106] . CT or MRI segmentation and volumetry can provide crucial secondary information for radiologists to carry out radiological interventions. Advancements in radiation therapy procedures and segmentation technology effectively reduce GI toxicity (i.e., toxicity in small intestine and stomach) and spinal cord toxicity caused by inadequate/inaccurate dosage determination, leading to more liver dysfunction. For example, radiation-induced liver disease can result from inadequate treatment SBRT planning [107] . Having an accurate liver tumor segmentation enables the secure computation of chemical and radio dosages. Therefore, accuracy and precision are essential considerations for segmentation methods for these applications.

Deep-learning approaches that involve cascaded U-Net derived architectures are accurate with respect to performance [5] . According to the literature, the top three ranked methods according to tumor burden estimation (quantified by RMSE) include Li et al. 's [108] Adaboost to identify tumor boundaries, Wu et al. 's [9] supervoxelbased graph cuts, and Wang et al. 's [109] adaptive mesh expansion model (AMEM) for segmentation of liver. These methods accurately predict the tumor volume with a root mean square error (RMSE) in tumor burden of 0.0150, 0.0160, and 0.0160, respectively [5] . In general, these methods also achieve good area-overlap with the ground truth with Dice scores of 0.9650, 0.9590, and 0.962, respectively [5] . Yuan et al. [110] have also proposed a hierarchical CDNN that performs coarse to fine liver, tumor segmentation, and tumor burden analysis. Despite the state-of-the-art performance, automatically segmenting small liver tumors remains a difficult task. This limitation suggests that future improvements can be made by investigating methods that segment a broad spectrum of liver tumors. Table 4 summarizes the methods, datasets, and performance metrics, for radiation therapy of the liver.

The diagnosis and treatment planning of hepatic diseases (like HCC) are generally decided by the location and spread of liver lesions, proximity to the vasculature, severity of underlying liver dysfunction, availability of medical technology, and expertise of the clinician. Thus, the choice of segmentation methods for a particular intervention is determined by the method's robustness, segmentation accuracy, precision, and the extent of automation. For this reason, we have categorized the popular segmentation methods based on their utility. The categorization allows the clinicians to select appropriate methods for the most effective treatment pathways, subsequently aiding clinical decision making in complex diagnostic protocols, and in treatment planning. As a result, it is meant to be a supplementary aid aimed to improve clinical outcomes. Based on the BCLC staging classification (Figs. 3, 4) , we suggest segmentation Table 4 Segmentation methods for radiation therapy (RT)

Li et al. [108] Voxel-based Adaboost is used for liver localization. Shape and appearance models are employed to segment the liver, followed by free form deformation for refinement methods for liver transplantation, resection, and radiological intervention. Next, we sub-classify the papers based on their aim to address automation, volume estimation, and segmentation. The models presented by Alirr et al. [81] and Vorontsov et al. [83] use FCNs to achieve high accuracy and precision results for liver segmentation. Wang et al. [79] propose a two-dimensional U-Net CNN to estimate liver volumes in CT scans. Vivanti et al. [95] present an automatic method for liver and tumors segmentation using deep CNNs.

Over the past few decades, a significant number of articles have been published to assist the treatment of hepatic diseases (e.g., liver cancer) by proposing different methods for liver and tumor segmentation in CT and MRI scans. However, medical image segmentation of the liver still faces unaddressed challenges in technical and clinical settings. Figure 6 summarizes the clinical and technical challenges for HCC and liver segmentation algorithms, respectively. Some of the technical challenges are lack of standard benchmarks, reliance on conventional segmentation methods, and usage of firstgeneration deep learning architectures. Similarly, clinical challenges include shortage of compatible liver donors, lengthy diagnosis and treatment planning periods, limited understanding of tumor shapes and morphology.

Comparative studies are critical for discovering state-ofthe-art treatment methods for any disease. Many such studies aim to determine the ideal technique for liver segmentation but rely on private, or custom datasets [79, 80, 82, 83, 95] . The datasets often have built-in biases (e.g., subjects diagnosed with a particular disease) and do not cover more than a few hundred patients. Furthermore, the comparison of these works in literature is challenging due to their differing performance metrics. This variability in research due to custom datasets and varying performance metrics add to the challenge of evaluating different methods based on their claimed results, reinforcing the need to create benchmark datasets.

Benchmark datasets and their evaluation using standard metrics are crucial for fair quantitative comparison of the existing methods. It should be ensured that the datasets satisfy both technical and clinical standards. A reliable technical benchmark should contain samples from many patients (with minimal bias) and several slices per scan for each patient. From a clinical standpoint, multiple experts should medically validate and annotate datasets to account for subjective annotation. These guidelines would allow the creation of well-rounded datasets that could aid both technical and clinical challenges facing liver and tumor segmentation.

Robust and accurate segmentation methods are a necessity for clinicians performing liver transplants or resections [111] . After reviewing several works, we have observed that the conventional methods and first-generation neural networks (like plain U-nets and FCNs) have undesirable sensitivity to segment the liver and its tumors, high computation time, and over-segmentation. There has been a rapid development of neural networkbased frameworks such as the nnU-Net [13] that automatically configures preprocessing, network architecture, training, and post-processing for biomedical image segmentation, providing good out-of-the-box performance. However, the recent challenges (Like FLARE 21, KiTS 21, and CHAOS [15] ) for abdominal organ segmentation of CT and MRI scans have shown that custom CNN architectures [14] with dedicated modules designed for overcoming the pathological challenges outperform the nnU-Net framework. These shortcomings highlight the need for sophisticated deep learning models and dedicated network modules with optimizations to help reduce the networks' parameter count, memory footprint, and computation time. To improve the current state-of-theart methods, we propose exploring three avenues in supervised deep learning. First, we encourage employing transfer learning and larger benchmark datasets to tackle overfitting in neural networks. Second, using preprocessing on the datasets to enhance anatomical boundaries and contrast of the images. The preprocessing in CT images is critical due to image noise, poor contrast, and organs with overlapping boundaries. Researchers have utilized conventional denoising algorithms for enhancing CT images. However, new deep learning-based architectures that could simultaneously denoise and segment CT images will be an asset for clinicians to diagnose and treat HCC. Third, we suggest using standard baselines and recent models for comparison (e.g., nnU-Net), providing the community with meaningful results.

Evaluation metrics quantify the effectiveness of segmentation methods for different clinical scenarios. Volume-based metrics (i.e., DSC, IoU, VOE) and boundary-based metrics (i.e., Hausdorff distance (HD), ASSD, etc.) are used to evaluate the segmentation methods. Reinke et al. [112] summarize the pitfalls of the existing segmentation metrics by providing several ground-truth and prediction scenarios where an undesired prediction may receive an acceptable metric score. The authors highlight that DSC has high sensitivity when the target region of interest (ROI) has a size comparable to the pixel size. A similar sensitivity is observed for HD when the resolution of the ground truth varies. The authors highlight that a higher HD is obtained for low-resolution ROI. These findings are crucial for liver tumor segmentation in lowresolution CT images where the tumor is few pixels wide. The authors also show that over-segmentation results in higher DSC for a reference ground truth than under segmentation. Thus, a segmentation method trained to maximize DSC for a liver tumor may tend to over-segment, thereby increasing the chances of resection/ablation of healthy tissues during surgeries. Based on the above findings, we suggest to develop a more robust evaluation metric that effectively quantifies volume and boundary overlap across different ROI sizes/resolutions. Visualization tool functionalities and user interface may also impact the utility of the segmentation methods in clinical surgeries. Fischer et al. [113] propose a 3D presentation state (3DPR) for parameterizing and storing 3D images. The authors show that 3DPR minimizes user interactions and provides a storage efficient representation for 3D image visualization. One limitation of the approach is that it doesn't store the segmented 3D image data within the 3DPR object, requiring clinicians to deal with the 3D scan and its segmentation mask separately. Fischer et al. [114] also propose a system that incorporates multiple segmentation methods as plugins and renders the output as a single 3D image. This approach is clinically meaningful, because segmentation masks of different anatomical structures allow radiologists to understand the relationships between the neighboring organs and vessel trees. However, the system doesn't provide an option to manipulate and remove specific structures from the combined segmentation masks. This functionality is crucial, when the segmentation methods over/under-segments the target anatomical ROI. We recommend developing a visualization system that can incorporate the output of multiple segmentation methods, while providing functionalities to edit the combined segmentation mask. The ability to amend the segmentation masks can allow the radiologist to generate complex annotations for medical image analysis.

RFA has shown promising results in treating HCC and metastatic diseases such as colorectal cancer (CRC) [115] . Though surgical resection is the gold standard, RFA plays a crucial role in treating patients with inadequate residual liver functionality, multi-lobar lesions, extra-hepatic disease, proximity to prominent vascular structures and co-morbidities, making the patients ineligible for surgery, [116, 117] . However, RFA is associated with vascular and visceral damage, hemorrhagic complications, infections, biliary tract damage, liver failure, and local tumor relapse [118] .This limits the usage of RFA in practice, primarily due to the high chance of post-interventional complications. Furthermore, the clinical and technical challenges like insufficient ablation of tumors due to constraints of ablation needles, cooling of tissue by the adjacent blood vessels, large tumor mass, and tumors in the surroundings of heat-sensitive organs adds further complexity to the RFA procedure [118] . Nevertheless, we think that RFA will soon promote its clinical standing in treating advanced-stage liver tumors, primarily because of its potential to be used with multi-model imaging modalities.

The tumor location and the affected liver segment determine the nature of the radiological intervention and the prescribed segmentation method. It is recommended to use graph cut and gradient vector flow methods instead of active contour segmentation when the tumor is near the surface. This is due to the fact that the active contours can easily stream into the neighboring organs and cause over-segmentation, while gradient vector flow methods have demonstrated effective performance even for broken edges and subjective contours [21] . Furthermore, the liver segment location is vital in radiological interventions, because it can affect the dose limit in treatment planning [105] . For example, if the tumor is located in the caudate lobe, necessary safety precautions should be taken for positioning accuracy and quality assurance to avoid harming the gastrointestinal track [105] . A high degree of accuracy in target delineation and the use of image-guided radiotherapy (IGRT) can provide tighter margins that will minimize induced toxicity. Therefore, it is also recommended to segment the patient's liver according to the Couinaud classification to recognize the risks associated with every segment.

Recurrence of HCC after resection is a frequent postoperative occurrence [119] . The literature shows that there is a difference in opinion among clinicians on the precise chance for recurrence after transplant or partial resection. Some works claim that over 50-80% of patients following resection develop recurrences (over the first two years) [120, 121] , while others claim that only over 20% of patients are at the risk of developing tumor relapse [119] . Nevertheless, studies emphasize mandatory postoperative surveillance and regular screening [122] . Several key clinical indicators (like the presence of microscopic venous invasion, slow growth of small and inactive tumors) are signs of HCC recurrence [123] . It may be noted that the detection of tumor relapse also relies heavily on the postoperative imaging modality and segmentation tools. Thus, accurate and precise segmentation tools are required to detect minute lesions for early diagnosis of recurrent HCC. Models proposed by Alirr et al. [81] , Wang et al. [79] , and Vorontsov et al. [83] are well-suited for this task. Moreover, a combination of precise segmentation models with other clinical treatment methods like nucleic acid analogs and interferon (IFN) [124] can potentially become a robust curative option for HCC.

Though RFA, liver resection, and transplantation are effective remedying measures for treating HCC and most other hepatic diseases, they are not always used in isolation. Depending on the patient's clinical history and pathological context, clinicians sometimes may need to use a combination of surgical and radiological interventions to achieve optimal results. Targeting numerous pathways in the HCC cascade with a variety of treatments can help in accomplishing personalized care aimed to improve overall survival [125] . Clinicians hope that combination therapies would have higher treatment efficacy and efficiency. Some promising compound treatment methods blend direct cytotoxicity from chemotherapeutic agents and ischemia from selective embolization to cause tumor necrosis. In addition, embolization results in reducing washout and systemic chemotherapy toxicity [126] .

In addition to the patient's clinical history, the diagnosis of HCC also relies on multiple technical (such as the choice of contrast, imaging modality, etc.) and pathological (liver heterogeneity, liver diseases, tumor size, and intensity) factors [22] [23] [24] . Due to this, the optimum course of treatment depends on many contextual factors and differs from patient to patient. Currently, wellestablished clinical guidelines and protocols attempt to account for these contextual factors to assist treatment decisions. They oversee the detection of small lesions and the classification of benign (regenerative nodule) or premalignant nodules from HCC [25] .

This work aims to supplement the well-established clinical guidelines by suggesting varying segmentation methods appropriate for assisting diagnostic and therapeutic decision-making. However, we believe that our work still needs improvement: firstly, in order to manage the scope of our work, we have made the assumption that the segmentation methods are well-established and validated. Thus, we do not assess the pros and cons of each of the segmentation methods discussed, and focus instead on proposing a categorization schema of these segmentation methods with respect to their clinical utility. Secondly, in order to limit the scope of our work, we have not tried to address the existing challenges for the discussed segmentation methods and imaging modalities. It is expected that each segmentation method in combination with the imaging modality will have unique and patient-specific challenges due to heterogeneity in liver morphology and pathology. These challenges would need to be tackled as part of future work focusing on each segmentation method and procedure. Nevertheless, in this work, we have strived to establish a knowledge base of segmentation methods as an adjunct to an existing, well-established clinical decision-making process (i.e., outcomes of clinical protocols), thereby expediting the segmentation tool selection for treatment of HCC.

In future, we aim to assess the pros and cons of different segmentation methods for surgical and radiological interventions. Based on our assessment, we will design segmentation methods to overcome the limitations of existing methods and imaging modalities. Furthermore, we will explore segmentation methods for liver subsegmentation to improve the success rate of radiological interventions preventing any harm to the healthy portions of the liver. Finally, we aim to evaluate the effectiveness of combined surgical and radiological interventions.

This paper reviews state-of-the-art segmentation methods and categorizes them into three types of clinical intervention based on well-established clinical guidelines: transplantation, partial resection, and radiological interventions. This categorization is based on critical technical requirements or expectations from the algorithm to provide the best possible segmentation needed by the surgeon for a specific type of intervention. The broader aim is to contribute to optimal postinterventional outcome by aiding and supplementing the well-established yet complicated clinical diagnostic and therapeutic protocols. Considering each application (preoperative planning or image-guided intervention) have their own requirements in terms of accuracy and automation, we have also summarized the methods matching to their most appropriate corresponding clinical applications and types of intervention. We have observed that no single algorithm provides a 'one-sizefits-all' solution. Therefore, we believe that our work could help the clinicians in choosing the appropriate algorithms on a case-by-case basis, ensuring optimized healthcare outcomes.

Who report on cancer: setting priorities, investing wisely and providing care for all 2020

Liver segmentation from computed tomography scans: a survey and a new algorithm

Medical image segmentation methods, algorithms, and applications

Medical image segmentation algorithms using deformable models: a review

The liver tumor segmentation benchmark (lits)

Radiotherapy plus chemotherapy with or without surgical resection for stage III non-small-cell lung cancer: a phase III randomised controlled trial

Automatic liver segmentation by integrating fully convolutional networks into active contour models

Automatic liver segmentation using a statistical shape model with optimal surface detection

Automatic liver segmentation on volumetric CT images using supervoxel-based graph cuts

A survey of image segmentation techniques

Artificial intelligence in radiology

Comparison of semi-automatic and deep learning-based automatic methods for liver segmentation in living liver transplant donors

nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation

Efficient context-aware network for abdominal multi-organ segmentation

Chaos challenge-combined (CT-MR) healthy abdominal organ segmentation

Deep sequential segmentation of organs in volumetric medical scans

Covid-19 lung ct image segmentation using deep learning methods: U-net versus segnet

Miscnn: a framework for medical image segmentation with convolutional neural networks and deep learning

Grayscale medical image segmentation method based on 2d&3d object detection with deep learning

BSCN: bidirectional symmetric cascade network for retinal vessel segmentation

Overview of technical elements of liver segmentation

Infiltrative hepatocellular carcinoma: what radiologists need to know

EASL-EORTC clinical practice guidelines: management of hepatocellular carcinoma

Comments on management of hepatocellular carcinoma: an update. Zhonghua gan zang bing za zhi= Zhonghua ganzangbing zazhi= Chinese journal of hepatology

Differentiation of early hepatocellular carcinoma from benign hepatocellular nodules on gadoxetic acidenhanced MRI

Extending 2-d convolutional neural networks to 3-d for advancing deep learning cancer classification with application to mri liver tumor differentiation

An automatic diagnostic system for ct liver image classification

Googlenet-based ensemble fcnet classifier for focal liver lesion diagnosis

Use of chaos concept in medical image segmentation

Automatic left ventricular contour extraction from cardiac magnetic resonance images using cantilever beam and random walk approach

Performance divergence with data discrepancy: a review

Detection of left ventricular myocardial contours from ischemic cardiac mr images

Towards left ventricle segmentation from magnetic resonance images

Anatomic segmentation and volumetric calculations in nuclear magnetic resonance imaging

Random walks for image segmentation

Weighting function in random walk based left ventricle segmentation

Fuzzy Markovian segmentation in application of magnetic resonance images

Multi-spectral image analysis and classification of melanoma using fuzzy membership based partitions

Partitional clustering techniques for multispectral image segmentation

Multiresolution analysis using wavelet, ridgelet, and curvelet transforms for medical image segmentation

Survey on liver ct image segmentation methods

Tumor burden analysis on computed tomography by automated liver and tumor segmentation

Fully automatic segmentations of liver and hepatic tumors from 3-d computed tomography abdominal images: comparative evaluation of two automatic methods

ACM-based automatic liver segmentation from 3-d ct images by combining multiple atlases and improved mean-shift techniques

Automatic liver segmentation based on shape constraints and deformable graph cut in ct images

Atlas-based liver segmentation and hepatic fatfraction assessment for clinical trials

Liver segmentation from ct images using a sparse priori statistical shape model (sp-ssm)

Vascular active contour for vessel tree segmentation

Liver segmentation on CT and MR using Laplacian mesh optimization

An efficient and clinical-oriented 3d liver segmentation method

A likelihood and local constraint level set model for liver tumor segmentation from ct volumes

Liver segmentation with constrained convex variational model

Improved segmentation of low-contrast lesions using sigmoid edge model

Automatic hepatic tumor segmentation using composite hypotheses

Shape-constraint region growing for delineation of hepatic metastases on contrast-enhanced computed tomograph scans

3d multi-scale line filter for segmentation and visualization of curvilinear structures in medical images

Variational curve skeletons using gradient vector flow

Usability of semiautomatic segmentation algorithms for tumor volume determination

A review of vessel extraction techniques and algorithms

Segmentation of pathological features of rat bile duct carcinoma from hyperspectral images

U-net: Convolutional networks for biomedical image segmentation

Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks

A coarse-to-fine framework for the 2021 kidney and kidney tumor segmentation challenge

A concise review of updated guidelines regarding the management of hepatocellular carcinoma around the world

Prospective validation of the clip score: a new prognostic system for patients with cirrhosis and hepatocellular carcinoma

Development of hong kong liver cancer staging system with treatment stratification for patients with hepatocellular carcinoma

Natural history of hepatocellular carcinoma and prognosis in relation to treatment study of 850 patients

Prognosis of hepatocellular carcinoma: the bclc staging classification

Outcomes of liver transplantation in 490 patients with hepatocellular carcinoma: validation of a uniform staging after surgical treatment

Prognostic staging system for hepatocellular carcinoma (clip score): its value and limitations, and a proposal for a new staging system, the japan integrated staging score (jis score)

Liver resection in selected patients with metastatic breast cancer: a single-centre analysis and review of literature

Risk factors of poor prognosis and portal vein tumor thrombosis after curative resection of solitary hepatocellular carcinoma

Sustained complete response and complications rates after radiofrequency ablation of very early hepatocellular carcinoma in cirrhosis: Is resection still the treatment of choice?

Improved long-term survival after major resection for hepatocellular carcinoma: a multicenter analysis based on a new definition of major hepatectomy

Anatomic resection of liver segments 6-8 for hepatocellular carcinoma

Living donor liver transplant: The facts. UCSF Health

Review of liver segmentation and computer assisted detection/diagnosis methods in computed tomography

Automatic 3d liver location and segmentation via convolutional neural network and graph cut

Automated ct and mri liver segmentation and biometry using a generalized convolutional neural network

Automated hepatic volumetry for living related liver transplantation at multisection ct

Deep learning and level set approach for liver and tumor segmentation from ct scans

Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrastenhanced ct: a preliminary study

Deep learning for automated segmentation of liver lesions at ct in patients with colorectal cancer liver metastases

Surgical resection of colorectal carcinoma metastases to the liver: a prognostic scoring system to improve case selection, based on 1568 patients

Tumor burden assessment and its implication for a prognostic model in advanced diffuse large-cell lymphoma

The burden of liver disease in Europe: a review of available epidemiological data

H-denseunet: Hybrid densely connected unet for liver and tumor segmentation from ct volumes

Deep learning initialized and gradient enhanced level-set based segmentation for liver tumor from ct images

Cascade u-resnets for simultaneous liver and lesion segmentation

Liver tumor segmentation based on multiscale candidate generation and fractal residual network

Liver cancer detection using hybridized fully convolutional neural network based on deep learning framework

Inference with collaborative model for interactive tumor segmentation in medical image sequences

Ahcnet: An application of attention mechanism and hybrid connection for liver tumor segmentation in ct volumes

Modified u-net (munet) with incorporation of object-dependent high level features for improved liver and liver-tumor segmentation in ct images

Automatic liver tumor segmentation in follow-up ct studies using convolutional neural networks

Hepatocellular carcinoma and cirrhosis in 746 patients: long-term results of percutaneous ethanol injection

Interactive liver tumor segmentation from ct scans using support vector classification with watershed

Risk assessment of computer-aided diagnostic software for hepatic resection

Other non-surgical treatments for liver cancer

Other non-surgical treatments for liver cancer

Transarterial radioembolization for hepatocellular carcinoma: an update and perspectives

The current practice of transarterial chemoembolization for the treatment of hepatocellular carcinoma

Study of severe and rare complications of transarterial chemoembolization (tace) for liver cancer

Liver segmentation: indications, techniques and future directions

Stereotactic body radiation therapy in hepatocellular carcinoma: optimal treatment strategies based on liver segmentation and functional hepatic reserve

Liver segmentation: a primer for interventional radiologists

High-dose iodized oil transcatheter arterial chemoembolization for patients with large hepatocellular carcinoma

research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations maximum visibility for your research: over 100M website views per year submit your research ? Choose BMC

Adaptive mesh expansion model (amem) for liver segmentation from ct image

Hierarchical convolutional-deconvolutional neural networks for automatic liver and tumor segmentation

Reduced-risk drinking as a treatment goal: what clinicians need to know

Systematic parameterization, storage, and representation of volumetric dicom data

Integrating segmentation methods from different tools into a visualization program using an object-based plug-in interface

Radiofrequency ablation of liver tumors: actual limitations and potential solutions in the future

Which patients will benefit from percutaneous radiofrequency ablation of colorectal liver metastases? critically appraised topic

Radio-frequency ablation of colorectal liver metastases in 167 patients

Adverse effects of radiofrequency ablation of liver tumours in the netherlands

Surgical treatment of intra hepatic recurrence of hepatocellular carcinoma

How common is recurrence of hepatocellular carcinoma (HCC)?

Consensus and controversy in the management of hepatocellular carcinoma

Detection of recurrent hepatocellular carcinoma after surgical resection: non-contrast liver mr imaging with diffusion-weighted imaging versus gadoxetic acid-enhanced mr imaging

Risk factors, prevention, and management of postoperative recurrence after resection of hepatocellular carcinoma

Supportive therapies for prevention of hepatocellular carcinoma recurrence and preservation of liver function

Goals and targets for personalized therapy for HCC

Chemoembolization for hepatocellular carcinoma

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations