Recent advances in computerized imaging and its vital roles in liver disease diagnosis,preoperative planning,and interventional liver surgery: A review

Paramate Horkaew,Jirapa Chansangrat,Nattawut Keeratibharat,Doan Cong Le

Abstract The earliest and most accurate detection of the pathological manifestations of hepatic diseases ensures effective treatments and thus positive prognostic outcomes.In clinical settings,screening and determining the extent of a pathology are prominent factors in preparing remedial agents and administering appropriate therapeutic procedures.Moreover,in a patient undergoing liver resection,a realistic preoperative simulation of the subject-specific anatomy and physiology also plays a vital part in conducting initial assessments,making surgical decisions during the procedure,and anticipating postoperative results.Conventionally,various medical imaging modalities,e.g.,computed tomography,magnetic resonance imaging,and positron emission tomography,have been employed to assist in these tasks.In fact,several standardized procedures,such as lesion detection and liver segmentation,are also incorporated into prominent commercial software packages.Thus far,most integrated software as a medical device typically involves tedious interactions from the physician,such as manual delineation and empirical adjustments,as per a given patient.With the rapid progress in digital health approaches,especially medical image analysis,a wide range of computer algorithms have been proposed to facilitate those procedures.They include pattern recognition of a liver,its periphery,and lesion,as well as pre-and postoperative simulations.Prior to clinical adoption,however,software must conform to regulatory requirements set by the governing agency,for instance,valid clinical association and analytical and clinical validation.Therefore,this paper provides a detailed account and discussion of the state-ofthe-art methods for liver image analyses,visualization,and simulation in the literature.Emphasis is placed upon their concepts,algorithmic classifications,merits,limitations,clinical considerations,and future research trends.

Key Words: Computer aided diagnosis;Medical image analysis;Pattern recognition;Artificial intelligence;Surgical simulation;Liver surgery

INTRODUCTION

It is estimated that there are 20 million new cancer cases worldwide and 10 million cancer-related deaths every year[1].Among these cases,liver cancer is the third leading cause of cancer death.In 2020,905700 people globally were diagnosed with liver cancer,and 830200 people died from the disease.Scientists have estimated that in 2040,approximately 1.4 million people will be diagnosed with the disease,while 1.3 million people will die from it[2].

The vital function of the liver is filtering blood flow from the digestive tract before circulating the blood back to the rest of the body.Consequently,the liver is subject to various diseases,e.g.,fascioliasis,cirrhosis,hepatitis,and alcoholic liver disease[3].In particular,cancer is associated with increases in both the number and size of abnormal cells.If diagnosed early,it can be treated by interventional radiology,chemotherapy,radiation therapy,or a combination thereof.Among these treatments,liver surgery removing the tumors is efficient in preventing their recurrence and prolonging the life expectancy of the patient,especially those in primary and secondary stages[4].Liver surgery is a complex and challenging procedure that requires comprehensive knowledge of the liver anatomy,blood supply,and tumor locations and characteristics.Consequently,preoperative imaging is necessary for its planning.

In recent years,there have been significant advancements in diagnostic and interventional imaging technologies,including the use of software equipped with artificial intelligence (AI),to enhance the accuracy of preoperative imaging.This comprehensive review,therefore,aims to offer an in-depth exploration of the latest progress and applications of imaging techniques.In particular,it highlights their pivotal role in improving the outcomes of diagnoses,preoperative planning,and interventional liver surgeries.The main topics discussed in the remainder of this paper cover liver segmentation,diagnostic imaging,preoperative planning and simulation,surgical and therapeutic intervention,and finally software as a medical device.

LIVER SEGMENTATION

The functional anatomy of a liver is considered in terms of its dual blood supply,as well as its venous and biliary drainage systems.It is divided into four sectors by the three hepatic veins,each of which drains into the inferior vena cava (IVC) and runs within its scissurae.This nomenclature system was famously described by Couinaud in 1957 and later amended at the Brisbane meeting in 2000.Its primary advantage is enabling anatomical resection of this seemingly almost asymmetrical organ.With this classification,each liver subdivision is self-contained in its artery and portal venous supply and biliary drainage[5,6].

Couinaud scheme

According to the Couinaud classification[5],the liver is divided into eight functionally independent segments,each of which has its own vascular in-and outflows,as well as biliary drainage (Figure 1).There are three major planes that divide the liver vertically: The right hepatic vein plane divides the right hepatic lobe into anterior and posterior segments;the middle hepatic vein divides the liver into right and left lobes;and the umbilical plane running from the falciform ligament to the IVC divides the left lobe into medial and lateral parts.The portal vein divides the liver into superior and inferior segments.

Figure 1 Segmental anatomy according to the Couinaud classification. RHV: Right hepatic vein;MHV: Middle hepatic vein;LHV: Left hepatic vein;MPV: Main portal vein;IVC: Interior vena cava.

Image analyses of the liver

Diagnosis,treatment,and prognosis of diseases of the liver involve defining its anatomical boundary as well as characterizing its pathologies,typically from computed tomography (CT) and magnetic resonance (MR) images.On an image plane,a liver is described by closed contours separating itself from the background.Conventionally,physicians must mentally reconstruct the whole liver and relevant structure in 3-dimension (3D) while navigating through its tomographic planes.However,recent advances in computing technology have enabled virtual 3D reconstruction,modeling,and simulation of the organinvivo[7,8].

With these technologies,physicians can accurately calculate the hepatic volumetry.In addition,visualization of a 3D liver can also help locate its arteries,veins,and biliary tracts and,hence,determine its functional segments.In surgical planning,the spatial relationship between tumors and hepatic vasculature with a 3D model increases the precision of proposal resection over that with a 2D counterpart by up to 31%[9].Last but not least,the use of such a model also reduces time and strain during surgical planning and intervention[10].

Modeling a liver from medical images first involves delineating its boundary from other connective tissues and adjacent organs[11].One of the key challenges is that,as a complex organ,the liver comprises not only its parenchyma but also an extensive vascular network,as well as lesions in pathological cases.In addition,despite recent advances in tomographic imaging[12],liver images remain contaminated with noise of various distributions,depending on the modality.These problems call for the development of effective preconditioning and robust image analysis algorithms.This section,therefore,investigates state-of-the-art methods,as well as their features,limitations,and challenges.These approaches include data preprocessing and fully automatic and semiautomatic segmentation methods.

Data preprocessing

Medical images are often degraded by noise and artifacts during acquisition.Depending on their model assumptions,various noise reduction strategies are applied prior to image analyses[13].The perturbation function due to noise is normally random and hence unknown,except for only their distribution.Therefore,in medical imaging,the most frequently assumed distribution models include Gaussian,Poisson,and Rician distributions for charge couple device,Xray,CT,and MR images,respectively.However,directly applying an inverse filter to reduce noise and possibly other inherent artifacts could adversely affect smaller features or abate anatomical boundaries,such as vasculature,calcification,and connective tissue.Instead,several applications adopt structural adaptive anisotropic[14],spatial frequency or wavelet[15],blind deconvolution[16],regularized diffusion[17] filters or,much more recently,those based on machine learning (ML) or AI models,e.g.,convolutional neural networks (CNNs)[18,19].

On measuring their performance,the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) are often considered.However,a recent study[20] revealed that although visual quality is clearly improved with advanced filters,correlations between PSNR and SSIM and application-specific performance,such as classification (i.e.,based on area under the curve),are not clearly present.In fact,fine tuning neural network parameters to a particular noise model is recommended.

In practice,to balance complexity and intended analysis,a trivial anisotropic diffusion filter has been applied to denoise a T1-weighted MR image of a liver while enhancing its border[21] prior to 3D surface generation.Meanwhile,in some other studies,the least commitment principle[22] has been adopted with no preprocessing of an image other than adjusting its windows and levels,but taking noise into account during subsequent analyses[23].Once preconditioned,a series of cross-sectional images proceeds to the next stage,in which the liver,its peripherals,and lesions are separated.

Fully automatic segmentation

Since liver pixels appear very similar to those of other nearby organs,existing automatic schemes thus rely on auxiliary information,e.g.,data-driven appearance models or empirical understanding of its morphology[23].Accordingly,its main advantage is low inter-and intraobserver variability due to manual intervention.Recent extensive surveys on the topic are found in the literature[24,25].Several early works were developed and validated based on a public dataset called SLIVER07[26].They contained 3D CT liver images of 20 and 10 subjects for training and testing,respectively.These images had 512 × 512 pixels at 0.56 to 0.86 mm2in-plane resolution and covered 64 to 502 slices with spacing between 0.7 mm and 5 mm.Another recent dataset is 3D-IRCADb[27].It contains CT images of 20 patients,three thirds of which contain hepatic tumors.Their voxel size and resolution are similar to those of its predecessor.Furthermore,each image is labeled with not only pathologies but also segmentation challenges.Based on these datasets,a number of computerized methods were proposed to delineate a liver and benchmarked.Examples of the recent results are summarized in Table 1[28-34].Note that these studies may employ different accuracy metrics,e.g.,volumetric overlap error or dice similarity.

Table 1 Selected fully automatic liver segmentation algorithms[28]

The most promising approach in this category is one based on modeling from pretrained data,both statistically[26,30,33,35-37] and using CNNs[34,38-40].The former iteratively deforms a liver model to fit underlying imaging features while imposing anatomically plausible constraints found in the training,e.g.,the active appearance model.The latter learns from some segmented livers,their spatial architecture,and the relationship among their convolutional features,cascaded through a deep network,and fuses them with weighted nonlinear functions.Subsequently,pixels of an unseen image proceed through the same network and are labeled accordingly,resulting in the final segmentation.However,these methods require sufficiently large prelabeled samples and hence a substantial amount of computing power for model learning.

Semiautomatic segmentation

It is evident that various factors,e.g.,nearby organs such as the stomach,pancreas,duodenum,and heart,as well as artifacts due to implants,could have adverse effects on segmentation quality.Currently,it remains challenging to incorporate computable elements to automatically address this issue.Therefore,user interaction is often involved but kept to a minimum,i.e.,at initialization[23,41],during the process[42,43],in final adjustment[23],or with a combination thereof.The recent works are summarized in Table 2[21,28,41,43-46].

Table 2 Selected semiautomatic liver segmentation algorithms[28]

At varying degrees of interaction,many methods can achieve reasonable accuracy without previously trained liver data.For instance,trivial thresholding with K-means clustering has been applied to CT angiography to separate the liver from the kidneys and ribs[47].Similar methods automatically set these thresholds by learning the pattern of abdominal histograms[48,49] or that of textures[50].However,they often require prior knowledge of the anatomy for initialization and postprocessing,e.g.,manual editing or morphological operators,to remove oversegmented regions.Instead of defining pixel membership by thresholds,many researchers have expanded a region of interest accumulatively from seeding points.They then have exploited different strategies to control new inclusions,e.g.,convex hulls[51],binary morphology and anatomical constraints[52],significant differences in boundaries[44,53],and anatomical priors[42].

In addition to region-based approaches,delineating contours around a liver has also attracted considerable interest.Initially,the active contour model and its variants were explored[54,55] based on gradient and curvature and later extended to the level set[43,56-59].By these methods,starting contours were specified by a user or estimated by other segmentations.They were then implicitly driven by gradient and embedding surface curvature.Unlike its counterparts,any aberration would be regularized by geometric continuity on the hypersurface.Empirical and anatomical knowledge,e.g.,distance to centroid,nominal contrast,and segmental and anatomical markers,were translated to computable conditions to assert the evolution of these contours.

Another approach poses the segmentation problem as that of graph optimization[60].In liver imaging,the deviation of intensity at a pixel from a predefined distribution and its gradient strength are formulated as region and boundary cost functions,respectively[61].Similar functions are obtained from texture images and supervoxels[32] or constrained by statistically trained shape[45] and intensity[32] models or initialized by CNN[46].It was shown that automation of the remaining steps was possible if anatomical constraints (e.g.,vena cava and tumor) were imposed for postprocessing[62].

DIAGNOSTIC IMAGING

It has been discussed previously that although the liver and its vasculature are clearly presented to a radiologist in medical images,mentally extracting them requires substantial knowledge and expertise regarding hepatic anatomy and physiology.Therefore,automating this process with algorithmic codes remains an open research challenge.Similarly,diagnosing a hepatic disease involves assessing liver damage and characterizing its lesions based on their vascularity and composition,as well as their implications on adjacent vessels[63,64].A range of radiological and computerized imaging techniques can be utilized and are summarized as follows.

Anatomy of the vasculature

Arterial anatomy:Hepatobiliary surgery,liver transplantation procedures,and endovascular treatments can all benefit from information on anatomical variations in hepatic arteries.The existence of such variations may call for adjustment of surgical procedures to avoid unintentional vascular damage,hemorrhage,and biliary problems.

The Michels classification[65] and its modification by Hiattetal[66] are the most frequently used categories for describing hepatic arterial variations in the literature.Approximately 55%-60% of people have the classic pattern of the common hepatic artery branching from the celiac artery,with the hepatic artery normally splitting off into the right andleft hepatic arteries to supply the entire liver.Replaced and accessory left or right hepatic arteries are the most often found anatomical variants.In cases of transarterial embolization of traumatic liver injury or embolization of liver tumors,if the bleeding point or arterial feeders cannot be demonstrated on conventional hepatic angiogram,searching for these possible anatomical variants is crucial.Another example is when left hepatectomy is performed in a patient with a replaced or accessory left hepatic artery,ligation of the left hepatic artery at its origin in the left gastric artery is needed.

Although the anatomical variant classification has been widely accepted,not all variants are surgically significant.Furthermore,the course of the hepatic artery and its topographic relationship to the surrounding structures,such as the portal vein and bile ducts,are not taken into consideration[67].

Portal vein anatomy:There are numerous variations in the portal vein branching patterns.The classic anatomy,which is found in approximately 65% of patients,consists of the main portal vein branching into the right and left portal veins at the porta hepatis.The right portal vein later subdivides into anterior and posterior branches.Found in approximately 35% of patients,the two most common variants are trifurcation of the portal vein trunk and a right posterior branch as the first branch of the portal vein trunk,with the latter being more common and known as the Z-type pattern[68,69].

Hepatic vein anatomy:Accurate perception of the hepatic vein anatomy before liver surgery is crucial.Inadvertent injury of the hepatic veins leads to a higher risk of bleeding and functional loss of the hepatic segment with a compromised venous outflow.Generally,there are three hepatic veins: The right hepatic vein drains segments V,VI,and VII,the middle hepatic vein drains segments IV,V,and VIII,and the left hepatic vein drains segments II and III.The classic anatomy of the hepatic veins,which form a common trunk between the left and middle hepatic veins,is found in approximately 65%-85% of patients[70,71].

Biliary anatomy:The normal biliary anatomy found in approximately 58% of the population comprises the right hepatic duct draining the right hepatic lobe and the left hepatic duct draining the left hepatic lobe.The right hepatic duct divides into the right posterior sectional duct,coursing in a horizontal plane and draining segments VI and VII;the right anterior sectional duct,coursing in a vertical plane,drains segments V and VIII.The left hepatic duct divides into the left superior sectional duct and drains segment IVa,and the left inferior sectional duct drains segments II,III,and IVb[72].The caudate lobe usually drains into the proximal left or right hepatic duct.

Preoperative evaluation of liver tumors

Generally,20%-30% of patients have synchronous hepatic disease,while hepatic metastasis occurs in more than 50% of colorectal cancer patients[73].Primary liver tumors,such as hepatocellular carcinoma (HCC),mass-forming cholangiocarcinoma,hepatic adenoma,or focal nodular hyperplasia,as well as liver metastasis,have distinct cellular components and,hence,unique imaging appearances.As such,they can be characterized by means of CT and MRI.

To date,the only treatment associated with long-term survival for both HCC and colorectal liver metastasis is surgical resection.Imaging studies are essential for identifying potential surgical candidates.Specifically,for the best outcome in the resection procedure,all lesions need to be removed,while a sufficient functioning liver must be preserved.One of the major challenges is that because metachronous hepatic metastasis can occur in over 50% of patients with colorectal cancer[74,75],the imaging sensitivity should be sufficiently high to detect these lesions.Although CT is available worldwide and enables evaluation of extrahepatic disease and vascular structures,the modality has some limitations.These include an inferior ability to delineate the tumor margin,to perform tissue characterization,and to detect and characterize small lesions and associated radiation.Alternatively,MRI with hepatocyte-specific agents is currently the most accurate imaging modality to identify hepatic disease in patients with colorectal cancer[76-78].Despite its sensitivity,additional metastatic foci can be found intraoperatively in up to 25% of patients after MRI[79,80].Another drawback of MRI is that in patients with coexisting benign focal liver lesions,such as hemangioma,an ill-defined heterogeneous echoic nodule could lead to confusion during surgery.To resolve the ambiguity,contrast-enhanced ultrasound has increasingly been adopted intraoperatively as a complement.Table 3 summarizes the existing research related to the sensitivity of focal liver tumor detection[63,78,81-83].

Computer-aided diagnosis

The anatomy of both the liver and its peripherals has been extensively explored in the medical literature,and the most common patterns have been firmly established.Despite the highly deformable structure and large intersubject variability of this organ,it has been continually demonstrated that computerized methods can be applied to extract relevant objects with reasonable degrees of accuracy[24,26].Thus far,pathological manifestations can result in irregular appearances of the interconnecting parts,undermining their merits in clinical and surgical practice[4].In fact,with the recent advances in ML and AI,research focus has now been particularly directed toward identifying,delineating,and characterizing lesions from tomographic images.Prominent works in the field are summarized and discussed here.

ML algorithms have been widely employed in segmenting the tumoral liver.After a seed point was estimated within a lesion,fuzzy C-means (FCM) was used to expand the coverage toward its margin[29].Likewise,a watershed was applied to CT images to extract supervoxels with similar characteristics.Subsequently,tumors were identified from the liver and other objects by merging those subregions with FCM and K-means clustering using their textural information[50],i.e.,pixel intensities,directional derivative,local binary pattern,and local differences.Based on 22 trained and 22 tested instances,the highest classification accuracies of 95.64% to 98.88% were reported.K-means clustering was applied to approximate liver contours,which were later refined by Graph-Cut[62].Once the vena cava had been detected and other segments had been discarded by anatomical templates,tumors were extracted by cavity filling.Note,however,that this assumption failed to identify those on the liver boundary.For percutaneous radiofrequency ablation (RFA) to remove inoperable primary or metastatic tumors,the ablation zone was first determined by max-flow min-cuts of a 3D spherical graph expanded from a seed point[84].It was later automated by using FCM to extract the ablation zone and then cyclic morphology to refine one[85].

Meanwhile,with rapid development in AI and CNNs in particular,a number of network architectures have been adopted for diagnosing tumoral livers.Lietal[38] used 2D and 3D DenseUNets to extract within-slice features and to learn their spatial relationships between slices,respectively.The results from both networks were finally fused to produce labels of both liver and tumor pixels.Despite relatively high benchmark scores (i.e.,93.7%),their models took 30 h in total for training with only limited series.Another example[86] applied a simple 3D U-Net to first extract the liver.Super pixel blocks of tumors were localized by a multiscale candidate generation method.The exact regions of these candidates were defined and then refined by a 3D fractal residual network and active contour,which reached a 67% accuracy during evaluation.It was pointed out in another study that a main drawback of data-driven AI is the imbalanced proportion between healthy livers and those with pathologies[87].As a result,many existing dice loss (DL) models tend to predict lesions as part of the liver or backgrounds.To address this issue,they tried to assemble cascade U-ResNets,each trained with a different loss function,i.e.,weighted cross entropy,DL,weighted DL,Teverskry loss (TL),and weighted TL.With ensemble learning,tumors could be segmented with a 75% accuracy,compared to the approximately 65%-70% accuracy obtained by competing networks.The same accuracy of 74.5% was achieved by a 2.5D fully CNN whose loss function consisted of cross-entropy,a similarity coefficient,and a novel boundary loss function[88].The latter was prescribed based on the boundary between segmented objects by means of logical morphology.Alternatively,using a two-stage densely connected network,where a liver was first localized by an encoder-decoder CNN,tumors were detected with attention modules at a 72.5% accuracy[89].

Since these methods recognize tumors by their features,implicitly learned by examples,the irregularities found on the boundaries of lesions are not precisely traced,while adjacent lesions are sometimes merged.Thus,postprocessing by another empirical model or manual processing by radiologists is often needed.In fact,some studies have shown that,as a baseline,even skilled human raters could achieve only a 78% accuracy.

PREOPERATIVE PLANNING AND SIMULATION

Conducting a virtual liver resection prior to the live procedure is highly beneficial.For example,the right portal pedicle divides into anterior and posterior branches,each of which further splits into two segments (i.e.,V and VIII,and VI and VII,respectively).The left portal pedicle has a longer and more horizontal extrahepatic course.This allows the surgeon access and exposure to the relevant areas,for instance,during biliary system reconstruction.Moreover,segmental branches arise from the left portal pedicle supplying segments II,III,and IV.The ligamentum venosum,a fibrous remnant of the ductus venosum in the fetus,which connects the left portal vein to the left hepatic vein at the IVC,serves as a distinctive landmark for gaining access to the left portal pedicle and the left hepatic vein,whose terminating discharge is often merged with the middle hepatic vein and thus can be challenging to identify and control during a left hepatectomy.

Upon estimating potential risks of the surgical procedure,liver cirrhosis and portal hypertension (which are usually associated with liver cancer,particularly HCC),must be diagnosed preoperatively.To this end,multidisciplinary teams consider any evidence of advanced cirrhosis or inadequate liver function while devising an appropriate management plan,e.g.,by using the Child-Pugh scoring system[90,91].Specifically,for patients having cirrhosis and meeting certain criteria,liver resection and transplantation address an underlying field change that predisposes the parenchyma to tumor recurrence.Options for patients who are not candidates for those procedures include RFA,microwave ablation,transarterial chemoembolization,and other locoregional therapies[92].

Liver volumetry and future liver remnant

Surgical resection is a primary curative treatment for patients with primary and metastatic liver tumors.Unfortunately,fewer than 25% of patients are suitable for surgery[93,94].With a better understanding of hepatic anatomy and surgical technique refinements,the extent of liver sections that can be surgically removed is expanding.However,a tendency toward more aggressive liver resections in patients with preexisting liver disease requires thorough evaluation of hepatic function,especially the amount and quality of the postoperative future liver remnant (FLR).

It has been established that an inadequate liver volume following surgery is a robust,independent predictor of postoperative hepatic dysfunction and complications[95].Generally,the FLR per total liver volume (TLV) ratio must be 25%-30% to minimize postoperative complications[96,97].Patients with hepatotoxic chemotherapy or hepatic steatosis should have an FLR ratio of greater than 30%,whereas those with cirrhosis should maintain an above 40% ratio.Likewise,in living donor transplantation,the donor’s liver volume has to be 30%-35% more than that of the recipient[98],or 40% in cases with hepatic disease[99].

Computerized imaging for FLR

Volumetric CT has currently become the gold standard for determining whether hepatectomy can be performed[100].To this end,computer software is employed to reconstruct a 3D liver and estimate the ratio of FLR to nontumorous TLV.Normally,the latter is measured directly by CT.Alternatively,it may be estimated from the patient’s body surface area[101].These methods are called mTLV and eTLV,respectively.It was found in some studies[4,102] that eTLV could identify cases where mTLV was previously underestimated.In addition,there have been a number of recent advancements in automated liver volumetry by medical image computing.

Unlike image segmentation,where whole liver boundaries are traced on an underlying volumetric image,functional segmentation or resection involves estimating its composition of independently functional segments,each of which has its own vascular in-and outflow,according to Couinaud’s scheme.MeVis LiverAnalyzer? (MeVis Medical Solution,Germany) and Synapse Vincent? (Fujifilm,Japan) are currently standard software programs for virtual liver resection that rely on the surgeon’s judgment[103].Common practice involves the user’s interactive tracing of the segments with respect to their major vascular tracks.Not only are individual experiences and skills needed,but the process is also tedious and time-consuming,as well as inducing large inter-and intraobserver variability.Meanwhile,many novel imaging algorithms have also been continuously developed to assist or complement this task and have rapidly become an emerging area of investigation.

Provided that portal and hepatic veins are extracted,liver segments are defined with respect to voxel distances to specific branches[104,105],voxel projections onto vascular intersections[106],or categorical search by Voronoi diagram[107-109].However,these methods suffer from computationally intensive voxel sorting.Moreover,the topology of voxel aggregation is neither validated nor rectified.Alternatively,to accelerate computation and ensure surgically plausible resection,an extracted liver volume is first converted to a surface and subdivided,based on vasculature and salient anatomical landmarks,by differential geometry[110-115].Thus far,manual correction is often inevitable.Otherwise,additional anatomical constraints,e.g.,from a statistically trained deformable atlas[116],are necessary.

Unfortunately,not every method was able to segment all eight Couinaud segments nor was it always validated on the same liver dataset.Thus,a recent study[115] compared some prominent algorithms only according to their volumetric ratios,i.e.,at lobe and sectional levels.The average values are listed in Table 4[11,106,108,112,113,115].Since these methods rely on accurate extraction of the hepatic vasculature and liver boundary,future directions worth exploring are advanced pattern recognition (PR) of the gastrointestinal structures and integration of other imaging modalities.

Table 4 Average proportion (in percent) of functional segment groups[115]

Once the resection is made,removal of pathological segments could be planned,and FLR could hence be estimated.One of the most widely utilized 3D software programs in preoperative liver surgery is Synapse Vincent? (Fujifilm,Japan)[103].It helps automate liver segmentations and their volumetric assessment.However,with recent surgical techniques,liver resection is no longer limited to only right hepatectomy.Several surgical plans have been devised or tailored for an individual,i.e.,patient-specific strategies.Therefore,FLR should be resilient to variations in such planning.In addition,other volumetric assessments are also equally important posttherapy,e.g.,graft regeneration after transplantation and responses to cancer treatment[117].

Liver function

In addition to resected volumetry,liver function also needs to be evaluated.In fact,technical limitations of resection and its safety have been exceeded by continually developed procedures,aiming at increasing FLR in patients with insufficient liver volume by utilizing its regeneration in response to blood flow,also known as flow modulation.Among the most often chosen procedures is portal vein embolization (PVE),where the portal vein on the opposite side of the FLR has a catheter radiologically inserted and is then embolized with vascular plugs,coils,particles,or glue[117].Consequences of a diseased liver parenchyma in terms of liver function may be analyzed by biopsy,performed on the living donor liver prior to transplantation[118].In patients who may need PVE to enhance the FLR ratio,these anatomical and functional criteria are also relevant,given the proper context.Comprehensive assessment of liver conditions is required prior to any therapy because livers with such as cirrhosis and steatosis,for instance,demand a significantly greater FLR than a healthy liver[119].

The indocyanine green (ICG) clearance test,asialoglycoprotein receptor scintigraphy using 99mTc-galactosyl human serum albumin,and serum hyaluronic acid level assessment are prominent methods that can be used to evaluate the residual liver's ability to function[120,121].Injection of ICG,a tricarbocyanine dye,causes it to bond with albumin and be carried throughout the bodyviathe circulatory system.Elimination of ICG occurs solely through biliary excretion.Thus,serum blood tests or an optical sensor on the finger can reveal the excretion level.ICG levels in the blood should be below 10% at 15 min after injection (ICG-R15).Therefore,blood samples taken at 5-min intervals postinjection can be analyzed to determine the plasma disappearance rate of ICG (ICGK),which is calculated by using linear regression of the plasma ICG concentration[122].

Although there is accurate automated anatomical liver volumetry and identification of its biomarkers,there is currently no computer software with biomarker mapping that can precisely delineate the area of residual functioning liver.A multidisciplinary approach is therefore recommended to determine both the liver volume and function.The selection of the surgical plane,feasibility of resection,visualization of the tumor and its extent,etc.,all rely on maintaining an ongoing interaction between the surgeon and radiologist,as well as reliable,though probably not the most precise,imaging software.

Postoperative risk assessment

The risk of postoperative hepatic failure is substantially associated with the extent of liver resection.Although this is logical and simple to assess,the volume of the liver that remains present is more indicative and must be precisely determined.Additionally,because the segmental anatomy and its volume significantly vary among patients,only determining the segmental numbers is inconclusive.Specifically,the right side of the liver accounts for more than half of the TLV in most people,but its variations extend from 49% to 82%,while those of the left side range from 17% to 49%[123].Therefore,a formal radiologic volumetric assessment is necessary to ensure accurate FLR,especially when planning a major liver resection.

SURGICAL AND THERAPEUTIC INTERVENTION

CT or MRI is typically performed to characterize lesions and devise preoperative planning[21].Consequently,FLR is estimated from the planned resection outlines by sequential marking on respective cross-sectional images,given slice thickness and voxel dimensions.It has been shown that both the intended and actual FLR as well as their actual surface and volumes are highly correlated radiographically[124,125].As such,additional tests are not required during the procedure.

For patients with an insufficient FLR who are being considered for hepatic resection,FLR augmentation by PVEviainterventional radiography is most widely used.With this procedure,the portal vein (with or without segment IV branches) is embolized.Usually,the procedure is performed with percutaneous vascular access[117].Subsequently,those who exhibit more than 2.0% growth per week on repeated volumetry have no risk of liver failure during the periop-erative phase following hepatectomy[126].

Liver resection

An anatomic resection involves the removal of a Couinaud segment by selective ligation of the main HPV and portal triad.With this approach,there is a higher chance of obtaining disease-free margins because it resects areas distal to the tumor that are at risk for vascular micrometastasis.Alternatively,nonanatomic resection or parenchymal transection disregards those segmental planes;it is often employed for benign tumors,debulking treatment,or when attempting to preserve the residual parenchyma.A microscopic margin negative (R0) resection must be performed to minimize local recurrence.It has been shown that a resection margin of 1 cm or smaller is safe[127].

Standard anatomic hepatectomy involves controlling both vascular inflow and outflow prior to parenchymal transection.Accordingly,removal can be performed without affecting adjacent hepatic segments.Generally,intraoperative ultrasonography is utilized to determine the presence of vascular structures and to assess the location and size of the tumor as well as their relation to the surrounding vasculature.

Minimally invasive surgery

The development of computerized imaging techniques to further enhance minimally invasive liver surgery has rapidly progressed[128,129].For instance,near-infrared fluorescence is adopted in laparoscopic and robotic camera systems,allowing the identification of different preoperatively injected dyes (e.g.,indocyanine green).This contrast agent propagates through the biliary tree while illuminating the structure after being metabolized mostly by hepatocytes.Recently,this modality has been exploited to differentiate between well-and inadequately-perfused hepatic parenchyma to guide parenchymal dissection following vascular control[130-132].

Computer-assisted surgery

It has been established that computer-based 3D reconstruction of liver tumors could improve the accuracy of their localization and the precision of surgical planning[9].Thus far,2D/3D image reviewing during surgery on a traditional picture archiving and communication system in the operating room has been found to be distracting.Therefore,real-time localization of lesions and the identification of arteries and biliary structures by using intraoperative ultrasonography are usually preferred.Nevertheless,similar to what was pointed out in another survey on tumor surgery[133],the need for additional port sites to interpret 2D images and hence to mentally recreate the 3D anatomy with respect to the orientations of ultrasound probes has restricted its wider adoption in minimally invasive surgery.

Currently,an augmented reality (AR) endoscopic overlay of the patient-specific anatomy with associated virtual reality(VR) models has attracted considerable attention as it could increase the surgical efficiency in real-time with intelligent operative guidance[134-136].With this approach,3D reconstructed data can be precisely overlaid onto the operated area.Effectively,cognitive strain conventionally imposed on the surgeon could be lessened.For uterine myomectomy,it has been shown that spatial recognition based on AR could improve the localization accuracy[137].

To adopt VR and AR in hepatobiliary surgery,one has to confront the technical challenges of continuously coregistering the computer-generated models to a mobile liver with significant tissue deformation.To address this issue,a recent study[28],for example,applied conformal parameterization to an extracted liver surface.With this technique,the triangle mesh of genus-0 of the surface was mapped onto its topological homeomorphism[138].Given a set of landmarks on a liver surface,representing the resection paths according to Couinaud’s definition,a deformation that bijectively maps a liver and its section onto another instance with minimal distortion could be realized.However,since the liver is morphologically diverse,it was suggested that localized alignment should be the focus.In fact,to ensure physiologically plausible correspondence within or across subjects,statistical deformable models[30,43] are incorporated.Additionally,clinical management aspects,e.g.,tumor board evaluations,preoperative strategy,and intraoperative access,also need to be considered.

SOFTWARE AS A MEDICAL DEVICE

Since the early 1990s,Digital Imaging and Communication in Medicine (DICOM) developed by the American College of Radiology and the National Electrical Manufacturers Association has been a gold standard for archiving,transferring,and presenting imaging data among acquiring and processing devices[139] in radiological practices[140].It features a unique structural content[141],consisting of not only an imaging matrix and its encoding but also relevant medical data,e.g.,patient information and study details,as well as a scanning protocol.In a typical liver examination,for instance,its DICOM structure contains a series of multislice CT images in the axial direction,covering the upper abdomen,and each image is numbered and labeled with physical geometry,thickness,and resolution,and perhaps suggested window-level settings.This information is vital for accurately reconstructing a whole 3D liver for diagnostic and intervention purposes.Hence,most current medical image computing software does support this standard by default.In addition,the Neuroimaging Informatics Technology Initiative (NIfTI)[142] has been specifically designed and developed by neuroimaging scientists to resolve physical orientation objects within a brain image.Nonetheless,this data format is also adopted in other fields,where geometric information is needed.

Presently,computer software has increasingly been integrated into digital platforms that serve medical purposes.Software that is a medical device in its own right is called Software as a Medical Device (SaMD)[143].It is to be distinguished from software in a medical device and that used in manufacturing or maintaining a medical device.Specifically,the International Medical Device Regulators Forum (IMDRF) defines SaMD as “software intended to be used for one or more medical purposes (e.g.,to make clinical decisions) that performs these purposes without being part of a hardware medical device.” With its unique features,a working group by IMDRF as the representative of regulators worldwide developed a common framework aiming to support innovation and timely access by both patients and providers to safe and effective SaMD.

In liver imaging,SaMD,regardless of its computing platforms,may be used for diagnoses bothinvivoandinvitro,prevention,screening or monitoring,and treatment or alleviation of liver diseases.A manufacturer who intends to make SaMD available for use under their name would be subject to regulations,not only throughout its software engineering life cycle (e.g.,ISO/IEC 14764:2006 Software Engineering) but also postmarket surveillance and any subsequent updates,in which risk identification and countermeasures are established[144].

To maintain regulatory compliance,the roles of an SaMD and its deployment in clinical environments must be declared.Its recommendation for intended uses (i.e.,diagnosing or treating a disease and informing or driving clinical management) with potential adverse consequences (i.e.,critical,serious,and nonserious situations or conditions) must be classified.Most importantly,software evaluation (according to established protocols)[145],clinical evaluations,and relevant evidence must be attached.Finally,its linguistic design and instructions must conform to standard medical terms.Other considerations include technology and sociotechnical system,environment,and information security with respect to safety.

CONCLUSION

This paper has provided an extensive review on computerized imaging in both current and emerging clinical practices and when integrated with state-of-the-art algorithms.The vital roles of this modality include the diagnosis of liver disease and its curative planning,treatment,and surgical intervention.It has been demonstrated in the recent literature that,depending on the data condition,prior knowledge,and amount of user interaction involved,various computer algorithms yield reasonable diagnostic and simulation accuracies.Nonetheless,it is worth noting that while these algorithms perform particularly well for functional segment classification of normal or slightly pathological livers,their performance on hepatic lesion characterization remains to be much further improved.

Although ML and AI strategies have rapidly become the main players in liver imaging and thus far have exhibited promising results,it remains challenging to acquire sufficiently large and heterogeneous datasets with labeled ground truth for training.This issue has been partly addressed in many less critical applications by using,for instance,big and crowdsourced data.

Finally,with advances in medical imaging,many computer algorithms will be adopted and implemented in SaMD.Therefore,researchers,digital health manufacturers,and physicians should be made aware of relevant regulatory requirements and guidelines to ensure the safety of patients.

FOOTNOTES

Author contributions:Horkaew P analyzed the literature,discussed the studies,and wrote the paper;Chansangrat J,Keeratibharat N,and Le DC compiled the studies,drafted the review,and wrote the paper.

Conflict-of-interest statement:The authors declare no conflicts of interest for this article.

Open-Access:This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers.It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license,which permits others to distribute,remix,adapt,build upon this work non-commercially,and license their derivative works on different terms,provided the original work is properly cited and the use is non-commercial.See: https://creativecommons.org/Licenses/by-nc/4.0/

Country/Territory of origin:Thailand

ORCID number:Paramate Horkaew 0000-0003-0879-7125;Jirapa Chansangrat 0000-0002-7004-1065;Nattawut Keeratibharat 0000-0002-8676-5164;Doan Cong Le 0000-0002-2229-413X.

S-Editor:Yan JP

L-Editor:Wang TQ

P-Editor:Yu HG