Using receiver operating characteristic curves to evaluate the diagnostic value of the combination of multislice spiral CT and alpha-fetoprotein levels for small hepatocellular carcinoma in cirrhotic patients

Harbin, China

Using receiver operating characteristic curves to evaluate the diagnostic value of the combination of multislice spiral CT and alpha-fetoprotein levels for small hepatocellular carcinoma in cirrhotic patients

Guang-Sheng Jia, Guang-Long Feng, Jin-Ping Li, Hai-Long Xu, Hui Wang, Yi-Peng Cheng, Lin-Lin Yan and Hui-Jie Jiang

Harbin, China

BACKGROUND: The various combination of multiphase enhancement multislice spiral CT (MSCT) makes the diagnosis of a small hepatocellular carcinoma (sHCC) on the background of liver cirrhosis possible. This study was to explore whether the combination of MSCT enhancement scan and alpha-fetoprotein (AFP) level could increase the diagnostic efif ciency for sHCC.

METHODS: This study included 35 sHCC patients and 52 cirrhotic patients without image evidence of HCC as a control group. The diagnoses were made by three radiologists employing a 5-point rating scale, with postoperative pathologic results as the gold standard. Receiver operating characteristic (ROC) curve analysis was performed to evaluate the diagnostic value of the three MSCT combination modes (arterial phase+portal-venous phase, arterial phase+delayed phase, arterial phase+portal-venous phase+delayed phase) and AFP levels for sHCC on the background of liver cirrhosis.

RESULTS: The area under ROC curve (AUC), sensitivity, and speci ficity of the combination of arterial phase+portalvenous phase+delayed phase were 0.93, 93%, and 82%, respectively. The average AUC of the arterial phase+portalvenous phase+delayed phase combination was signi ficantly greater than that of the arterial phase+portal-venous phase(AUC=0.84,P=0.01) and arterial phase+delayed phase (AUC=0.85,P=0.03). Arterial phase+portal-venous phase had a smaller AUC (0.84) than arterial phase+delayed phase (0.85), but the difference was insigni ficant (P=0.15). After combining MSCT enhancement scan with AFP, the AUC, sensitivity, and speci ficity were 0.95, 94%, and 83%, respectively, indicating a greatly increased diagnostic ef ficiency for sHCC.

CONCLUSIONS: The combination of AFP and 3 phases MSCT enhancement scan could increase the diagnostic ef ficiency for sHCC on the background of liver cirrhosis. The application of ROC curve analysis has provided a new method and reference in HCC diagnosis.

(Hepatobiliary Pancreat Dis Int 2017;16:303-309)

hepatocellular carcinoma; receiver operating characteristic; multi-slice spiral CT; alpha-fetoprotein; delayed phase imaging

Introduction

Hepatocellular carcinoma (HCC) is the sixth most common cancer worldwide and has shown an increasing incidence in recent years.[1]Liver cirrhotic patients have a far higher risk of HCC than other populations.[2]Small hepatocellular carcinoma (sHCC) refers to HCC measuring less than 3 cm for 1 nodule or 2 nodules together.[3]Early diagnosis of HCC is crucial to improving the prognostic effect of treatments. CT scanning is the primary method for HCC detection and diagnosis, in particular multislice spiral CT (MSCT), which plays an important role in sHCC diagnosis, differentialdiagnosis, and even HCC staging.[4,5]In recent years, some researchers[6-9]have started to focus on the diagnostic value of delayed phase imaging for sHCC. Takayasu et al[8]found that delayed phase imaging could improve diagnostic sensitivity for HCC (especially sHCC). Regarding HCC detection, CT enhancement imaging in the portal-venous phase is usually less sensitive than that in the arterial phase, because the majority of the blood supply of the hepatic parenchyma is through the portal vein. There is also hepatic parenchymal enhancement in the portal-venous phase; because hepatic parenchymal enhancement and tumor enhancement usually present similar densities, tumor detection rate is significantly reduced in the portal-venous phase. However, CT enhancement imaging in the portal-venous phase helps detect hypovascular liver tumors, such as liver metastasis from colorectal cancer, since hepatic parenchyma obtains the maximal level of enhancement in the portal-venous phase, resulting in a clear difference in the CT enhancement level from non-enhancement liver metastasis. In addition, portal-venous enhancement imaging helps identify some portal-venous complications, such as tumor thrombus. sHCCs are usually highly differentiated hypovascular tumors. Compared with large HCC or latestage HCC, sHCC has less arterial blood supply and no enhancement; thus, it does not have obvious enhancement in arterial phase imaging, resulting in a lower detection rate of sHCC in arterial phase than that in latestage HCC.

HCC patients usually have relatively elevated alphafetoprotein (AFP) levels, while cirrhotic patients have lower AFP levels.[10]Although AFP level is commonly utilized as a biomarker for sHCC screening, its validity for sHCC diagnosis is still controversial. For instance, the study conducted by Sangiovanni et al found that only 38% of sHCC patients had elevated AFP levels.[2]AFP levels depend on tumor size; late-stage HCC patients have relatively high AFP levels.[11,12]In order to further determine the diagnostic value of AFP levels for sHCC, the receiver operating characteristic (ROC) curve[13]analysis method has been adopted by many researchers. There are currently two types of studies; the first type independently evaluates the diagnostic value for sHCC by calculating sensitivity, specificity, diagnostic accuracy, the area under ROC curve (AUC). In order to determine the value of AFP levels for diagnosis of sHCC on the background of liver cirrhosis, Sarwar et al[14]plotted the ROC curve and found that the AUC was 0.85 and the best diagnostic threshold of AFP was 20.85 ng/mL. They suggested that although the AFP method has a low sensitivity in sHCC diagnosis, it is still an effective tumor marker for sHCC screening. The second approach determines AFP’s contribution to sHCC diagnosis by comparing sHCC diagnosis indexes between AFP and other tumor markers. Jiang et al[15]performed ROC curve analysis to compare the HCC diagnosis contributions of AFP and AFP-IgM immunocomplex, and the best diagnostic thresholds were determined to be 10 μg/L and 300 AU/mL, respectively. They suggested that AFP has a certain value in sHCC diagnosis, and measuring it in combination with other tumor markers or imaging diagnostic methods would greatly improve the ability of sHCC diagnosis.

There are so far no studies that have utilized the ROC curve approach to evaluate sHCC diagnosis efficiency by the combination of different MSCT enhancement phases and AFP levels. In this study, ROC curve analysis was used to evaluate three combinations of MSCT enhancement phases (arterial phase+portal-venous phase, arterial phase+delayed phase, arterial phase+portal-venous phase+delayed phase) through a 5-point rating scale, in combination with AFP levels, in order to evaluate the diagnostic value for sHCC in cirrhotic patients. This study aimed to provide new methods and references for evaluating the clinical value of different combinations of CT enhancement phases in HCC diagnosis.

Methods

General information

The study group consisted of pathologically proved 47 sHCC in 35 patients in the Second Affiliated Hospital of Harbin Medical University between January 2013 and December 2014. The study group consisted of 21 men and 14 women, aged 45-80 years old with an average age of 62 years. The control group consisted of 52 cirrhotic patients (patients received an ultrasound examination once every 3 months; each examination was conducted by three experienced radiologists using a double blind method for comprehensive diagnosis, and cancer nodule was never observed), including 30 men and 22 women, aged 34-85 years old with an average age of 61 years. The AFP values of all patients were recorded.

Instruments and methods

Lightspeed 64 Slice spiral CT (GE, USA) was used to scan all patients. The scanning range was from the diaphragmatic dome to the lower edge of the liver; the patients were in a supine position for the upper abdomen scan; the tube voltage was 120 kV; effective tube current was 200-300 mA; thread pitch was 1.375:1; and the rotate speed was 0.6-0.8 s/r. As a contrast agent, 80-100 mL of iopromide (Bayer Schering Pharma AG, Germany) was injected through the cubital vein at a flow rate of 3-3.5 mL/s. Enhancement scans of the three phases were per-formed at 20-30 seconds, 50-60 seconds, and 2-3 minutes after contrast agent injection.

Images analysis

All CT images were uploaded to a workstation with all identification information removed, including patient’s name, age, gender, and hospital record number. These images were evaluated blindly and independently by three experienced radiologists. In order to more accurately evaluate the diagnostic values of the arterial phase, portal-venous phase, and delayed phase for sHCC, the radiologists applied combinations of arterial phase+portal-venous phase, arterial phase+delayed phase, and arterial phase+portal-venous phase+delayed phase for separate diagnoses, and the interval between any two diagnoses was set at 2 weeks. In other words, each radiologist needed to wait for 2 weeks after each diagnosis to make a 2nd diagnosis using another combination of CT images in the same patient. In order to ensure the objectivity and repeatability of the analyses in this study, distinguishing criteria for HCC, dysplastic nodules, arterioportal shunt and focal nodular hyperplasia (FNH) were developed. The criterion for HCC is nodules showing obvious homogeneous or heterogeneous enhancement and appearing hyper-dense compared to surrounding tissue in the arterial phase, iso- or hypo-dense in the portal-venous phase, and hypo-dense in the delayed phase.[16]Many well differentiated sHCCs share the similar enhancement pattern with dysplastic nodules, which is iso- or hypo-dense in the arterial and delayed phases and hypo-dense in the delayed phase.[17]These two types of nodules are generally distinguished by size: the diameter of dysplastic nodules is usually smaller than that of well differentiated sHCC, and 10 mm is the cut-off value; nodules larger than 10 mm are considered HCC, while smaller than 10 mm are considered dysplastic nodules.[18]Meanwhile, differential diagnosis needs to be performed for HCC nodules and arterioportal shunt, because these two usually present similar enhancement patterns: hyper-dense in the arterial phase, hyper- or iso-dense in the portal-venous phase, and iso-dense in the delayed phase. If a focus lesion manifests as a typical wedge shape with or without an internal linear branching structure, it is seen as an arterioportal shunt. If a lesion presents a round shape, it is presumed to be HCC;[19]however, the confidence level was set at 3 (might happen). Since FNH is a benign tumor that is usually confused with HCC, it is necessary to differentiate it from HCC. On contrastenhanced CT, FNH usually presents homogenous hyperenhancement in the arterial and portal-venous phases and iso-enhancement in the delayed phase. A common feature of FNH is its central scar which is visible on contrast-enhanced CT. Therefore, if it manifests the typical stellate scars at the central of the nodule and iso-dense in the delayed phase, the nodule is seen as an FNH. The radiologists used a 5-point rating scale for their diagnosis:“1” corresponds to a definite negative, “2” to a possible negative, “3” to a possible positive, “4” to a highly possible positive, and “5” to a definite positive diagnosis.[20]In order to calculate diagnostic accuracy, sensitivity, and specificity, results equal to or greater than 3 were considered sHCC, otherwise, the diagnosis was liver cirrhosis only. In order to confirm whether measuring AFP levels helps increase the diagnostic value of MSCT enhancement scan for sHCC, the radiologists were asked to re-assess with the 5-point rating scale based on a combination of AFP levels (a study[14]showed that the best diagnostic threshold of AFP for HCC on the background of liver cirrhosis was 20.85 ng/mL; the corresponding sensitivity and specificity were 72.2% and 86.2%, respectively) and images of arterial phase+portal-venous phase+delayed phase. The 2nd review has to wait for at least 2 weeks after the 1st review. The same 5-point rating scale was used, and follow-up statistical analyses were also performed.

Statistical analysis

The detection rate is defined as the ratio of the total number of hyper-dense to hypo-dense foci found in a given MSCT phase. Data analysis was performed with SPSS 18.0 software. Pathological examination of the tumor tissue was employed as the gold standard. Based on the radiologists’ diagnoses and the patients’ pathological results, ROC curves were plotted for the diagnostic results (rescored based on AFP and 3-phase combination images) of MSCT images of arterial phase+portalvenous phase, arterial phase+delayed phase, and arterial phase+portal-venous phase+delayed phase, as well as 3-phase combination MSCT+AFP level. AUCs were obtained to evaluate the diagnostic value of different enhancement phase combinations for sHCC on the background of liver cirrhosis. Measurement data were analyzed through Wilcoxon signed-rank test; enumeration data were analyzed by Fisher’s exact test; andP〈0.05 was considered statistically significant. The kappa statistic was utilized to assess the consistency of diagnostic results in a pair-wise manner; a kappa value 〈0.4 suggests poor consistency, a kappa value between 0.41 and 0.75 suggests good consistency, and a kappa value ＞0.75 suggests high consistency.[21]

Results

Enhancement patterns of sHCC

The diameter of sHCC ranged between 7 and 20 mm,with an average of 14 mm. The enhancement patterns of sHCC are shown in Table 1. Through statistical analysis of enhancement patterns, it was found that sHCC usually appeared hyper-dense compared to surrounding tissue in the arterial phase (Fig. 1A), iso- or hypo-dense in the portal-venous phase (Fig. 1B), and hypo-dense in the delayed phase (Fig. 1C). Fisher’s exact test showed significant differences between the enhancement pattern of the arterial phase and that of the portal-venous phase and the delayed phase (bothP〈0.01), as well as a significant difference between the enhancement patterns of the portal-venous phase and the delayed phase (P〈0.01). The detection rates of the arterial, portal-venous and delayed phases were 83.0%, 61.7%, and 87.2%, respectively, with a significant difference in the detection rates of the three phases (P〈0.01). Moreover, the detection rates of the arterial and delayed phases were significantly higher than that of the portal-venous phase (P〈0.05;P〈0.01), and there was no significant difference between the detection rates of the arterial phase and the delayed phase (P=0.77).

Comparison of the diagnostic accuracy of the 3 different phase combinations

In this study, three radiologists participated in the sHCC diagnosis employing 3 phase combination modes of MSCT enhancement scan, while the kappa statistic was adopted to evaluate the diagnostic consistency between pairs of radiologists. The sHCC diagnostic results using any phase combination of MSCT enhancement scan were all highly consistent (kappa: 0.68-0.82). Arterial phase+portal-venous phase+delayed phase had the highest kappa value, and arterial phase+delayed phase had a slightly higher consistency than the combination of arterial phase+portal-venous phase (Table 2).

In Table 3, the arterial phase+portal-venous phase+ delayed phase combination had the highest accuracy (87%), sensitivity (93%), and specificity (82%), and the average AUC was 0.93, significantly greater than that of the arterial phase+portal-venous phase (AUC=0.84,P=0.01) and arterial phase+delayed phase (AUC=0.85,P=0.03). Arterial phase+portal-venous phase had a smaller AUC (0.84) than arterial phase+delayed phase (0.85), but the difference was insignificant (P=0.15).

In order to evaluate the overall diagnostic value of the 3 MSCT phase combinations (arterial phase+portal-venous phase, arterial phase+delayed phase, arterial phase+portal-venous phase+delayed phase) for sHCC on the background of liver cirrhosis, the three radiologists’ scores for each image set were averaged to plot ROC curves (Fig. 2), and the AUC values of arterial phase+portal-venous phase, arterial phase+delayed phase, and arterial phase+portal-venous phase+delayed phase were 0.88, 0.89, and 0.97, respectively, indicating that the 3-phase combination had the highest diagnostic accuracy, while the diagnostic accuracy of arterial phase+delayed phase was higher than that of arterial phase+portal-venous phase.

Table 1. Enhancement pattern of 47 small hepatocellular carcinomas and detection rate

Table 2. Kappa coefficient of three observers

Table 3. Accuracy, sensitivity, specificity and AUC of different phase combinations of MSCT and the combination of three MSCT phases and AFP

Fig. 1. A 67-year-old male patient with small focus (arrow) on the right lobe of the liver with an approximate size of 15×18 mm. A: Obvious complete enhancement in the arterial phase, hyper-density; B: Iso-density in the portal-venous phase; C: Hypo-density in the delayed phase.

Fig. 2. Comprehensive ROC curves for 3 combinations from 3 observers (scores from 3 observers were averaged for each combination).

Fig. 3. ROC curves from 3 observers of the combination of MSCT enhancement scan 3-phase combination mode and AFP for sHCC diagnosis.

The combination of MSCT enhancement with AFP improves the diagnostic accuracy of sHCC

The average AFP level of patients in the sHCC group was 165.30 ng/mL, significantly higher than that of the control group (15.40 ng/mL,P〈0.01). The radiologists re-scored based on AFP values and MSCT enhancement scan 3-phase combination images. After ROC curve analysis; the obtained AUC values were 0.94, 0.95, and 0.97 (Fig. 3), all greater than the AUC values obtained using only MSCT enhancement scan 3-phase combination. The sHCC diagnostic sensitivity, specificity and diagnostic accuracy based on both MSCT enhancement scan and AFP are shown in Table 3.

Discussion

Most sHCC cases have a “fast-in, fast-out” feature: in the arterial phase, tumors appear hyper-dense compared with surrounding hepatic parenchyma, which is the diagnostic basis of sHCC.[22]In the portal-venous phase, the degree of enhancement rapidly drops and the tumor appears iso- or hypo-dense. In the delayed phase, the enhancement degree keeps dropping and the tumor appears hypo-dense.[23,24]This is because most sHCC cases are hypervascular tumors; a great amount of portalvenous blood is supplied to the liver, resulting in peak values of the enhancement degree of hepatic parenchyma in the portal-venous phase. Because the hypervascular sHCC also presents a high enhancement state, the hepatic parenchyma and sHCC have similar densities, and thus some sHCC cases appear iso-dense in the portalvenous phase.[25]

Comprehensive ROC curve analysis showed the AUC of arterial phase+portal-venous phase+delayed phase was greater than that of arterial phase+portal-venous phase, or arterial phase+delayed phase. These data indicated that the 3-phase combination for sHCC on the background of liver cirrhosis had the highest diagnostic accuracy. It was pointed out by Choi et al[26]that the diagnostic accuracy of arterial phase+portal-venous phase was not significantly different from that of the 3-phase combination, and the delayed phase had no diagnostic significance for sHCC. However, sHCC cases in Choi’ s study were all hypervascular nodules, whereas many sHCC cases in clinical practice are hypovascular. Arterial phase and portal-venous phase have poor diagnostic value for hypovascular sHCC cases, while the delayed phase plays an important role. MSCT 3-phase enhancement scan can detect the features of sHCC cases with different vascular types, and is favorable for early HCC diagnosis.[27]Combining images of arterial phase, portal-venous phase, and delayed phase not only increase diagnostic accuracy of sHCC, but also improve focal lesion detection rate.

As a common HCC diagnostic method, MSCT multiphase enhancement scan has been frequently investigated by researchers, and selection of the best phase combination has become a research focus. Kim et al[28]compared the diagnostic values of dynamic enhancement CT 3-phase scan (arterial phase, portal-venous phase and delayed phase) and 4-phase scan (early arterial phase, arterial phase, portal-venous phase and delayed phase) for HCC, and found no significant differences in sensitivity, specificity, AUC between 3-phase scan and 4-phase scan. They concluded that 4-phase enhancement scan does not improve diagnostic accuracy for HCC. Denecke et al[23]performed MSCT plain scans and 3-phase scans (arte-rial phase, portal-venous phase and venous phase), calculated and compared the detection rates of plain scan and different phases of enhancement scan, and found all nodules were visible in the arterial or portal-venous phases, and that the detections rates of the arterial phase and the portal-venous phase were both signi ficantly higher than that of the plain scan and the venous phase.

Table 1 showed that the detection rate of the delayed phase (87.2%) was higher than that of the portal-venous phase (61.7%), indicating that the delayed phase played an important role in improving diagnostic accuracy for sHCC. This is consistent with the research finding of Iannaccone et al,[6]who compared the sensitivities and positive predictive values of MSCT enhancement images of different phase combinations in diagnosing sHCC on the background of liver cirrhosis, they found that the delayed phase could signi ficantly increase diagnostic sensitivity, presenting a supporting effect on CT dual-phase scan for HCC diagnosis. Lim et al[29]compared AUCs of the 3-phase combination and the arterial phase+portalvenous phase combination, and reached a similar conclusion: adding delayed phase imaging into dual-phase CT scan helps describe HCC features and improves diagnostic accuracy. Although adding delayed phase imaging will increase the patient’s exposure to radiation and add extra scanning time, it can improve detection rate and diagnostic accuracy for HCC, thus offsetting the cost of radiation exposure to some extent.

While imaging examination has many advantages, AFP is a speci fic tumor marker for HCC and has an important value in early sHCC diagnosis.[10,30]In this study, the combination of AFP and MSCT enhancement scan for sHCC diagnosis had an AUC of 0.95, higher than that of the AUC of the MSCT 3-phase combination (0.93). Through ROC curve analysis, this study further confirmed the important role of the combination of MSCT enhancement scan and AFP in improving the diagnostic accuracy for sHCC.

Although certain signi ficant results have been obtained, there are still some shortcomings in this study. First, although all patients in this study underwent the same 64-slice spiral CT scan, there have been some changes to the technology over time; since the examinations were performed at different times, there might have been an in fluence on sHCC detection. Second, parameters such as the dose and injection rate of contrast agent, and the scanning time of each phase were not customized, but were rather set as fixed values due to the limitations of clinical practice, which could also affect the results.

In conclusion, the combination of arterial phase+ portal-venous phase+delayed phase and AFP is the most accurate approach for the diagnosis of sHCC on the background of liver cirrhosis, with the highest AUC, sensitivity, and specificity. Compared with the combination of arterial phase+portal-venous phase, the combination of arterial phase+delayed phase had greater AUC in sHCC diagnosis, indicating that the delayed phase has an important role in sHCC diagnosis that should not be overlooked.

Contributors:JHJ proposed the study. JGS, FGL, LJP and XHL performed the research and wrote the first draft. WH, CYP and YLL collected and analyzed the data. All authors contributed to the design and interpretation of the study and to further drafts. JHJ is the guarantor.

Funding:This study was supported by grants from the National Natural Science Foundation of China (81301275, 81471736 and 81671760), the National Science and Technology Pillar Program during the Twelfth Five-Year Plan Period (2015BAI01B09), and Heilongjiang Province Foundation for Returness (LC2013C38).

Ethical approval:This study was approved by the institutional review board of Harbin Medical University.

Competing interest:No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.

1 Jemal A, Bray F, Center MM, Ferlay J, Ward E, Forman D. Global cancer statistics. CA Cancer J Clin 2011;61:69-90.

2 Sangiovanni A, Del Ninno E, Fasani P, De Fazio C, Ronchi G, Romeo R, et al. Increased survival of cirrhotic patients with a hepatocellular carcinoma detected during surveillance. Gastroenterology 2004;126:1005-1014.

3 Hernandez-Gea V, Turon F, Berzigotti A, Villanueva A. Management of small hepatocellular carcinoma in cirrhosis: focus on portal hypertension. World J Gastroenterol 2013;19:1193-1199.

4 Hennedige T, Venkatesh SK. Advances in computed tomography and magnetic resonance imaging of hepatocellular carcinoma. World J Gastroenterol 2016;22:205-220.

5 Hennedige T, Venkatesh SK. Imaging of hepatocellular carcinoma: diagnosis, staging and treatment monitoring. Cancer Imaging 2013;12:530-547.

6 Iannaccone R, Laghi A, Catalano C, Rossi P, Mangiapane F, Murakami T, et al. Hepatocellular carcinoma: role of unenhanced and delayed phase multi-detector row helical CT in patients with cirrhosis. Radiology 2005;234:460-467.

7 Lin YY, Chen CM, Huang YH, Lin CY, Chu SY, Hsu MY, et al. Liver metastasis from hepatoid adenocarcinoma of the stomach mimicking hepatocellular carcinoma: Dynamic computed tomography findings. World J Gastroenterol 2015;21:13524-13531.

8 Takayasu K, Muramatsu Y, Mizuguchi Y, Ojima H. CT Imaging of early hepatocellular carcinoma and the natural outcome of hypoattenuating nodular lesions in chronic liver disease. Oncology 2007;72:83-91.

9 Furlan A, Marin D, Vanzulli A, Patera GP, Ronzoni A, Midiri M, et al. Hepatocellular carcinoma in cirrhotic patients at multi-detector CT: hepatic venous phase versus delayed phase for the detection of tumour washout. Br J Radiol 2011;84:403-412.

10 Bertino G, Neri S, Bruno CM, Ardiri AM, Calvagno GS, Malaguarnera M, et al. Diagnostic and prognostic value of alpha-fetoprotein, des-γ-carboxy prothrombin and squamous cell carcinoma antigen immunoglobulin M complexes in hepatocellular carcinoma. Minerva Med 2011;102:363-371.

11 Li T, Qin LX, Gong X, Zhou J, Sun HC, Qiu SJ, et al. Hepatitis B virus surface antigen-negative and hepatitis C virus antibody-negative hepatocellular carcinoma: clinical characteristics, outcome, and risk factors for early and late intrahepatic recurrence after resection. Cancer 2013;119:126-135.

12 Tandon P, Garcia-Tsao G. Prognostic indicators in hepatocellular carcinoma: a systematic review of 72 studies. Liver Int 2009;29:502-510.

13 Hajian-Tilaki K. Receiver operating characteristic (ROC) curve analysis for medical diagnostic test evaluation. Caspian J Intern Med 2013;4:627-635.

14 Sarwar S, Khan AA, Tarique S. Validity of alpha fetoprotein for diagnosis of hepatocellular carcinoma in cirrhosis. J Coll Physicians Surg Pak 2014;24:18-22.

15 Jiang J, Wu C, Shen Y, Xu B, Zheng X, Li X, et al. Clinical application of determining serum AFP-IgM complexes for diagnosis of small hepatocellular carcinoma. Anticancer Res 2011;31:687-691.

16 Tang A, Cruite I, Sirlin CB. Toward a standardized system for hepatocellular carcinoma diagnosis using computed tomography and MRI. Expert Rev Gastroenterol Hepatol 2013;7:269-279.

17 Choi BI, Han JK, Hong SH, Kim TK, Song CS, Kim KW, et al. Dysplastic nodules of the liver: imaging findings. Abdom Imaging 1999;24:250-257.

18 Jang HJ, Lim JH, Lee SJ, Park CK, Park HS, Do YS. Hepatocellular carcinoma: are combined CT during arterial portography and CT hepatic arteriography in addition to triple-phase helical CT all necessary for preoperative evaluation? Radiology 2000;215:373-380.

19 Wei XB, Xu J, Li N, Yu Y, Shi J, Guo WX, et al. The role of three-dimensional imaging in optimizing diagnosis, classification and surgical treatment of hepatocellular carcinoma with portal vein tumor thrombus. HPB (Oxford) 2016;18:287-295.

20 Sofue K, Tsurusaki M, Kawasaki R, Fujii M, Sugimura K. Evaluation of hypervascular hepatocellular carcinoma in cirrhotic liver: comparison of different concentrations of contrast material with multi-detector row helical CT--a prospective randomized study. Eur J Radiol 2011;80:e237-242.

21 McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276-282.

22 Marrero JA, Hussain HK, Nghiem HV, Umar R, Fontana RJ, Lok AS. Improving the prediction of hepatocellular carcinoma in cirrhotic patients with an arterially-enhancing liver mass. Liver Transpl 2005;11:281-289.

23 Denecke T, Grieser C, Fr?ling V, Steffen IG, Rudolph B, Stelter L, et al. Multislice computed tomography using a triple-phase contrast protocol for preoperative assessment of hepatic tumor load in patients with hepatocellular carcinoma before liver transplantation. Transpl Int 2009;22:395-402.

24 Haberman D, Mela M, Martínez A, Mancinelli A, Laguens R, Gruz F, et al. Accuracy of multislice computed tomography in the diagnosis of hepatocellular carcinoma in patients with cirrhosis evaluated for liver transplantation. Acta Gastroenterol Latinoam 2011;41:190-198.

25 Cartier V, Aubé C. Diagnosis of hepatocellular carcinoma. Diagn Interv Imaging 2014;95:709-719.

26 Choi BI, Lee HJ, Han JK, Choi DS, Seo JB, Han MC. Detection of hypervascular nodular hepatocellular carcinomas: value of triphasic helical CT compared with iodized-oil CT. AJR Am J Roentgenol 1997;168:219-224.

27 Koh TS, Thng CH, Hartono S, Lee PS, Choo SP, Poon DY, et al. Dynamic contrast-enhanced CT imaging of hepatocellular carcinoma in cirrhosis: feasibility of a prolonged dual-phase imaging protocol with tracer kinetics modeling. Eur Radiol 2009;19:1184-1196.

28 Kim SK, Lim JH, Lee WJ, Kim SH, Choi D, Lee SJ, et al. Detection of hepatocellular carcinoma: comparison of dynamic three-phase computed tomography images and four-phase computed tomography images using multidetector row helical computed tomography. J Comput Assist Tomogr 2002;26:691-698.

29 Lim JH, Choi D, Kim SH, Lee SJ, Lee WJ, Lim HK, et al. Detection of hepatocellular carcinoma: value of adding delayed phase imaging to dual-phase helical CT. AJR Am J Roentgenol 2002;179:67-73.

30 Asrih M, Lenglet S, Mach F, Montecucco F. Alpha-fetoprotein: a controversial prognostic biomarker for small hepatocellular carcinoma. World J Gastroenterol 2013;19:328-330.

April 17, 2016

Accepted after revision December 7, 2016

Author Affiliations: Department of Radiology, Second Affiliated Hospital, Harbin Medical University, Harbin 150086, China (Jia GS, Feng GL, Li JP, Xu HL, Wang H, Cheng YP, Yan LL and Jiang HJ)

Hui-Jie Jiang, MD, PhD, Department of Radiology, Second Affiliated Hospital, Harbin Medical University, Harbin 150086, China (Tel: +86-451-86605576; Email: jhjemail@163.com)

? 2017, Hepatobiliary Pancreat Dis Int. All rights reserved.

10.1016/S1499-3872(17)60018-3

Published online May 18, 2017.