Robust Magnification Independent Colon Biopsy Grading System over Multiple Data Sources

Tina Babu,Deepa Gupta,Tripty Singh,＊,Shahin Hameed,Mohammed Zakariah and Yousef Ajami Alotaibi

1Department of Computer Science and Engineering,Amrita School of Engineering,Amrita Vishwa Vidyapeetham,Bengaluru,India

2Department of Pathology,MVR Cancer Center and Research Institute,Poolacode,Kerala,India

3College of Computer and Information Sciences,King Saud University,Saudi Arabia

4Computer Engineering Department,College of Computer and Information Sciences,King Saud University,Saudi Arabia

Abstract:Automated grading of colon biopsy images across all magnifications is challenging because of tailored segmentation and dependent features on each magnification.This work presents a novel approach of robust magnification-independent colon cancer grading framework to distinguish colon biopsy images into four classes:normal,well,moderate,and poor.The contribution of this research is to develop a magnification invariant hybrid feature set comprising cartoon feature,Gabor wavelet,wavelet moments,HSV histogram,color auto-correlogram,color moments,and morphological features that can be used to characterize different grades.Besides,the classifier is modeled as a multiclass structure with six binary class Bayesian optimized random forest(BO-RF)classifiers.This study uses four datasets(two collected from Indian hospitals—Ishita Pathology Center(IPC)of 4X,10X,and 40X and Aster Medcity(AMC)of 10X,20X,and 40X—two benchmark datasets—gland segmentation(GlaS)of 20X and IMEDIATREAT of 10X)comprising multiple microscope magnifications.Experimental results demonstrate that the proposed method outperforms the other methods used for colon cancer grading in terms of accuracy(97.25%-IPC,94.40%-AMC,97.58%-GlaS,99.16%-Imediatreat),sensitivity(0.9725-IPC,0.9440-AMC,0.9807-GlaS,0.9923-Imediatreat),specificity(0.9908-IPC,0.9813-AMC,0.9907-GlaS,0.9971-Imediatreat)and F-score(0.9725-IPC,0.9441-AMC,0.9780-GlaS,0.9923-Imediatreat).The generalizability of the model to any magnified input image is validated by training in one dataset and testing in another dataset,highlighting strong concordance in multiclass classification and evidencing its effective use in the first level of automatic biopsy grading and second opinion.

Keywords:Colon cancer;grading;texture features;color features;morphological features;feature extraction;Bayesian optimized random forest classifier

1 Introduction

Colorectal cancer is one of the world’s most common cancers and is the second leading cause of cancer death[1].In 2018,it ranked the third and second-most-common cancer for both genders’incidence and mortality globally,constituting respectively 6.1% and 5.8% of the number of new cases and deaths,among all cancers combined worldwide[2].The general cancer diagnosis process is tedious and reliant on experts using microscopic analysis of biopsy samples.An essential task for pathologists who analyze colon specimens across various magnifications in a microscope(4X,5X,10X,20X,and 40X)is to distinguish invasive cancer and,to provide an accurate diagnosis and grading critical for the treatment plan.The subjective character of grading evaluation and the different patterns that many tumors exhibit render it difficult to achieve consistency between pathologists.This method requires a substantial amount of time to provide results in both inter-and intra-observer variations[3,4].Owing to the visual discrepancy among observations,analyzing the sample under a microscope at various magnifications is crucial for an accurate diagnosis.The golden standard for diagnosis is an analysis by pathologists with subspecific expertise and specialty in gastrointestinal malignancy.However,second opinions are slow to come,work-intensive,and often not possible in areas with scarce resources.Advanced computerized pathology over numerous magnifications offers an assisted and suitable solution to this issue[4,5].In particular,with numerous digitized images of histology slides being progressively ubiquitous,automated diagnosis can help the pathologist by providing second opinions through machine learning.Automatic cancer screening is the first level of diagnosis followed by grades determination across various magnifications.To solve this multiclass classification problem,a magnification-independent framework is essential for investigating pathological images using image processing and machine learning techniques.

Most medical applications use image features and image processing techniques[6].A very recent and comprehensive literature review was performed to extract clinical details from histological slides[7,8].An overview of recent literature in two key directions on colon cancer diagnosis,i.e.,detection and grading of colon biopsy images,is reviewed in the current research.

Several automated approaches are available to distinguish between normal and malignant colon lesions.Rathore et al.[9,10]proposed an ellipse fitting algorithm with K-means clustering to segment the glands specifically on 10X magnified colon images and extracted a hybrid feature set(morphological,geometric,texture-based,scale-invariant feature transform,and elliptical Fourier descriptor features)and lumen characteristic dependent on the segmented region of interest(ROI)s and classified with SVM classifier into normal and malignant images.Furthermore,Rathore et al.[11]optimized the segmentation parameters for each magnification(4X,5X,10X,and 40X)for ellipse fitting algorithm using genetic algorithm and extracted gray-level co-occurrence matrix(GLCM)-based as well as gray-level histogram moment features from the segmented ROI to classify colon biopsy images through an SVM classifier,thereby attaining 92.33% average accuracy.Across various magnified colon images(10X,20X,40X),for cancer detection,texture,shape,and wavelet features were analyzed and classified using multi-classifier models in[12–15].Abdulhay et al.[16]suggested a strategy for the segmentation of blood leukocytes using static microscopes to classify 100 unique magnified microscopic pictures(72-abnormal,38-normal)by using SVM for the tuned segmentation and filtering of the non-ROI image using local binary patterns and texture characteristics with a 95.3% accuracy.With image,local,and gland features extracted from imagespecific tuned the multistep gland segmentation,Rathore et al.[17]encoded the glandular patterns and morphology of cells and detected cancer using a score-based ensemble SVM classifier.Their method was evaluated on the GlaS dataset[18]and 10X-magnified colon biopsy images,attaining accuracies of 98.30% and 97.60%,respectively.For 100 samples of BRATS Brain MRI data sets,Husham et al.[19]compared active contour and otsu threshold algorithms where the segmentation parameters were set for that dataset,and the supremacy of active contour was confirmed.Hussein et al.[20]proposed a new version for Viola-James that segments ultrasound images of the breast(250 images)and ovarian(100 images)that generate ROI with active contour tuned for these images and magnification and achieved a classification accuracy of 95.43% and 94.84.0% for breast and ovarian images with new features dependent on the segmented region,characterize the lesion.Recently,deep neural networks have been widely applied in medical image processing and digital pathology[21].Motivated by the LeNet-5 structure,glandular artifact and clustered gland segregation were detected using two convolutional neural networks(CNN)[22].Further,cancer was detected with 95% accuracy using the 20X-magnified images of the GlaS dataset.Xu et al.[23]utilized the activation features extracted from the CNN trained on Imagenet for segmentation and classification.The SVM classifier was used to classify the 10X-magnified colon and brain biopsy images with 98% and 97.8% accuracy,respectively.A deep CNN network was used for gland segmentation and characterization;then,the best alignment matrix(BAM)feature extracted from this segmented region was used for two-class classification with a 97%accuracy on the GlaS dataset[24].Later,Lichtblau et al.[25]implemented transfer learning on Alexnet to extract high-level features to classify the target images into benign and malignant samples with six classifiers’probability score.The classifier weights are optimized via differential evolution and achieved an accuracy of 96.66% on the GlaS dataset,and with BreaKHis[26]dataset accuracies of 83.9%,86%,89.1%,and 86.6% were tabulated for 40X,100X,200X,and 400X magnified microscopic images respectively.Iizuka et al.[27]extracted the high-level features with the Inception-v3 CNN network.They used a recurrent neural network and max-pooling to classify the images into two classes:adenocarcinoma,adenoma of the stomach,and colon whole slide images with an area under the curve of 0.980,0.974,respectively.

Many techniques have been explored in the grading/multiclass classification of colon biopsy images.Rathore et al.[10],using 10X magnified colon images,graded the malignant images into three classes:well,moderate,and poor with an SVM classifier based on the lumen area characteristics extracted from the lumen through the ellipse fitting algorithm on the white cluster obtained through K-means clustering for this dataset with 93.47% accuracy.Furthermore,Kather et al.[28],using conventional features such as GLCM,Histogram,local binary patterns,and Gabor,classified the colon tissue samples into eight classes utilizing an SVM classifier with 87.4%accuracy.With the GlaS dataset,Saroja et al.[29]implemented adaptive pillar K-means clustering to extract the lumen features;then,using a score-based decision tree,graded the malignant colon images into three classes with 93% accuracy.Boruz et al.[30],based on the texture and topological features extracted from the gland segmented image,classified the 10X-magnified Imediatreat[31]colon image dataset into four classes:healthy,well,moderate,and poor,and obtained an accuracy of 89.75% with an SVM classifier.The cell morphology,glandular structures,and texture are considered from tailored multi-step gland segmentation for the 10X-magnified images and GlaS dataset.The image,local,and gland features are extracted from these segmented images and graded malignant colon images into three classes;therein,both datasets achieved 98.6% accuracy using score-based ensemble SVM[17].Nawadhar et al.[32]proposed a stratified squamous epithelial biopsy image classifier that takes majority voting of the five classifiers for grading 676 oral mucosa 40X-magnified images into four classes:normal,well,moderate,and poor with 95.56% accuracy with the color,texture and shape features extracted from the segmented region.The cellular regions were segmented with unsupervised K-means clustering and Moore-neighbor tracing algorithm with Jacob’s stopping criteria tuned for this dataset.Rathore et al.[33],with ROI,delineated 20X glioma images,graded into high and low grades with the conventional,clinical,and texture features dependent on the ROI,with SVM classifier with 91.48% accuracy.Deep learning techniques were also explored for the grading or multiclass classification of biopsy images.Xu et al.[23],with the high-level features extracted from the Imagenet CNN model,segmentation of patches is performed with supervised learning using linear SVM and classified the 10X-magnified colon tissue images into six classes with 87% accuracy.Gland segmentation was performed using CNN based on UNet architecture,wherein BAM was extracted from the segmented glands,thereby using glandular aberration features with the SVM classifier for grading 20X-magnified colon biopsy images into three classes:normal,low grade,and high grade with 95.33% accuracy[24].Lichtblau et al.[25]optimized the ensemble weights of six distinct classifiers with differential evolution algorithm,thereby considering individual classifiers probabilities for grading each sample into four classes.Thereby,using to grade 10X-magnified colon image Imediatreat[31]dataset into four classes using the activation features extracted from the Alexnet CNN model with 98.29% accuracy.

In the majority of literature,where color-based clustering,segmentation,and features[9–11]are used,the techniques depend on the image color intensities that subsequently depend on the staining concentrations and illumination conditions[5];hence,affect the post-processing using color features[34,35].Besides,the traditional approaches for cancer detection or grading include segmentation methods tuned for specific magnified images(mostly 10X-magnified)and performance deteriorates with other image magnifications(4X,20X,40X)as the parameters are set for a particular magnification[9–11,16,17,19,20,29,30].Thus,finding a region of interest(ROI)is tedious for each image magnification.Further,features extracted from these segmented regions,including geometric,lumen,morphological,and topological features that depend on spatial domain,differ across image magnification.Although deep learning plays a vital role in many classification problems where CNN automatically and optimally adjusts feature extraction for the desired classification[24,27],it requires massive,detailed annotated medical data that is scarce,complex hardware,and high computation time.Binary class problems are better classified using deep learning models.However,for grading or multiclass problems,activation features are extracted from the existing CNN models,and classifiers are optimized to boost classification accuracy[23,25].Moreover,in traditional methods and deep learning models,training and testing were performed with the respective datasets and magnification.A thorough literature review reveals the need for an efficient magnification-independent colon cancer grading framework for biopsy images applicable across various H&E colon biopsy image datasets.

This work’s primary objective is to simplify the automated magnification-independent fourclass grading framework on a set of images from histopathological colon tissue slides where the grading ranges from normal/healthy to three grade levels—well,moderate,and poor.A robust magnification-invariant rich hybrid feature set is proposed that explores the structural,textural,color,and shape properties across magnifications.Further,training ensembles of Bayesian optimized random forest classifiers eased the grading problem by using a majority voting to obtain the final classification label.The pursued contributions are as follows.

?Image pre-processing as stain normalization for stain concentrations to ensure image uniformity within and across multiple datasets.

?A robust,rich hybrid feature set independent of the spatial variations is proposed,containing texture(cartoon features,Gabor wavelet,wavelet moments),color(HSV histogram,color auto-correlogram,color moments),and morphological features.

?Using Ensemble Bayesian Optimized Random Forest classifiers,the proposed framework classifies the images as a multiclass structure with six classifiers to ease the multiclass grading problem,and according to the maximum similar population,the final class is predicted to ensure optimal classification accuracy.

?The model’s generalizability proposed on various magnified datasets is evaluated across four colon biopsy image datasets(two collected from Indian hospitals and two benchmark datasets).

?Training in one dataset and testing with other datasets provides a better outcome with the robust classification model,ensuring any magnified input colon biopsy images’applicability.

The rest of the paper is organized as follows:Section 2 presents input colon image characteristics and datasets used,while the proposed methodology is described in Section 3.Performance measures used for evaluation and results are described in Section 4 and discussed in Section 5.Finally,in Section 6,the conclusion and future work are presented.

2 Input Colon Biopsy Image Datasets

H&E-stained colon biopsy images contain pink-colored connecting tissues,purple-colored nuclei,and white-colored epithelial cells and lumen[9,10].The structure of a normal/healthy colon biopsy image has a definite glandular structure for the white-colored epithelial cells[9,36],as shown in Fig.1a.However,this definite structure is distorted when cancer occurs as the whitecolored epithelial cells and lumen gradually combine with the pink-colored connecting tissues,and the deformation increases as the grade of cancer advances.The differentiability of malignant cells is quantified by three colon cancer grades wherein their color composition and texture vary[36].The glandular shape is almost maintained in well-differentiated tumors(Fig.1b),whereas the moderately differentiable grade differs from the normal shape(Fig.1c).The epithelial cells that form the glandular border irregularly scatter in poorly differentiated tumors,making it difficult to determine individual glands border(Fig.1d).Thus,developing a framework that classifies H&E-stained images into four grades:normal,well,moderate,and poor,is difficult.

Figure 1:Four classes of colon biopsy images:(a)normal,(b)well,(c)moderate,and(d)poor

The proposed framework is evaluated using the colon pathological image data obtained from four independent sources(two collected from Indian hospitals and two benchmark datasets)from different locations and at different microscope magnifications at which the pathologist observed the tissue sample:

?Ishita Pathology Center dataset:1200 images at a resolution of 640 × 480 were collected from H&E-stained colon biopsy samples of 5–6μmm thick tissue section slides from IPC,Allahabad,India for magnifications of 4X,10X,and 40X.For each grade under a particular magnification,there are 100 images(normal = 100,well = 100,moderate = 100,and poor = 100).A Magcam CD5 with Olympus CX33 was used to capture the images.Dr.Ranjana Srivastava,the Senior Consultant at IPC,analyzed the H&E slides and prepared the ground truth labels for the dataset.

?AMC dataset:840 images at a resolution of 640 × 480 were collected from H&E-stained colon biopsy samples of 5–6μmm thick tissue section slides from the Department of Pathology,Aster Medcity(AMC),Kochi,India,for magnifications of 10X,20X,and 40X.For each grade under a particular magnification,there are 70 images(normal = 70,well = 70,moderate = 70,and poor = 70).A NIS element viewer microscope was used to view the slides,and a Nikon eclipse Ci was used to capture the images.Dr.Sarah Kuruvila(Former Senior Consultant,Pathology Department,Aster Medcity,Kochi,India)and Dr.Shahin Hameed(Consultant Pathologist,MVR Cancer Center and Research Institute,Poolacode,Kerala,India)analyzed the H&E slides of the colon biopsy.They prepared the dataset and provided the ground truth labels.

?GlaS dataset[18]:165 images acquired at a 20X magnification with 640 × 480 resolution were collected from the GlaS dataset.Images were labeled by an expert pathologist as normal =74,moderate = 47,moderate-to-poor = 20,and poor = 24.

?IMEDIATREAT dataset[31]:357 10X-magnified images were acquired at a resolution of 800 × 600 with 62 normal(G0)records,96 of the first grade(G1),99 of the second grade(G2),and 100 of the third grade(G3).

Figure 2:Images of normal samples from the IPC and AMC datasets at different magnifications

The pathologist followed the eighth edition of the manual for tumor node metastasis(TNM)defined by the American Joint Committee on Cancer(AJCC)for the preparation and ground truth labeling of IPC and AMC datasets[37].The images of GlaS and IMEDIATREAT datasets were labeled as normal,well,moderate,and poor,respectively,and resized to 640 × 480 resolution to maintain the uniformity of the images and labels across the four datasets.Fig.2 shows normal colon biopsy images acquired from the IPC and AMC datasets at various microscopic magnifications,providing an understanding of how the colon biopsy images vary across different magnification and staining conditions.

3 Proposed Methodology

The schematic framework of the proposed colon cancer grading framework comprises three modules:(i)preprocessing,(ii)feature extraction,and(iii)classification,as shown in Fig.3,which is discussed in detail in the following subsections.

Figure 3:Block diagram of the proposed framework

3.1 Pre-Processing Module

In the first phase of preprocessing,stain normalization[5]and contrast enhancement[38]are conducted to increase image quality.As the input images are from different datasets and slides that undergo distinct staining and illumination conditions,stain normalization is performed,wherein there is a reference image(chosen by the expert pathologist)to which all other images need to be stain-normalized.Fig.4a shows the input image that has to be stain-normalized with respect to the reference image(Fig.4b)and stain-normalized image(Fig.4c).Thus,post stain normalization,all input colon biopsy images are further contrast-enhanced.Later,for extracting texture features,stain-normalized contrast-enhanced images are converted to grayscale.

Figure 4:Stain Normalization:(a)raw image;(b)reference image;and(c)normalized image

The components of colon biopsy images are typically distinguished as nuclei in purple color,connecting tissues in pink,and the epithelial and lumen in white color[9–11].Therefore,to obtain these clusters,K-means clustering[39]was performed on stain-normalized contrast-enhanced images with K = 3.The white cluster obtained from K-means is considered for morphological feature extraction as the lumen and epithelial cells constitute the geometric parts and undergo distortion as the cancer grade progresses[10,11].Fig.5a shows the preprocessed image that undergoes K-means clustering and results in the pink(Fig.5b),purple(Fig.5c),and white clusters(Fig.5d).

Figure 5:K-means clustering,K = 3:(a)preprocessed image;(b)pink cluster;(c)purple cluster;and(d)white cluster

3.2 Feature Extraction Module

The variation in texture and color across various magnified images and grades of cancer must be captured using a proper feature set.In the feature extraction phase,the various features extracted from the image were combined to form a novel,rich hybrid feature set to categorize the colon images into four classes.Three significant extracted features are texture,color,and morphology.The texture feature vector,including cartoon texture features,Gabor wavelet,and wavelet moments,is extracted from the grayscale preprocessed image,whereas color features such as HSV histogram,color auto-correlogram,and color moments are extracted from the preprocessed stainnormalized contrast-enhanced image.The morphological features are extracted from the white cluster obtained post-K-means clustering.These feature vectors are then unified to form a rich hybrid feature set grading colon biopsy images at various magnifications.

3.2.1 Texture Feature

The preprocessed grayscale image was used to extract the following texture features.

?Cartoon Texture Feature:These features primarily contain geometric parts,such as piecewisesmooth regions and edge contours on a large scale.They utilize both local and nonlocal systems,which can exploit similar patches for textures’sparse representation.More rich features are required when malignancy changes according to grade and microscopic magnifications,in which case cartoon features provide better edge detection quality.Further,cartoon features extract more detailed texture by considering the difference between the original image and its cartoon component.As the different grades differ in the structures,the structural deformities could be measured irrespective of the magnification with cartoon texture features as the images are decomposed in the temporal domain.Thus,the cartoon imagec(x)and texture imaget(x)are obtained from Eq.(1)for an imageIfor every pixelx[40].

3.2.2 Color Features

Features that can capture variation in the color of the images of healthy and malignant colon color cells are essential.The following color features are extracted for the proposed framework.

?HSV Histogram:As the color composition varies for different grades of the colon biopsy images,a color model aims to generalize and standardize the representation of colors in these images.Hence,an image pixel value is converted from the RGB representation to HSV using the formula given in Eq.(4).

Formally,the color histogram is defined ashH,S,V[a,b,c]=N,prob{H=a,S=b,V=c},whereH,S,andVrepresent the color bands in the chosen color space(HSV),andNrepresents the number of dots in the image.The dimension of the histogram was reduced via the Kherfi et al.[43]solution.The color space was deconstructed into 27 subspaces by dividing each color strip’s intensities into three equal parts.The result is a vector of only 27 cells.

?Color Auto-correlogram:This three-dimensional histogram characterizes the color distribution and spatial correlation between color pairs.The first and second dimensions of the histogram represent the colors of any pair of pixels,and the third dimension represents the spatial distance between them[44].A color correlogram can be treated as a table indexed by color pairs,where thekthentry for(i,j)specifies the probability that a color pixeljis at a distancekfrom another color pixeliin the image.LetHbe the set of pixels of an image andHc(j)be the set of pixels of colorc(j);then,the image’s correlogram is defined as in Eq.(5).

where,i,j∈{1,2,3,...,N},k∈{1,2,3,...,d} and |p1?p2|is the distance between pixelsp1andp2andpris the probability function.The extracted color auto-correlogram feature vector is of length 64.

?Color Moments:If the value of theithcolor channel at thejthimage pixel isIij,and the number of pixels isN,then the index entries related to this color channel and the color modelrare known as the color moments defined as in Eq.(6)[11].

hereEr,i(1 ≤i≤3)presents the average color(mean)of the regionr;σr,irepresents the standard deviation of the color model r and the extracted color features are given by the feature vectorfc={E1,1,σ1,1E2,2,σ2,2E3,3,σ3,3................Er,i,σr,i}.Color moments are thus extracted for the RGB and HSV color model and the feature vector is of length 12.

The three-color features,when concatenated,yield a feature-length of 103.

3.2.3 Morphological Features

These features are extracted to quantify the shape of the white cluster components because grading affects this cluster,wherein the distortions become severe as the grade progresses.These features are extracted from the white cluster’s binary form obtained after K-means clustering[10].Morphological operations,erosion,and dilation were performed on the cluster,and connected components were identified.Based on these connected components,morphological descriptors such as area,perimeter,eccentricity,Euler number,extent,orientation,compactness,and major and minor axis lengths are tabulated.The average morphological values were then determined using all connected cluster components[9],where the morphological features were of length 9.

A rich hybrid feature set is generated by concatenating all individual features with 692 as the feature-length from all the extracted texture,color,and morphological features.

3.3 Classification Module

The generated hybrid feature set was formulated via 10-fold cross-validation[45]and classified into four classes with ensemble RF optimized using the Bayesian optimization algorithm(BOA);majority voting was implemented to predict the samples.RF classifier is commonly used in medical applications due to its high predictive precision,management of input data at various scales,and its ability to decrease overfitting features[46–48].Hyperparameter tuning with Bayesian reasoning aid will minimize the time taken to achieve the optimal parameters and yield better results in test set generalization[49].

Hyperparameter tuning of an RF of decision trees is achieved using quantile error(QE),a parameter tuned for minimizing the classification error.It is required for multidimensional data such as histopathological images and Bayesian optimization[50,51].θ1....,θnare the hyperparameters of the decision tree,?1,...,and ?n,denotes the respective domains,andnrepresents the number of hyperparameters.The algorithm hyperparameter space is defined as ?=?1×...×?n.When trained withθ∈?on dataFtrain,the QE on dataFvalidisQE(θ,Ftrain,Fvalid).Usingkfold cross-validation,the hyperparameter optimization for the given datasetFis formulated to minimize as in Eq.(7):

Algorithm 1:Optimization algorithm of the Bayesian method(fF,N,?,θ1:n)Input:Target function fF;Limit N;Hyperparameter space ?;initial design θ1:t = 〈θ1,...θt〉Output:Best hyperparameter obtained θ?(1)For i ←1 to n do yi ←evaluate fF(θi)(2)For j ←n+1 toN do steps 3,4,5(3) M ←fit model on performance data 〈θi,yi〉j?1i=1(4)Select θj ∈argmax θ∈?a(θ,M)(5) yj ←evaluate fF(θj)(6)Return θ?∈argminθj ∈{θ1,...θN}yj

As described in Algorithm 1,Bayesian optimization begins with functionfatNvalues in the initial design and recording(input,output)pairs 〈θi,f(θi)〉i=1t.Then,it iterates the operation in three phases:(1)fit a probabilistic modelMto the considered(input,output)pairs;(2)use the probabilistic modelMto select a promising inputθto evaluate the next by quantifying the desirability of obtaining the function value at arbitrary inputs(θ∈?)through an acquisition functiona(θ,M);and(3)evaluate the function at the new inputθ.

The role of the acquisition functiona(θ,M)is trade-off exploration in hyperparameter regions where the modelMis uncertain with exploitation in regions with low predicted QE.The acquisition function’s expected improvement over the best input found thus far[46]is represented by Eq.(8).

Fig.6 visualizes the change in the objective function value versus the number of function evaluations for the Bayesian optimized RF.Therein,the objective function reaches its global minimum within 30 iterations at maximum.It reiterates the BOA’s efficiency in optimizing the considered algorithms.

RF parameters were optimized using the BOA.The training set was constructed using hybrid feature variables obtained using the proposed method.Before the RF model was trained,the RF parameters were determined,including the number of trees,ntree;the number of leaves per tree,nleaf;and the number of random variables used for each node split,mtry.If minimum classification loss is considered the goal,the number of decision trees can drastically increase.The two parameters were optimized to improve classification accuracy.If thenleafvalue is too large,it results in overfitting;if thenleafvalue is too small,it results in underfitting.The RF parametersnleafandmtrywere tuned using BOA and set withntree= 300,nleaf ?[1,20],andmtry?[1,10].The objective function of BOA is the QE.Fig.6 shows the objective function model and shows the relationship between function evaluations and the minimum objective.The optimized RF parameters were calculated asnleaf= 7 andmtry= 5,and the observed minimum of the objective function was 0.005.

Once the RF classifiers were optimized,determining the number of binary Bayesian optimized RF classifiers was important for appropriate four-class classification,as shown in Fig.3.Hence,there is a need to build theBayesian optimized RF classifiers:one classifier to distinguish each pair of classesiandj,whereNis the number of classes.Letfijbe the classifier where classirepresents positive examples and classjrepresents negative examples,wherefji= ?fijclassify usingIf the binary classification models predict a numerical class membership,such as a probability,then theargmaxof the sum of the scores,which is the class with the largest sum score,is predicted as the class label.

Figure 6:Bayesian optimized random forest(a)objective function model and(b)minimum objective vs. number of function evaluations

4 Results

In this section,first,the performance measures used to evaluate the proposed framework are discussed.Later,the results of the proposed framework are analyzed at five levels.

4.1 Performance Measures

The proposed system is quantitatively evaluated based on performance metrics such as accuracy,error rate,sensitivity,specificity,precision,false-positive rate,F-score,Mathew correlation coefficient(MCC),and kappa statistics described in Tab.1.Accuracy and error rate is measured in percentage,MCC varies from ?1 to +1,and rest all measures scale from 0–1(1 is best and 0 worst)[52].A 4×4 confusion matrix with true positive(TP),false positive(FP),true negative(TN),and false negative(FN)is used to tabulate the performance measures.

Table 1:Performance evaluation measures

The overall MCC is determined using the technique macro-averaging for a multi-class classification.Assume 1,2,3,and 4 are four categories that classify the samples.Then,with the 4×4 confusion matrix,TP,TN,FP,and FN are computed as:TP = TP1 + TP2 + TP3 + TP4;TN= TN1 + TN2 + TN3 + TN4;FP = FP1 + FP2 + FP3 + FP4;and FN = FN1 + FN2 +FN3 + FN4.The cumulative MCC is estimated using these values.

4.2 Experimental Results and Analysis

This section analyzes the efficiency of the proposed method through different datasets and examines the findings.The results of the proposed framework were analyzed in five phases.(1)The first phase of analysis included the evaluation of the magnification-independent framework across various datasets;(2)In the second step,the model was generalized for which evaluation was done using one dataset training and another dataset testing;(3)The third phase comprised the performance analysis of the proposed framework under each considered magnification;(4)In the fourth phase,the performance and interpretation of features were analyzed;and(5)In the fifth phase of analysis,the proposed framework was compared with existing techniques,on the benchmark datasets.

4.2.1 Performance of the Proposed Colon Cancer Grading Framework

The proposed four-class colon cancer grading framework was evaluated using four different datasets,including various magnifications.To evaluate the proposed framework’s magnificationindependent nature,for training and testing,colon biopsy images of various microscopic magnifications were considered from IPC(4X,10X,and 40X microscope magnifications)and AMC(10X,20X,and 40X microscope magnifications)datasets.Tab.2 summarizes the performance measures of the proposed model for different datasets.

Table 2:Performance evaluation measures of the proposed framework on different datasets

The four-class grading performed with the Bayesian optimized random forest classifier was most accurate for the IMEDIATREAT dataset,with 99.16% accuracy.In contrast,the GlaS,IPC,and AMC datasets were 97.58%,97.25%,and 94.40% accurate,respectively.The calculated MCC was highest for the IMEDIATREAT dataset,at 0.9894,and the F-score was also higher in the IMEDIATREAT dataset,at 0.9923.The AMC dataset had the lowest MCC value(0.9257).The IMEDIATREAT dataset was most accurate with the proposed system,and the average accuracy calculated for all datasets was 97.09%.Sensitivity is an essential measure in the medical field;hence,the proposed model yields better sensitivity values of 0.9725,0.9440,0.9807,and 0.9923 for IPC,AMC,GlaS,and IMEDIATREAT datasets,respectively.Thus,irrespective of various magnified images considered for training and testing with IPC and AMC datasets,the proposed framework is robust across magnifications and datasets.

The 4×4 confusion matrix obtained from the BO-RF ensemble classifier appears in Fig.7,where the rich hybrid feature set is used for the four-class classification.The confusion matrix of the IPC dataset(Fig.7a)demonstrates that TP for the normal class is 98.3%,and the class’s misclassifications have occurred with the well class.When considering the well class,95.7% constitute the TP,and the misclassifications happen with the normal and moderate classes.Similarly,for the moderate class,the misclassifications occur with the well and poor class with a TP of 97.3%.As the poor class structure is entirely different,its misclassification occurs with the moderate and has a TP of 98%.The class-wise analysis of TP for various grades:well(95.7%-IPC,91%-AMC,95.7%-Imediatreat),moderate(97.3%-IPC,94.3%-AMC,95%-GlaS,99%-Imediatreat),and poor(96%-IPC,95.7%-AMC,100%-GlaS,100%-Imediatreat)across datasets shows the robustness of the proposed grading irrespective of datasets and magnifications.Further,misclassification occurs with normal and well,well and moderate,and moderate and poor classes as their structure varies little between classes.The minimum misclassifications occur in the poor class as its structure is entirely different from the other classes.The number of FP and FN are minimum for all datasets,thereby boosting the sensitivity.The proposed model uses majority voting with six BO–RF,thereby reducing the misclassifications with an average accuracy of >96%.

Figure 7:Confusion matrix plot for the proposed model on different datasets.(a)IPC(b)AMC(c)GlaS(d)IMEDIATREAT

A receiver operating characteristic(ROC)analysis has been conducted;the corresponding results are presented in Fig.8.Each of the classes-normal,well,moderate,and poor,in every dataset demonstrate good ROC as the curve is toward the top left corner even though the respective class rankings vary.The IMEDIATREAT dataset exhibits better ROC for each class as all ROC curves are toward the top left corner.The ROCs across datasets reveal the robustness of the model across multiple magnifications and datasets.

Figure 8:Receiver operating characteristic plot of the proposed framework for four classes on different datasets.(a)IPC(b)AMC(c)GlaS(d)IMEDIATREAT

4.2.2 Performance of the Proposed Model with Training on One Dataset and Testing with Another

The proposed model was trained on one dataset and tested with another dataset and vice versa to assess the proposed model’s generalizability.Cross-training and testing ensure the prediction model’s performance using an unknown dataset,and the performance measures are illustrated in Tab.3.The proposed system was evaluated for different training and testing scenarios under all magnifications.The model was trained with the IPC dataset with all magnified images,and it was tested across the AMC,GlaS,and IMEDIATREAT datasets.The highest accuracy(95.80%)was observed on the IMEDIATREAT dataset,and the accuracy on the AMC dataset(91.43%)slightly outperformed that on the GlaS dataset(88.48%).As the training was performed with 4X,10X,and 40X images,IMEDIATREAT and AMC datasets containing 10X images exhibited considerable outperformance than other datasets,whereas the performance with the GlaS dataset was found to be on the lower side when 20X images were used for testing.Similarly,the AMC dataset comprising 10X,20X,and 40X images were trained with the system model and tested against the IPC,GlaS,and IMEDIATREAT datasets.When tested,the IPC dataset exhibited the highest accuracy(94.42%)as the testing contained 10X and 40X images,followed by the GlaS(92.73%)and IMEDIATREAT(91.60%)datasets,in that order.GlaS and IMEDIATREAT datasets have shown comparable results as their magnifications were used for training.Subsequently,the GlaS dataset was trained and tested against the IPC,AMC,and IMEDIATREAT datasets.The highest accuracy was 89.88% for the AMC dataset,whereas the IPC dataset yielded a lower accuracy of 86.08%.Compared to other datasets,when trained with the GlaS dataset,the test datasets’performance dipped because of the training image sets being few,single magnified,and imbalanced images across the four classes.When the IMEDIATREAT dataset containing 10X images was used for training,and IPC,AMC,and GlaS datasets were used for testing,the highest accuracy was achieved for the GlaS dataset(94.55%)because it contained a single magnification while other datasets contained multiple magnifications for testing.

Analyzing the overall statistical measures for the cross-training and testing outcome from Tab.3 indicates the model’s generalization capability across various datasets.When trained with the IPC dataset,the average accuracy was 91.90%.Similarly,when trained with the AMC dataset,the average accuracy was 92.91%,and when trained with GlaS,the average accuracy was 88.16%;IMEDIATREAT yielded an average accuracy of 92.18%.

4.2.3 Performance Analysis of the Proposed Framework under each Magnification

To determine the supremacy of the proposed framework,the analysis under each magnification was performed for IPC and AMC datasets.The model was also tested for cross-training and testing under each magnification across datasets for generalizability.

The proposed magnification-independent framework was evaluated for each magnified image in IPC and AMC datasets.The respective magnified images were considered for training and testing to analyze each magnification’s proposed model’s performance.Tab.4 illustrates the calculated accuracy,i.e.,94.25%,96.50%,and 97.50% for the IPC dataset for image magnifications of 4X,10X,and 40X,respectively.For the IPC dataset,40X magnification provides higher accuracy than lower magnifications,whereas,in the AMC dataset,a lower magnification of 10X provides higher accuracy(98.57%).F-Scores of 0.9425,0.9650,and 0.9749 and 0.9857,0.9643,and 0.9447 are observed on the IPC and AMC datasets for 4X,10X,and 40X and for 10X,20X and 40X magnifications,respectively.For the IPC dataset,MCC at 40X was 0.9668,and it was 0.9810 in the AMC dataset at 10X magnification.The difference in data acquisition,lighting,and staining conditions can cause variation in the feature responses,thereby affecting performance across magnifications.

Tab.5 demonstrates the proposed model’s performance accuracy when trained and tested with independent datasets at different magnifications.The model is trained with one particular magnified image of a dataset and tested with other datasets’same magnified images.The crosstraining and testing accuracy when the IPC dataset at 10X magnification was trained and tested with IMEDIATREAT was 97.20%,whereas training with IMEDIATREAT and testing with the IPC dataset at 10X magnification yields a lower accuracy(94.50%)than that obtained using the earlier dataset.Similarly,if trained with the AMC dataset at a 20X magnification and tested with GlaS,the system’s accuracy was 91.52%,and when the same process was reversed,the accuracy improved to 97.14%.The performance variation is caused by the difference in image acquisition,quality,and staining properties.When comparing the same magnification,such as 40X and training with the AMC dataset,testing with the IPC dataset achieved an accuracy of 97.25%.When training and testing were conducted vice versa,the accuracy of the system was 94.44%.Thus,even with regards to magnifications when the independent datasets are sampled for testing and training,the performance is comparable and demonstrates the model’s robustness.

Table 3:Performance measures of the proposed grading framework with cross-training and testing

4.2.4 Performance and Interpretation of Features

A quantitative and qualitative evaluation of the proposed framework for individual and combined features with accuracy and F-score is shown in Figs.9a and 9b,respectively.For the IPC,AMC,GlaS,and IMEDIATREAT datasets,the proposed rich hybrid feature set’s average accuracy was 97.25%,94.44%,97.58%,and 99.16%,respectively,at the higher side.When individual features were analyzed,the cartoon feature yielded the highest accuracy for the IPC(94.40%)and IMEDIATREAT(97.22%)datasets,and for AMC and GlaS,the highest contributing features varied.Color-moment-based features exhibited a lower accuracy of fit(86.11%)for the IPC and AMC datasets.For the GlaS and IMEDIATREAT datasets,morphological features and wavelets exhibited the lowest system performances of 86.21% and 89.94%,respectively.The texture features contributed more than other features across all datasets for the grading,with accurate data fits of 96.55%,93.10%,95.59%,and 96.55% for the IPC,AMC,GlaS,and IMEDIATREAT datasets,respectively.The individual accuracy and F-score for color and morphological features were higher when considered separately rather than when they were combined.An accurate data fit of 90.83%(IPC),86.11%(AMC),90.70%(GlaS),and 89.60%(IMEDIATREAT)was found for the combination of color and morphology,which was lower than that obtained when the color and morphology features were considered separately.The texture feature combined with the morphological feature provided the next contributing features with accuracies of 95.59%,92.86%,94.85%,and 94.17% across the IPC,AMC,GlaS,and IMEDIATREAT datasets,respectively.Accuracy levels dropped when texture and color were combined.Thus,features,when concatenated,boost accuracy by 1%–3%.The accuracy and F-score achieved for the proposed hybrid feature are higher for all datasets when compared with the individual features.

Table 4:Performance evaluation of the proposed framework across different magnifications for the IPC and AMC datasets

Table 5:Cross-training and testing accuracy for different magnifications

Fig.10 illustrates the mosaic plot for the different feature set distributions extracted from different datasets across magnifications.The feature distribution was plotted for the IPC,AMC,and IMEDIATREAT datasets at 10X magnification,the AMC and GlaS datasets at 20X magnification,and the IPC and AMC datasets at 40X magnification.Different grades of colon images yielded variation in the extracted features.In IPC,the healthy colon images showed less variation than other grades.The cartoon features are less sensitive toward magnification variation.They exhibited a symmetrical structure in the mosaic plot for 10X,20X,and 40X magnifications for different colon cancer image grades.Morphological features changed at different magnifications.An evident difference existed in 10X,20X,and 40X magnifications for different grades in different datasets.The above mosaic plots indicate feature variation for different grades for colon cancer analysis.Thus,the proposed hybrid features provide a rich classifier platform for better classification of the four-class cancer grading framework across multiple image sources.

Figure 9:(a)Accuracy and(b)F-Score for individual features and feature combinations on the proposed framework

The hybrid feature distribution across datasets in the boxplot from Fig.11 shows the system’s performance with cross-training and testing.First,the proposed system’s hybrid feature is less skewed than other features.Skewness indicates that the data may not be normally distributed.Hence,the extracted hybrid feature has a stable distribution of data for the classifier as a training sample.Second,the IPC and AMC datasets are less skewed in the hybrid-feature-based plot.The median range is in the same range for hybrid features,ranging from 0.056 to 0.070.The IMEDIATREAT dataset variation is more favorable than those in the IPC,AMC,and GlaS datasets for the hybrid features.Thus,the median weights of the notch plots are nearly similar.

Thus,none of the features are individually adequate to separate the four classes;however,multivariate examination through machine learning precisely categorizes normal,well,moderate,and poor classes.

4.2.5 Comparison of the Proposed Model with Existing Techniques

The proposed framework’s performance is compared with existing techniques in two aspects,i.e.,comparing the activation features extracted from the existing CNN models for four-class classification and comparison with existing techniques on two benchmark datasets.

Figure 10:Feature distribution across different magnifications for different datasets

Figure 11:Boxplot for Hybrid feature distribution across various datasets

In the literature[22,23,25],histopathological images were trained over existing CNN models and activation features were extracted for classification as there is scarce annotated medical data,and training from scratch requires extensive data.Commonly used existing CNN models on histopathological image data such as Alexnet[25],VGG-16[53],Inception v3[54],and Inception-Resnet v2[55]are trained on the various colon image datasets to extract high-level features to classify the images into four classes,normal,well,moderate,and poor,with the Bayesian optimized RF classifier,and the comparison with the proposed magnification-independent model is illustrated in Fig.12.The analysis shows that the proposed framework performs better than other CNN models across all datasets regarding the accuracy,sensitivity,specificity,F-score,and MCC.The high-level features extracted from the CNN models are generic features that are not specifically extracted to perform on various image magnifications and grades.The proposed robust hybrid features are meant to extract the varying texture,color,and geometric features across multiple image magnifications and grades.Inception-Resnet v2 is the best CNN model across IPC,AMC,and GlaS datasets,whereas Alexnet performs better on the IMEDIATREAT dataset.System performance across the CNN models differs as the number of levels differs in each of the networks chosen;consequently,system performance varies across the datasets.The proposed magnification-independent multiclass grading framework is a generalized framework that can work across four colon image datasets with multiple magnifications.

Figure 12:Comparison of the proposed colon cancer grading model with existing CNN models on feature learning on different datasets.(a)IPC(b)AMC(c)GlaS(d)IMEDIATREAT

Table 6:Comparison of the proposed grading framework with existing techniques on the GlaS and IMEDIATREAT datasets

Comparative analysis of the proposed framework with existing techniques on the benchmark datasets,i.e.,GlaS and IMEDIATREAT datasets,is illustrated in Tab.6.The proposed model is a four-class magnification-independent colon cancer grading framework evaluated on four different datasets with various magnifications;the accuracy of 97.58% and 99.16% were obtained for GlaS and IMEDIATREAT datasets,respectively.The performance of the proposed magnificationindependent framework on GlaS dataset has surpassed previous studies[24]and[29].However,the method presented in[17]exhibited slightly better accuracy(98.60%)than that of the proposed method because the segmentation performed is meant to work on specific magnified images(10X and 20X),and a three-class grading classification has been performed.Thus,the gland features extracted from these segmented regions are also dependent on segmentation outcomes,which are subsequently appropriate for a specific magnification and may not perform well for other low or high magnifications.Moreover,the proposed magnification-independent four-class grading framework shows better sensitivity(0.9807),specificity(0.9907),and MCC(0.9780)than the sensitivity(0.9730),specificity(0.9900),and MCC(0.9640)achieved in[17]and evaluating only the accuracy would be a biased decision.For the IMEDIATREAT dataset,the study presented in[30]developed segmentation with intensity-based thresholding,and morphological features were extracted for four-class grading to attain 89.75% accuracies;another study presented in[25]classified images into four-class using a tandem of classifiers with extracted deep CNN features on Alexnet and attained an accuracy of 98.29%.These accuracies are lesser than those acquired using the proposed method(99.16%)and evaluated under the same datasets.Sensitivity and F-score values of 0.9923 also show the proposed model’s supremacy over existing techniques on the IMEDIATREAT dataset.Thus,the proposed magnification-independent colon cancer multiclass framework is a generalized framework over multiple datasets and magnifications.

As the images are from different image sources acquired through different staining conditions,the proposed method stain normalizes images,making them uniform across datasets.The proposed framework is modeled as a magnification-independent framework evaluated to work when trained respective or irrespective of magnifications and classifies any input samples as crosstraining,with testing performed across magnifications.Thus,the proposed colon cancer grading method is an effective,generalized system with an average accuracy in the range of 94.40%–99.16% across four different datasets from different country locations and various magnifications(4X,10X,20X,and 40X).

5 Discussion

The proposed grading model demonstrated accurate four-class grading of colon cancer samples as an automated computational prototype.This research focuses on extracting various features,such as morphology,texture,and color for different colon image magnifications.The experimental analysis was conducted on various datasets,and the calculated outcome was satisfactory and superior to that presented in the literature.The proposed hybrid features are intended to extract all possible features for the four classes.The multi-feature-based classification method yielded better results than the individual-feature-based classification methods.Further,the proposed RF classifier hyperparameter was optimized using Bayesian optimization,which is more accurate than the traditional method.A one-vs-one strategy was adopted,ensuring an accurate outcome for multiclass classification to achieve consistent classification modeling for four-class grading.There are various advantages to our proposed system model over existing techniques.First,the proposed framework is a magnification-independent model that can work with any magnification of colon samples.Second,this algorithm requires no training and can be applied without a pre-trained model to any new specimen.Finally,the process does not require complex hardware and can be performed on desktop computers using any processor.

In particular,our approach has achieved great precision for the four-class colon cancer grading(IMEDIATREAT = 99.16% GlaS = 97.58%,IPC = 97.25% and AMC = 94.40%).Notably,the most discriminatory features emphasized by the proposed,containing cartoon features,Gabor features,color features,and morphological features,are the dominant features used to grade colon cancer samples’malignancy.When the features were considered cumulative as a hybrid feature set,the model was less sensitive towards different magnifications and could grade the colon images more precisely.

The model was trained on a dataset from one source and tested on a dataset from another source to ensure that the proposed model was suitable for various data sources.Previous studies focused on the three-class grading of colon cancer[17,24,29]for the GlaS dataset.The results from testing the proposed grading method(Tabs.2–5)support the four-class grading system and evidence the framework’s efficiency.The proposed classification and feature combinations herein provide a novel,reliable categorization of colorectal cancer image datasets from various sources irrespective of magnifications.The proposed model performs for any dataset input image even if it is not included in the training sample.Our proposed technology assessment shows strongly that our model functions well in typical clinical contexts where dataset samples are more varied than in controlled laboratory environments.However,the proposed method lacks the precise geometric tabulation of the cells across different grades as it is meant to work on different magnifications.The imbalanced dataset images and noise variations in the images can deteriorate the performance of the model.

6 Conclusion

The presented work proposes a magnification-independent colon cancer grading framework with a hybrid set of features,i.e.,texture,color,and morphological features,and classifies images into four-class colon grades:normal,well,moderate,and poor.The proposed colon cancer grading framework includes a preprocessing phase comprising stain normalization,contrast enhancement,grayscale conversion,and K-means clustering to enhance the image quality and normalize the images across multiple datasets.The rich information regarding the image texture,edges,and structures across magnifications and grades are extracted from the texture features,including the cartoon features,Gabor wavelets,and wavelet moments.The color distribution across various grades was quantified with the color feature set comprising the HSV histogram,color autocorrelogram,and color moments.Morphological features extracted from the white cluster obtained through K-means clustering quantified the geometric variations across magnifications and grades.All extracted features were concatenated to create a rich,hybrid feature set for classification using majority voting on six Bayesian optimized RF classifiers.The experiments were conducted on four datasets with different magnification factors:IPC(4X,10X,40X),AMC(10X,20X,40X),GlaS(20X),and IMEDIATREAT(10X)to analyze the robustness of the proposed system model,wherein the IMEDIATREAT dataset calculated the highest accuracy of 99.16% followed by GlaS(97.58%),IPC(97.25%),and AMC(94.40%)datasets.Multiclass classification with optimized RF ensures the optimal accuracy of the proposed system.The proposed grading system was evaluated under various validation structures for generalizability and cross-training,and testing it as an independent model displayed promising results.In the future,magnification-independent segmentation can be implemented for grading and used to calculate and compare clinical results.

Acknowledgement:The authors thank Aster Medcity(Kochi,India)and Ishita Pathology Center(Allahabad,India)for the research’s involvement by providing quality images that are applaudable.The continuous support of Dr.Ranjana Srivastava(Ishita Pathology Center),Dr.Sarah Kuruvila(Former Senior Consultant,Aster Medcity,Kerala),and Dr.Jyotima Agarwal is highly commendable.At the time of dataset image collection,Dr.Shahin Hameed was part of Aster Medcity and has extended his valuable support throughout the research.

Funding Statement:This work was partially supported by the Research Groups Program(Research Group Number RG-1439-033),under the Deanship of Scientific Research,King Saud University,Riyadh,Saudi Arabia.

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.