Fusion-Based Machine Learning Architecture for Heart Disease Prediction

Muhammad Waqas Nadeem,Hock Guan Goh,Muhammad Adnan Khan,Muzammil Hussain,Muhammad Faheem Mushtaq and Vasaki a/p Ponnusamy

1Faculty of Information and Communication Technology(FICT),Universiti Tunku Abdul Rahman(UTAR),Kampar,Perak,31900,Malaysia

2Department of Computer Science,Lahore Garrison University,Lahore,54000,Pakistan

3Department of Computer Science,Faculty of Computing,Riphah International University,Lahore Campus,Lahore,54000,Pakistan

4Department of Computer Science,School of Systems and Technology,University of Management and Technology,Lahore,54000,Pakistan

5Department of Information Technology,Khwaja Fareed University of Engineering and Information Technology,Rahim Yar Khan,64200,Pakistan

Abstract:The contemporary evolution in healthcare technologies plays a considerable and significant role to improve medical services and save human lives.Heart disease or cardiovascular disease is the most fatal and complex disease which it is hardly to be detected through our naked eyes,as numerous people have been suffering from this disease globally.Heart attacks occur when the ranges of vital signs such as blood pressure,pulse rate,and body temperature exceed their normal values.The efficient diagnosis of heart diseases could play a substantial role in the field of cardiology,while diagnostic time could be reduced.It has been a key challenge for researchers and medical experts to diagnose heart diseases accurately and timely.Therefore,machine learning-based techniques are used for the diagnosis with higher accuracy,using datasets compiled from former medical patients’reports.In recent years,numerous studies have been presented in the literature propose machine learning techniques for diagnosing heart diseases.However,the existing techniques have some limitations in terms of their accuracy.In this paper,a novel Support Vector Machine (SVM) based architecture for heart disease prediction,empowered with a fuzzy based decision level fusion,is presented.The SVMbased architecture has improved the accuracy significantly as compared to existing solutions,where 96.23%accuracy has been achieved.

Keywords:Heart disease;machine learning;support vector machine;fuzzy logic;fusion;cardiovascular

1 Introduction

Heart disease (HD) is a serious health issue around the world and numerous peoples are affected by this disease [1].The most common symptoms of HD are physical body weakness,breath shortness,and swollen feet [2].In recent years,many researchers present various machine learning methods and techniques for early prediction of heart disease but the existing diagnostic techniques for heart disease are not efficient and effective due to several reasons such as execution time and accuracy of the machine learning models [3].Due to the unavailability of a medical expert and modern technology,the diagnosis and treatment of heart disease are difficult to be carried out appropriately [4].The life of numerous people can be saved by using effective and accurate diagnostic technologies [5].According to the European Society of Cardiology,there are 3.6 million people have diagnosed as HD patients annually around the world [6,7].Most of the people in the United States (US) are affected by HD [8].Approximately 50% of heart patients that are suffering from heart disease can survive within 1-2 years,and 3% of the financial healthcare budget is used for the management of heart disease [9].Traditionally,the physician use concerning symptoms,patient medical history,and physical examination reports for the diagnosis of heart disease.The results obtained from these methods are not effective and accurate for the identification of HD patient.Moreover,these methods are computationally difficult and expensive [10].

The development of machine learning-based noninvasive diagnostic systems is needed for effective diagnosis of HD [11-16].Machine learning-based expert decision systems and applications of Artificial Fuzzy Logic (AFL) efficiently diagnose the heart disease patient that results in the decreases in death ratio [17,18].The Cleveland heart disease data set used by several researchers [19-24]for the prediction of HD.The proper data are required by the predictive machine learning models for their training and testing.The use of balanced data set for the training and testing improves the performance machine learning model.Furthermore,the use of proper and related features from the data set improves the predictive capabilities of the model.Hence,features selection and data balancing are key parameters to improve the performance of the model.In the literature,several machine learning-based diagnostic methods and techniques such as Neuro Fuzzy,Artificial Neural Network (ANN),Support Vector Machine (SVM),Decision Tree (DT),Na?ve Bayes (NB) etc.have been proposed by researchers,but these techniques have some limitations that include lack of large training data,inconsistency accuracy,proper data balancing,and so on.Furthermore,these techniques do not effectively diagnose heart disease.The data standardization at the data processing layer also improves the predictive capabilities of the machine learning models.Since more,some other preprocessing techniques that include Min-Max Scalar,removal of missing features from the dataset,and standard scalar improve the performance of the model [20].Several features selection techniques such as Principle Component Analysis (PCA),Local Learning-Based Features Selection (LLBFS),Greedy Algorithm (GA) etc.are used for the selection of important parameters.Furthermore,several optimization techniques that include Bacterial Foraging Optimization (BFO),Ant Colony Optimization (ACO) and so on also used for the optimization of features before training of machine learning models [25].

Furthermore,in recent years,several machine learning algorithms such as ANN,SVM,K-Nearst Neighbour (KNN) etc.are used in the Internet of Things (IoT) based systems for prediction and classification [26].The unsupervised machine learning algorithms are used to label the data which is collected by the different IoT devices.The data which is labeled by the machine learning algorithms gives more accurate results as compared to manual labeling.

Hence more,Neural Networks based tools achieved state-of-the-art performance for the prediction of brain and heart diseases.In recent years,Carotid Artery Stenting (CAS) treatment is commonly used in the field of medicine.The CAS methods give an overview of the Major Adverse Cardiovascular Events (MACE) of the HD patients at an early stage.The ANN produces more accurate results as compared to the simple CAS method [27].The proposed ANN-based methods do not only combine posterior probabilities but also produce vales from multiple predecessor techniques.The ANN-based methods achieved much better results as compared to existing methods [28].

In this paper,supervised machine learning architecture empowered with fuzzy-based decision level fusion medical expert system is proposed for the prediction of heart disease.The proposed architecture consists of two phases:supervised machine learning phase and fuzzy-based decision level fusion phase.The main objective of this proposed architecture is to improve the accuracy of machine learning-based solution for the diagnosis of heart disease.Furthermore,in recent years,many studies restrict the use of feature selection methods for the model.Therefore,the proposed model working on the mechanism of parallel computation that allows us to use all the features without any restriction of feature selection method at the pre-processing layer.The experiment results show that the proposed architecture has effective results in terms of accuracy for the diagnosis of heart disease as compared to existing machine learning methods.

The rest of the paper is organized as follows:Section 2 describes the related work.Section 3 presents the materials and methods for the diagnosis of heart disease.Section 4 discusses the simulation results of the proposed architecture.Section 5 concludes the study.

2 Related Work

In the literature,numerous machine learning-based medical expert systems were designed by the researchers for the diagnosis of heart disease.This paper gives an overview of some existing machine learning-based diagnosis systems for heart disease and highlights the importance of the proposed work.ANN-based diagnostics models give the highest prediction accuracy in the domain of healthcare [27].Similarly,Big Data and Optimal Artificial Neural Network (OANN)based model is presented in [29]that achieved the 90.91% prediction accuracy.Kaggle and UCI laboratory heart disease data sets have been used by the researchers to discover the patterns using different machine learning algorithms such as DT,ANN,NB,and SVM.The hybrids methods give more accuracy as compared with a single machine learning algorithm [30].

Furthermore,numerous machine learning-based noninvasive medical support systems such as ANN,SVM,DT,NB,KNN,Logistic Regression (LR),Fuzzy logic (FL),Adaboost (AB)are developed by the researchers in the recent years for the diagnosis of heart disease [18,31].The use of machine learning-based medical expert systems for the diagnosis of heart disease gradually increases which decreases the death ratio of heart patients [32].Several machine learning-based medical expert system for the diagnosis of HD has been reported in numerous scientific research studies.

Support Vector Machine and Principal Component Analysis (SVM-PCA) based system is present in [33]which achieved 88.24% classification accuracy.Another SVM based model is presented in [34]to predict the risk of heart disease and achieved 89.9% accuracy.In [35],ANN and Neuro Fuzzy based predictive model for heart disease that obtained 87.04% accuracy was presented.Olaniyi et al.[36]presented a three-phase ANN model to diagnose heart disease and achieved 88.89% classification accuracy and the proposed system easily deployed in health care information systems.Another Ensemble-based ANN predictive model is also presented in [29]which used statistical analysis technique for the diagnosis of heart disease and obtained 89.01%accuracy,95.91% specificity,and 80.09% sensitivity.Furthermore,ANN and Fuzzy Analytical Hierarchical Processing (F-AHP) based integrated decision support medical system is presented in [37]that achieved 91.10% classification accuracy.

3 Materials and Method

The section briefly describes the research method and materials of the paper.

3.1 Dataset

Two different heart disease datasets are used in this paper to train the supervised machine learning algorithm.The first “heart disease dataset 2019,”which is used by various researchers [13]in recent years for the diagnosis of heart disease.The “heart disease dataset 2019” is also publically available on the online Kaggle repository.The heart disease dataset has 1025 number of samples,13 features,and some missing values.The target output label has two classes that represent the patient is normal or heart patient.The second “cardiovascular disease dataset 2019”is also used in this paper.The cardiovascular disease dataset 2019 is also available on the online Kaggle repository.The cardiovascular disease dataset 2019 has 70,000 number of patient samples,11 unique features,and some missing values.The detailed description of these datasets is given in Tabs.1 and 2.

3.2 Experimental Design Setup

The supervised prediction experiment has been conducted to evaluate the performance of the proposed architecture.First,we evaluate the performance of the Support vector machine (SVM)on two different data sets.The K-fold cross-validation method is applied to split the data.To access the performance of the architecture several performance evaluation metrics are computed.All the computation experiment has been performed in Python 3.7 environment using several machine learning libraries on an Intel?Core?i3-3217U CPU@1.80 GHz PC.

3.3 Proposed System Model

The proposed supervised machine learning architecture empowered with fuzzy-based decision level fusion is presented in Fig.1.The data set which is generated by the Internet of Medical Things (IoMT) enabled devices are used for the training of machine learning algorithms.The proposed architecture consists of two phases:the supervised machine learning phase and the fuzzy-based decision level fusion phase.The supervised machine learning phase has three distinct layers that include the pre-processing layer,application layer,and performance layer.The preprocessing layer receives raw data and maybe the raw data has some missing values and noise.At the pre-processing layer,different methods such as mean,mode,and average are applied for the prediction of missing values and remove the noise using normalization.Furthermore,the application layer receives the processed data and the processed data is used to train the supervised machine learning technique named SVM.The same mechanism is executed in parallel inside the proposed architecture.

3.3.1 Preprocessing

In the proposed architecture,the preprocessing step includes handling missing values,moving average,and normalization are describe as follows:

At the first step,the null and missing values are filled in the data set,because they can lead towards the wrong prediction of any machine learning model.In the proposed architecture,mean method is selected to impute the missing or null values because the mean method is beneficial as it impute continuous data without introducing outliers.The mean method is formulated as:

wherexrepresents the instances of feature vectors that lies in n-dimensional space,x∈R.

Table 1:Description of heart disease dataset 2019 with feature information

Table 2:Description of cardiovascular disease dataset 2019 with feature information

In Moving average (MA),to reduce the noise from the data set,a series of averages is computed of different subsets using full data set.Arithmetic mean of given set of values is taken to calculate the moving average.The moving average is formulated as:

wherex1,x2,x3,...,xnrepresents instances of the feature vector andNrepresents total number of attributes.

In normalization,standardization or Z-score normalization techniques is used to rescale the values of attributes.The standardization method normalize the distribution of data with zero mean and also reduce the skewness of the data distribution.The standardization is formulated as:

wherexis the instances of feature vectors with n-dimensional space,x ∈Rn.andσrepresent mean and standard deviation of attributes respectively.

Figure 1:The systematic diagram of proposed supervised machine learning architecture empowered with fuzzy based decision level fusion

3.3.2 K-Fold Cross Validation

The K-fold cross validation method is widely used by the researchers for the selection of machine learning model and estimation of classifiers error [28].In the proposed architecture,5-fold cross-validation is used to split the data set for the training and testing of SVM.The fold-1 is used to train and fine-tune of hyper-parameters in inner loop where grid search algorithm is employed [29].In outer loop (k times),the performance of the model is evaluated using test data.Since more,the data sets which are used for the training and testing of proposed architecture has imbalanced negative and positive samples.The stratified KCV is used to preserve the ratio of each class.The final performance of the model is evaluated by using the following formula:where M denotes the final performance metric for the classifiers andPn∈R,n=1,2,3,...,K represents the performance metric for each fold.

SVM algorithm is used for regression and classification.In SVM based models,the data points are categorized into groups,represent on the space and the points which have similar properties falls in same group.In linear SVM,the p-dimensional vector is considered for the given data and separated by maximum of p?1 planes that are known as hyper planes.These planes are used to separate the data space among different data groups to regression and classification.The mathematical representation of SVM is formulated as:

The equation of the line is described as:

In Eq.(1) ‘x’ is the slope of the line and ‘b’ is intersect,so a1?a2x+b=0

The Eq.(2) is derived from 2-dimensional vectors.The above equation is also applicable for any number of dimensions.The Eq.(2) is also called the hyper lane equation.

where

As we know that

In other tales of the Children and the Ogre Aarne-Thompson classification, the children do not necessarily encounter a witch. The villain104 may be a giant, ogre, or other monster. You can read more on the Tales Similar to Hansel and Gretel page.Return to place in story.

So,Eq.(3) can also be written as

For n-dimensional vectors,the dot product of the above equation is computed as:

Let,f=y(·a+b)

If sign(f)>0 mean the classification is correct and sign(f)<0 mean the classification is incorrect.If D is given dataset,then f is computed on a training dataset

We also compute the functional margin (F) of a dataset as:

Through the comparison of hyperplanes,the hyperplane which has the largest F will be selected.Where F is known as the geometric mean of the dataset.We need to find the optimal values ofand b for the selection of optimal hyperplane.

The Lagrangian function is

By using Eqs.(5) and (6) we get

After the substitution of Lagrangian function L we get

Thus,

Subject to

Due to the inequality of constraints,the Lagrangian function can be extended to Karushkuhn-tucker (KKT) conditions.Eq.(9) describes the complementary conditions of KKT.

where a?is the optimal point andαis a positive value.For other points,the value ofαis ≈0

So,

Eq.(10) describe the support vectors which are closest points to the hyperplane.

The value of b is computed as

Multiply by y on both sides of Eq.(12)

Then

In Eq.(14) ‘s’ represents the number of support vectors.These support vectors make the hyperplanes and then hyperplanes are used for prediction.

The hypothesis function is described as:

If the point is above the hyperplane then it will be classified as +1 class mean the HD found and if the point is below the hyperplane then it is classified as?1 class mean the HD does not found.

3.3.4 Fuzzy Based Fusion

After the training of the SVM,different evaluation parameters such as accuracy,sensitivity,specificity etc.are used to evaluate the performance of the proposed architecture.Once the process of performance evaluation is completed for both SVM individually then a fuzzy-based decision level fusion process is applied to integrate the performance of both SVM for the final decision as:

In Eq.(16) the FHD denotes the fusion-based heart disease prediction.The t-norm function for fuzzy-based fusion is defined as:

t:[0,1]×[0,1]→[0,1]

After the implementation of the t-norm function fuzzy-based fusion implication is applied as:

μQ6(s1,s2)=min[μFP1(s1),μFP2(s2)]

The Eq.(17) shows the relationship between SVM1 and SVM2:

Eq.(18) integrates the performance of SVM1 and SVM2 in crisp form is as follow:

Defuzzifier is a very important component of an expert system.It is the process of mapping the fuzzy sets to the crisp output.The center of gravity defuzzifier is used to get the final fused decision of the proposed architecture.The center of gravity defuzzifier specifies theas the center of the area covered by the membership function ofφf(shuō)or fuzzy-based decision level fusion,that is,

4 Results and Discussion

The proposed supervised machine learning architecture empowered with fuzzy-based decision level fusion has been applied to two different datasets.The proposed architecture working on the mechanism of parallel computing.Furthermore,the k-fold cross-validation method is used to split the dataset into different folds for the training and testing of the proposed architecture.Different evaluation metrics are used to access the performance of the architecture which are as follows.

The proposed architecture predicts the output as positive (+1) and negative (?1).The +1 indicates the presence of heart disease and?1 indicates that no symptoms of heart disease found in the patient.The performance of the proposed supervised machine learning architecture empowered with fuzzy-based decision level fusion using different statistical metrics are shown in Tab.3.In Tab.3 it is clearly shown that the proposed architecture achieved effective results during fold-5 cross-validation.The architecture achieved 96.23%,95.64%,94.36%,97.01%,3.7%,4.36%,and 2.99% in terms of accuracy,specificity,precision,sensitivity,miss rate,false positive ratio,and false negative ratio respectively.

Table 3:The performance of the proposed architecture

The comparison of proposed supervised machine learning architecture empowered with fuzzybased decision level fusion with existing methods is described in Tab.4.Different machine learning methods and architectures for the diagnosis of heart disease which is presented by the researcher in recent years such as Multilayer Perceptron combine with SVM (MLP+SVM),Hybrid machine learning-based diagnostic system,ANN combine with FL (ANN+FL),Hybrid Random Forest with a Linear Model (HRFLM),ANN combine with Fuzzy Analytical Hierarchy Process (AHP)(ANN+Fuzzy AHP) etc.are studied for the comparative analysis of proposed architecture.The accuracy performance metric is used to compare the performance of proposed architecture with existing methods in the field of heart disease.It is observed that the proposed architecture gives better results in terms of accuracy as compared to the other existing methods.

Table 4:Comparative analysis of proposed architecture with existing methods

5 Conclusion

The early diagnosis of heart abnormalities and information related to heart condition from raw health care data is very important which could help to save human lives in the long term.In recent years,machine learning methods and techniques have achieved effective performance to process raw data and give a novel and new discernment toward heart disease.The prediction of heart disease is an important and a challenging task in the field of medical.However,the mortality rate of heart disease can be significantly controlled if heart disease is diagnosed at an early stage and adopt preventative measures.Furthermore,different machine learning methods and techniques for the diagnosis of heart disease are presented in recent years.The existing machine learning methods have some limitations in terms of accuracy.Therefore,the proposed supervised machine learning architecture empowered with fuzzy-based decision level fusion has achieved 96.23% accuracy which is much better than the existing methods.The proposed work can be extended by using different machine learning algorithms such as Artificial Neural Network,Decision Tree,Random Forest etc.along with SVM.

Acknowledgement:Thanks to our families &colleagues who supported us morally.

Funding Statement:The author(s) received no specific funding for this study.

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.