A modified harmony search algorithm and its applications in weighted fuzzy production rule extraction*#

Shaoqiang YE, Kaiqing ZHOU?, Azlan Mohd ZAIN, Fangling WANG, Yusliza YUSOFF

1School of Communication and Electronic Engineering, Jishou University, Jishou 416000, China

2Faculty of Computing, Universiti Teknologi Malaysia, Skudai 81310, Johor, Malaysia

E-mail: shaoq_ye@163.com; kqzhou@jsu.edu.cn; azlanmz@utm.my; fanglingong@163.com; yusliza@utm.my

Abstract: Harmony search (HS) is a form of stochastic meta-heuristic inspired by the improvisation process of musicians.In this study, a modified HS with a hybrid cuckoo search (CS) operator, HS-CS, is proposed to enhance global search ability while avoiding falling into local optima.First, the randomness of the HS pitch disturbance adjusting method is analyzed to generate an adaptive inertia weight according to the quality of solutions in the harmony memory and to reconstruct the fine-tuning bandwidth optimization.This is to improve the efficiency and accuracy of HS algorithm optimization.Second, the CS operator is introduced to expand the scope of the solution space and improve the density of the population, which can quickly jump out of the local optimum in the randomly generated harmony and update stage.Finally, a dynamic parameter adjustment mechanism is set to improve the efficiency of optimization.Three theorems are proved to reveal HS-CS as a global convergence meta-heuristic algorithm.In addition, 12 benchmark functions are selected for the optimization solution to verify the performance of HS-CS.The analysis shows that HS-CS is significantly better than other algorithms in optimizing high-dimensional problems with strong robustness, high convergence speed, and high convergence accuracy.For further verification, HS-CS is used to optimize the back propagation neural network (BPNN) to extract weighted fuzzy production rules.Simulation results show that the BPNN optimized by HS-CS can obtain higher classification accuracy of weighted fuzzy production rules.Therefore, the proposed HS-CS is proved to be effective.

Key words: Harmony search algorithm; Cuckoo search algorithm; Global convergence; Function optimization; Weighted fuzzy production rule extraction https://doi.org/10.1631/FITEE.2200334 CLC number: TP181

1 Introduction

Swarm intelligence (SI) optimization is a probabilistic search algorithm that operates by simulating the cooperative behavior of social biological groups.It has always been one of the research hotspots in the artificial intelligence domain (Tang et al., 2021).There are many complex and challenging optimization problems in this area.The traditional mathematical optimization schemes, such as the quasi-Newton method and gradient descent method, can hardly solve large-scale complex optimization problems.Due to the limitation of traditional schemes, the SI algorithm,using its stochastic walk search characteristics in solving a variety of optimization problems, has become more and more popular and is widely used in various applications such as cyber-physical social systems and feature selection (Tu et al., 2019).Over the past few decades, many SI algorithms have been developed, including the universal gravitational force between objects which was inspired by the gravitational search algorithm (Wang YR et al., 2021).Further SI algorithms have been developed, including the memetic algorithm (MA) which is based on a simulation of cultural evolution (Zhao et al., 2021), the flickering behavior among fireflies which simulates the firefly algorithm (FA) (Jagatheesan et al., 2019),the artificial bee colony (ABC) algorithm which is simulated by bee division and cooperative behavior(Karaboga et al., 2014), the moth-flame optimization(MFO) by simulation of the special navigation of moths at night (Mirjalili, 2015), and the harmony search (HS) algorithm inspired by the improvisation process in which musicians repeatedly adjust the pitch of different instruments to create the most beautiful music (Geem et al., 2001).

HS has the advantages of fewer parameters,simple operation, and easy implementation compared with peer algorithms.However, due to the disadvantages of low optimization speed, relatively poor population diversity, and easy premature convergence,scholars have proposed various improvements to it.Valaei and Behnamian (2017) proposed a new parameter adjustment method, using a dynamic adjustment Taguchi method to improve the pitch adjusting rate(PAR) and to obtain an improved multi-objective HS algorithm.Ouyang et al.(2018) employed a perturbation strategy, key parameter adjustment, and a global selection strategy to balance the capabilities of exploration and exploitation in an amended HS (AHS).AHS was applied to the reliability problem of largescale systems of non-convex integer nonlinear programming.The search efficiency and convergence performance of the AHS algorithm were then evaluated for performance verification.Shaffiei et al.(2019)proposed a constrained self-adaptive HS with 2-opt(CSAHS-2opt) for solving the driver scheduling problem of the university shuttle bus (DSPUSB).The bandwidth (also called fret width) value was dynamically changed and determined based on the current solution of each driver every week and, when generating a schedule under some constraints, the proposed algorithm could obtain the optimal scheduling scheme.Li et al.(2020) proposed a global optimal adaptive HS algorithm (AGOHS) to obtain weighted fuzzy generation rules from datasets.This algorithm realizes rule extraction by optimizing the back propagation neural network (BPNN) and shows high rule extraction accuracy.Abbasi et al.(2021) used the mutation operation of differential evolution (DE) to replace the bandwidth to improve the local exploitation ability.At the same time, a differential-based HS (DH/best) algorithm was proposed through dynamic distance adjustment between the harmonies in the harmony memory (HM).Harmony was used to enhance search ability in the process of algorithm update.When optimizing the IEEE 118-bus and 57-bus systems, the DH/best algorithm achieved the lowest actual power loss, improved the power of voltage, and reduced the minimizing active power of generation units.Singh and Kaur (2021)embedded the HS algorithm in a sine–cosine algorithm (SCA) to make up for the poor global optimization and low convergence speed of the SCA and proposed a hybrid SCA with HS (HSCAHS).Al-Shaikh et al.(2023) integrated the hill climbing algorithm with HS, and proposed a hybrid harmony search contact tracing (HHS-CT) algorithm for social network contact tracing of the COVID-19 infection.The algorithm could accurately track contacts through social networks and prevent further spread of the epidemic.To overcome the uncertain factors of medical datasets and improve the efficiency of medical diagnosis,Mousavi et al.(2021) used the Taguchi method to adjust the parameters of HS, so that the algorithm could find the best rule in a fuzzy rule based system.Meanwhile, external cross-validation (CV) and internal CV were used to verify and classify the obtained rules to obtain better classification results.Zhu et al.(2021) proposed the K-density based spatial clustering of application with noise (K-DBSCAN) clustering concept.However, the DBSCAN clustering algorithm has difficulty in predicting appropriate clustering parameters; therefore, HS was used to optimize clustering parameters.The experimental results demonstrated that the K-DBSCAN optimized by HS has good clustering performance and high clustering accuracy when dealing with four different datasets.To optimize the dynamic parallel row ordering problem(DPROP) on the production line, Gong et al.(2021)proposed a hybrid tabu search and HS.The results showed that the algorithm obtained the best solution when solving the DPROP with 30–40 large components.To overcome the defects of the traditional HS,Gupta (2022) proposed a modified HS algorithm(MHSA).A new calculation formula was reconstructed for the search strategy, harmony memory consideration rate (HMCR), and PAR of HS, to enhance search efficiency and accuracy.The algorithm obtained the optimal value in engineering structure design and the performance was far better than those of other comparison algorithms.Costa and Fernandez-Viagas(2022) proposed a self-adaptive HS (SAHS) mechanism to solve the single machine scheduling problem with flexible/variable maintenance in a specific time window.

To sum up, although HS has made some achievements in theoretical and practical application research, there are some shortcomings of the HS algorithm, such as poor convergence accuracy and ease of falling into a local extremum state, in solving highdimensional problems (Qin F et al., 2022).Therefore,in this paper we propose an improved HS algorithm.The local search strategy and updating mechanism in HS are improved by introducing cuckoo search (CS)(Yang and Deb, 2009; Ye et al., 2022).The proposed algorithm guides the iterative direction of the HS algorithm, prevents the algorithm from falling into stagnation, and enhances its population diversity and convergence efficiency.The specific improvement methods are as follows:

At the optimization stage of HS, due to the randomness of the HS pitch disturbance adjusting method,the exploitation optimization ability is insufficient because of poor solution accuracy and low convergence speed.To improve the optimization ability of HS, an adaptive inertia weight is constructed based on the relationship of information transfer between individuals in HM.Then the optimal solution in HM is used to replace the bandwidth for ensuring that convergence is towards the best position closely and quickly.Thus, these strategies guarantee that the proposed approach has the property to improve the search accuracy and efficiency of local optimization of HS in the later iterations.

At the global exploration stage of HS, new individuals are randomly generated, which causes some defects, such as weak global search ability and poor population diversity.To enhance the global search ability and population diversity of HS, the CS operator is employed in the random generation of new harmony (1-HMCR) to expand and update the solution vector of harmony.In addition, the CS operator is used to select candidate individuals from the poor HM,which enlarges the number of alternative solutions and avoids the HS from falling into the stagnation state of a local optimum during updating HM, thus further improving the accuracy of HS.

To strengthen the adaptability of HS, HMCR and PAR are adjusted adaptively.The former increases linearly and the latter decreases linearly as the number of iterations increases.This ensures that HS can quickly search the global optimal solution in the search region and enriches the adaptability of HS in the search process.

Furthermore, to provide the feasibility and robustness of the proposed HS-CS algorithm (a modified HS with a hybrid CS operator), three theorems are presented and proved to reveal that HS-CS is a global convergence meta-heuristic algorithm.

Finally, two experiments are conducted to verify the feasibility and robustness of HS-CS.Twelve classic functions are selected as benchmarks to reveal the highlights of the proposed HS-CS algorithm.As in our previous work, HS-CS is also used to extract the weighted fuzzy production rules from a given dataset to demonstrate some highlights of the proposed HS-CS, such as high convergence speed, high precision, and self-adaptation ability.

2 Standard HS and CS algorithms

The inspiration, critical operations, and implementations of the standard HS and CS are discussed in detail in this section.

2.1 Standard HS algorithm

HS is a meta-heuristic algorithm that simulates the process of musicians to create the most beautiful music through repeatedly adjusting the tones of different musical instruments.The core operation is to obtain the next generation of harmony solution vectors based on HMCR, PAR, and random selection.The primary thinking of the entire HS could be summarized as follows:

First, harmony solution vectors which keep the same volume as harmony memory size (HMS) will be created randomly and be stored in HM.Moreover,the harmony solution vectors will be selected from HM with the probability of HMCR, and the new harmony solution vectors will be generated with the probability of 1-HMCR.

Second, the selected harmony solution vector from HM will be tweaked by the probability of PAR to generate a new harmony solution vector.Meanwhile, the selected harmony solution vector in HM will not be processed with the probability of 1-PAR.

Finally, a judgment will be used to evaluate the new harmony solution, whether the vector is better than the worst harmony solution vector in HM or not.If yes, the worst harmony solution vector will be replaced by the new harmony solution vector.Otherwise, the HM will be unchanged until it fulfills the termination condition.

According to the above three main strategies,the implementation steps of the standard HS can be classified into the following five main steps:

Step 1: definition

In this step, the minimum optimization problem function is first defined as

wherex=(x1,x2, …,xn)T∈ Rnis the decision vector in then-dimensional decision space.In addition, four parameters are defined in this step.These parameters and the corresponding meanings are listed in Table 1.

Step 2: initialization

In this step, the HM will be initialized using Eq.(2):whereXiis theithharmony in HM, andxi,jrepresents thej-dimensional vector (j=1, 2, …,n) of theithharmony vector (i=1, 2, …, HMS).

HM is a matrix for storing HMS solutions.In HM, each vector can be considered as a solution, and each harmony vector is composed ofnvariables.HMS harmony variables should be generated to form the initial HM within the scope of the feasible solution space through the principle of random generation or under certain rules.

Step 3: improvisation

In this step, new solutions will be created to ensure that the current solution will approach the potential optimal solution step by step by operating the following sub-steps:

Step 3.1: memory consideration and random selection.Generate a random numberr1∈[0,1].Ifr1≤HMCR, then a harmony solution vector will be randomly selected from HM.Otherwise, ifr1>HMCR, a new harmony solution vector will be randomly generated and move to step 4.

Step 3.2: pitch disturbance adjusting.Generate a random numberr2∈[0,1].Ifr2≤PAR, a harmony solution vector will be randomly selected from HM to implement the fine-tuning of the perturbation based on the value of bandwidth (bw) for generating a new harmony solution vector.Otherwise, no modification is made.

The pitch disturbance adjusting formula is shown in Eq.(3):

Step 4: update

In this step, the objective function of each harmony calculation is used as the criterion to decide the further operation of the newly obtained harmony.If the new one is better than the worst one (f(Xnew)

Step 5: adjustment

In this step, an evaluation will be executed to check whether it fulfills the termination condition.If the termination condition of the maximum iteration is fulfilled, the algorithm will be automatically terminated and the result will be generated.Otherwise, we move to step 3 for implementing one more iteration.

2.2 Standard CS algorithm

Yang and Deb (2009) proposed CS to simulate cuckoos’ breeding behavior of laying eggs in other birds’ nests.

The core operation of CS is to randomly simulate the female cuckoo laying her eggs in the host’s nest.The female cuckoo employs an aggressive strategy of brood parasitism and then lets the host bird hatch her chicks.If the host bird recognizes an alien egg in the nest, it either rejects the egg or abandons the nest and builds a new nest somewhere else to raise another brood.Based on that, the implementations of the standard CS can be classified into the following five main steps:

Step 1: set parameters (including population size,search domain, dimension, and a maximum number of iterations), initialize the position of the bird’s nest randomly, and define the objective function.

Step 2: calculate and compare the fitness value of each bird’s nest position to obtain the current optimal fitness value.

Step 3: update the population position by calculating the fitness value using the Levy flight.The function of the fitness value is used to compare with the position of the bird’s nest of the previous generation, update the position with a better fitness value,and keep it to the next generation.

The Levy flight is defined as follows:

3 Proposed HS-CS algorithm

Due to the drawbacks of falling into a local extremum in the improvisation stage, the standard HS is difficult to jump out of the current state, which will further affect the overall convergence performance.By combining the CS operator with the tuning pitch adjustment operator, an improved HS-CS algorithm is proposed to overcome this shortage.The modification strategies can be summarized based on the following aspects:

1.Because the Levy distribution is a heavytailed distribution, and its tail is wider than that of a Gaussian distribution, the algorithm which uses the Levy flight mechanism will have a stronger global search ability and robustness disturbance while a large step size is generated occasionally, or a small step size is generated by rotating 90° suddenly in the search process.Therefore, the Levy flight is selected to update the population for enhancing the global search ability of the HS-CS algorithm.

2.Three main strategies are employed to strengthen the ergodicity and the search ability of HS;these are to select the harmony solution vector randomly by the constructed inertia weight operator in HM with the probability of HMCR, to select the new harmony solution vector randomly in the solution space with the probability of 1-HMCR, and to update the harmony solution vector using the Levy flight strategy in the CS operator.

3.The CS operator is used to find potential individuals from HM as candidate individuals, expand the population density, and improve the efficiency of HS for avoiding easy-to-premature convergence and falling into a stagnant state in the final stage of the standard HS.

3.1 Implementation steps and the corresponding flowchart of the HS-CS algorithm

Based on the above modifications, the proposed hybrid HS-CS algorithm can be separated into the following seven steps:

Step 1: set the related parameters of the proposed HS-CS algorithm, including constraints, the maximum number of iterationsTmax, current iteration numbert,and search space [LB, UB].

Step 2: initialize HM.Randomly generate the harmony solution vectors of the HMS group within the scope of the search space and create the harmony solution vectors by Eq.(6):

where LB and UB are the lower and upper limits of HMi,jrespectively, and rand( )∈[0, 1] is a random number.

Step 3: improvisation

Step 3.1: generate a random numberr1∈[0, 1].Ifr1>HMCR, move to step 3.2; otherwise, the latest solution can be obtained by Eq.(7) through selecting the values in HM:

Step 4: transmit the newly generated harmony obtained in step 3.1 to the CS operator for updating the harmony solution vector.The new harmony solution vector can be calculated by Eq.(10):

Step 7: determine whether the termination condition is fulfilled or not.If yes, output the optimal solution; otherwise, dynamically adjust the probabilities of HMCR and PAR using Eqs.(12) and (13) respectively, and move to step 3.

where HMCRmaxand HMCRminrepresent the maximum and minimum values of HMCR respectively,and PARmaxand PARminrepresent the maximum and minimum values of PAR respectively.

3.2 Time complexity of the HS-CS algorithm

In heuristic algorithms, the calculation of the objective function takes most of the time of the whole algorithm.Assuming that the dimension of the objective functionfis Dim and that the population number is HMS, the time complexity of the HS-CS algorithm can be analyzed below:

In the initial population stage, the time complexity isO(HMS×Dim), and the time complexity of evaluating the objective function of the individual in the population isO(f(Dim)).

In the“pitch adjusting and selecting the best”stage of the improvisation process, the time complexity isO(HMS×Dim), and the “memory consideration”stage is known from Eq.(11), the corresponding time complexity of which isO(HMS×(Dim+O(Levy))),whereO(Levy) represents a random number subject to the Levy distribution and its magnitude of computational complexity is a constant order.

In the population update stage, the time complexity isO(HMS×(Dim+O(Levy))).

According to the corresponding time complexity of each stage, the worst-case time complexity of the HS-CS algorithm can be calculated below, while the number of the current iterations is one:

O(HMS×Dim)+O(HMS×(Dim+f(Dim)))+O(HMS×(Dim+f(Dim)))+O(HMS×(Dim+f(Dim)))≈O(HMS×(Dim+f(Dim))).

Therefore, when the HS-CS algorithm reaches the maximum number of iterationsTmax, the time complexity isO(Tmax×HMS×(Dim+f(Dim))).

3.3 Convergence analysis of HS-CS

Convergence is one of the most important properties for ensuring the performance of a meta-heuristic algorithm.To provide the feasibility and robustness of the proposed HS-CS algorithm, three theorems are presented and proved to reveal HS-CS as a global convergence meta-heuristic algorithm based on the following methodologies.At first, a differential equation is used to protect that the limit exists in the iterative process of HS-CS.Then, stochastic functional analysis is used to prove the convergence in the iterative process of the solution of the HS-CS algorithm.At the same time, global convergence is finally proved using random functional analysis and the characteristics of the HS-CS algorithm.Moreover, to confirm it, the update mechanism of HS-CS (Eq.(10)) is considered as a linear dynamic system, and a secondorder linear differential equation is used to analyze the change of the global optimal position.

Assuming that Eq.(10) is a one-dimensional second-order linear differential equation, Eq.(14) can be obtained by iterative recursive calculation:

To sum up, the existence of the limits of the constant-coefficient differential equations which are transformed from HS-CS is verified.According to the conditions of known limit existence, the convergence of HS-CS in each process of iterative search can be proved step by step below.

The iterative process of solving HS-CS can be regarded as a mapping of the solution space.Due to this mapping relationship, the global and local search convergence properties of HS-CS may be analyzed using the stochastic functional analysis theory based on Solis and Wets (1981).

Theorem 1 Assuming that setAis a non-empty set,there existsX= (X1,X2, …,Xn) ∈Afor anyXj∈[LBj,UBj]?A.Notiondis a mapping from Cartesian products to non-negative real functions,d:A×A→R.Thendcan be expressed as

where ?Xi,Xj∈A,zis an integer large enough.

Hence, (A,d) is a complete and separable metric space.

Proof First, it is necessary to prove that (A,d) is a metric space.According to the definition of metric space,Ais a non-empty set anddis a real function onA×A.The triangle inequality conditions are also fulfilled.

Whend(Xi,Xj) meets the three mentioned conditions, it is concluded that (A,d) is a metric space.

With the iterative progress of HS-CS, the global search efficiency of the population gradually changes,i.e., from strong search to a new strategy (the global search ability is weakened while the local optimization ability is enhanced).The value of the variableα?Levy(λ)will also tend to decrease when the global search ability is weakened in the later iterations and the value ofα?Levy(λ) is close to 0.It indicates thatTt X→

Theorem 2 Mapφformed by each iteration update is a random compression operator in the fine-tuning and optimization (local search) stage of HS-CS.

Proof HS-CS may produce better individual conditions than the previous generation in each iteration because of the appropriate fine-tuning selection operator and greedy selection strategy that are adopted.There is a non-negative real random variableH(δ)which fulfills

Furthermore,

4 Function optimization problem using HS-CS

To verify the feasibility and robustness of the proposed HS-CS algorithm, two experiments are performed from two different aspects.One is the function optimization using 12 benchmark functions, and the other is the weighted fuzzy production rule extraction by the HS-CS algorithm and BPNN over IRIS.

These experiments are run under the environment of Intel?CoreTMI5-10200H CPU @2.40 GHz,16 GB memory, and the Windows 10 operating system.The programming language is Python 3.7.

4.1 Function optimization

In this experiment, 12 classical benchmark functions, including Sphere, LevyN13, Alpine, Rastrigin,Griewank, Ackley, Step, Schwefel 2.22, Bohachevsky,Quartic, Rosenbrock, and Schwefel (http://www.sfu.ca/～ssurjano/) are selected to test the performance of the proposed HS-CS algorithm.These benchmark functions can be divided into two classes: unimodal functions and multimodal functions.The details of these selected benchmark functions are listed in Table 2.

4.2 Parameter design and experimental results

In this experiment, the HS-CS algorithm is compared with three other HS algorithms and three CS algorithms.The three other HS algorithms are global optimal adaptive harmony search algorithm (AGOHS)(Li et al., 2020), improved differential harmony search algorithm (IDHS) (Wang L et al., 2019), and harmony search with differential mutation based on pitch adjustment (HSDM) (Qin AK and Forbes, 2011).The three CS algorithms are novel enhanced cuckoo search algorithm (ECS) (Kamoona and Patra, 2019), modified cuckoo search (MCS) (Ong and Zainuddin, 2019), and cuckoo search (CS) (Yang and Deb, 2009).In addition,to ensure the fairness of all algorithms in the experiment, the population size and the maximum number of iterations are set to 100 and 5000, respectively.The related parameter setting of these algorithms is shown in Table 3.

Table 2 Twelve benchmark functions

To reduce the randomness and maintain the fairness of the algorithms, the optimization operation of each function is independently run 30 times (Num=30).

Figs.1 and 2 show part of the experimental results.Furthermore, the entire experimental results and corresponding analysis are given in the supplementary materials.

As a unimodal function, the Step function is usually used to test the convergence accuracy and speed of the algorithm.The HS-CS algorithm is integrated with Levy flight to broaden the search scope of the population, enhance the position update strategy, and improve the global exploration ability.As shown in Fig.1, no matter whether the Step function is with dimension 30 or 50, the proposed algorithm always exhibits the best convergence curve.When the dimension is 30, although the convergence accuracies of AGOHS and ECS are similar to that of the proposed algorithm, they require more iterations.When the dimensionality of the test function increases, AGOHS requires more iterations to achieve the same convergence accuracy as the HS-CS algorithm.In the set number of iterations, ECS fails to achieve appreciable results.Therefore, the HS-CS algorithm also has certain advantages over the selected algorithms in terms of stability.

As a multimodal function, the Griewank function is usually used to test the ability of the algorithm to jump out of local optima.As shown in Fig.2, when the dimension of the test function is low, since the search mechanism of Levy flight of CS is adopted by ECS and HS-CS, the convergence efficiencies of these algorithms are similar.AGOHS requires more iterations to obtain similar convergence accuracy to the HS-CS algorithm.As the dimension of the test function increases, the convergence efficiencies of AGOHS and ECS decrease significantly.In the search process, the HS-CS algorithm indicates the direction of “pitch adjusting and selecting the best.” The Levy flight in the CS operator is adopted to find candidate individuals when updating HM, which increases the number of alternative solutions and strengthens the disturbance to avoid falling into stagnation prematurely in the search process.Thus, the HS-CS algorithm still maintains a good convergence efficiency.

Further analysis of the experiment using the Wilcoxon rank-sum test, the mean value (MEAN), and the standard deviation (SD)is given in detail in thesupplementary materials.Combining the analysis above and that in the supplementary materials, the HS-CS algorithm exhibits good performance in dealing with high-dimensional optimization problems, proving that it has good convergence efficiency and self-adaptability.

Table 3 Parameter settings of the examined algorithms*

Fig.2 Optimizing the convergence curve of the Griewank function with dimension 30 (a) and 50 (b)

5 Weighted fuzzy production rule extraction model based on HS-CS

To further verify the practicability of the HS-CS algorithm, another typical application-weighted fuzzy production rule extraction from the IRIS dataset using HS-CS and BPNN is demonstrated here.The entire extraction process can be separated into the following main modules: (1) data fuzzification, (2) BPNN generation, and (3) BPNN framework optimization using the HS-CS algorithm, the importance index matrix obtained from the trained BPNN, and the weighted fuzzy production rule extraction.

5.1 Experimental data

In the IRIS plant classification dataset, there are three classification problems: setosa, versicolor, and virginica.Each category contains 50 samples, thus a total of 150 samples.Moreover, each sample contains four attributes, i.e., sepal length (SL), sepal width(SW), petal length (PL), and petal width (PW).The measurement unit of each attribute is cm.

5.2 Implementation analysis of weighted fuzzy production rule extraction

According to Li et al.(2020), the entire extraction process includes the following key steps:

First, the IRIS dataset is fuzzified.For each example attribute of the IRIS dataset, three fuzzy sets(semantic attribute values) are used to mark each example attribute, which are large (LGR), medium(MED), and small (SM).

Second, the corresponding BPNN is established by the IRIS dataset.In this module, 12 semantic attribute values of the IRIS dataset are used to generate rule preconditions, and the three classification results are used to build 12 input nodes of the input layer, three output nodes of the output layer, one hidden layer,and four hidden nodes generated by experience.The sigmoid function is used to obtain BPNN as an activation function.Meanwhile, the fuzzy dataset is randomly divided into two parts, in which 80% of the examples are used as the training set and the remainder as the test set.

Third, HS-CS is employed to optimize the related parameters and the objective function of the obtained BPNN.The parameters of HS-CS are set as follows:HMS=100, HMCRmax=0.9, HMCRmin=0.8,PARmin=0.1, PARmax=0.9,α=0.01, the dimension of each harmonyD=12 (reflecting the number of input nodes),the learning rate is 0.02, the penalty factor is 0.001,and the momentum is 0.9.Furthermore, the threshold value is set as 0.5 after BPNN training is completed by the HS-CS algorithm.The connection weights connected to the threshold are “pruned” for removing redundant paths and obtaining a simple BPNN structure.

Fourth, the corresponding importance index matrixWn×m=OWn×k·IWk×mcan be gained while the BPNN optimization and training are completed, among which OWn×krepresents the connection weight matrix between the hidden layer and output layer in BPNN,krepresents the number of nodes in the hidden layer,nrepresents the number of output nodes in the output layer, IWk×mrepresents the connection weight matrix from the input layer to the hidden layer, andmrepresents the number of input nodes.

Through the above steps, the importance index matrix of the obtained BPNN by HS-CS is given in Eq.(20).

InW3×12, three rows and 12 columns represent three categories and 12 semantic attribute values corresponding to IRIS, respectively.Moreover, an attribute value does not play a role in IRIS classification if the corresponding values in a column are all 0.

In the importance index matrix, the classification results corresponding to each row may follow the same rule or multiple rules.If the attribute value element of the unified attribute is not 0, multiple rules will be generated; that is, there is no OR in the preceding one.If the attribute value element is negative, the corresponding NOT antecedent is generated.

Based on the above requirements, the weighted fuzzy production rules can be automatically extracted from the importance index matrix.In this paper,we list only the first row of the matrix, producing 18 classification rules as Iris-setosa (Table 4).The entire generated local weighted fuzzy production rules could be found in the supplementary materials.

To highlight the advantages of the proposed HSCS algorithm, two other algorithms, i.e., standard HS(Geem et al., 2001) and AGOHS (Li et al., 2020),are used to extract the weighted fuzzy production rules from the IRIS dataset under the same experimental environment.

The extracted rules are classified and verified,and then applied to the training set and test set for comparison.The related data are listed in Table 5.

Table 5 indicates that the proposed HS-CS algorithm has higher accuracy than AGOHS and HS while optimizing BPNN and extracting the weighted fuzzy generation rules from BPNN.Meanwhile, the accuracy obtained using HS-CS on the test set reaches 97.37%.It further verifies that the HS-CS algorithm has the advantage of enhancing the learning and generalization ability of BPNN.On the other hand, the execution time of implementing the IRIS classification of HS,HS-CS, and AGOHS is 35.51, 40.15, and 370.23 s, respectively.Although the execution time of HS is less than that of HS-CS, the accuracy of HS in data classification is lower than that of HS-CS.Within a reasonable execution time range, HS-CS can achieve a better classification effect.

5.3 Comparisons of the loss degree and precision curves of the optimized BPNN

Figs.3 and 4 show the convergence curve and convergence precision curve of the loss function of BPNN optimized by different HS algorithms in the training process of the IRIS dataset, respectively.

From the convergence diagram and convergence precision diagram of the loss degree function above,it is further indicated that the convergence speed and loss value of the BPNN optimized by the proposed HS-CS algorithm have obvious advantages compared with AGOHS and the standard HS algorithm.It is also illustrated that the HS-CS algorithm plays a certain role in the training process of the optimized BPNN.

To sum up, in the process of weighted fuzzy rule extraction using the HS-CS algorithm, the BPNN optimized by HS-CS extracts the weighted fuzzy production rules from the IRIS dataset in a relatively shortexecution time and achieves a better classification effect.The main functions of HS-CS are to improve the convergence speed of BPNN and reduce the loss during training.

Table 5 Comparison of IRIS classification accuracy results

Fig.4 Convergence precision curve of the loss function of BPNN with different HS algorithms (BPNN: back propagation neural network; HS: harmony search)

6 Conclusions

In this paper, a modified HS-CS algorithm is presented to solve the problems of the standard HS which is easily trapped into local optima and is weak in global search ability.In particular, the HS-CS algorithm uses Levy flight to explore the solution space,which further enriches the population density of the HS algorithm, enhances the global search ability, and avoids the algorithm getting trapped in local optima.On the other hand, “pitch adjusting and selecting the best” in the improvisation stage and the inertial weight operator are constructed and the degree of the internal individuals in HM is selected to improve the efficiency and convergence accuracy of the basic HS algorithm.

Two experiments are implemented to verify the effectiveness of the proposed HS-CS algorithm.First,the existence of the limit of the proposed algorithm is proved by differential equations, and the global optimal algorithm is verified by random functional analysis and random search theory.Second, HS-CS and six other algorithms are used to settle the function optimization issue of the 12 selected classical benchmark functions in different high dimensions.The results show that the HS-CS algorithm can still maintain the speed and precision of rapid convergence in the process of solving the optimization of high-dimensional functions and has strong robustness.Finally, HS-CS is applied to the IRIS classification for extracting the weighted fuzzy production rules.Through data analysis, it is easy to find that weighted fuzzy production rules obtained by combining BPNN and HS-CS have a high precision because the HS-CS algorithm can enhance the learning and generalization abilities of BPNN effectively.

This research is intended mainly to optimize the solution of a single objective problem for HS-CS without comprehensive consideration of the impact of multiple factors on the accuracy of the algorithm.Therefore, the influence of multiple factors will be considered in HS in future work, and the characteristics of the Pareto solution and HS will be used to solve the real-life multi-objective optimization problems,such as obstacle avoidance for multiple unmanned aerial vehicles in coordinated formation.

Contributors

Kaiqing ZHOU designed the research.Shaoqiang YE and Kaiqing ZHOU processed the data.Shaoqiang YE and Fangling WANG drafted the paper.Kaiqing ZHOU and Azlan Mohd ZAIN helped organize the paper.Azlan Mohd ZAIN and Yusliza YUSOFF revised and finalized the paper.

Compliance with ethics guidelines

Shaoqiang YE, Kaiqing ZHOU, Azlan Mohd ZAIN,Fangling WANG, and Yusliza YUSOFF declare that they have no conflict of interest.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.