Detection of the Pine Wilt Disease Tree Candidates for Drone Remote Sensing Using Artificial Intelligence Techniques

Mutiara Syifa, Sung-Jae Park, Chang-Wook Lee*

Division of Science Education, Kangwon National University, Gangwon-do 24341, Republic of Korea

Keywords:Pine wilt disease Drone remote sensing Artificial neural network Support vector machine Global positioning system

ABSTRACT Pine wilt disease(PWD)has recently caused substantial pine tree losses in Republic of Korea.PWD is considered a severe problem due to the importance of pine trees to Korean people, so this problem must be handled appropriately.Previously,we examined the history of PWD and found that it had already spread to some regions of Republic of Korea; these became our study area. Early detection of PWD is required.We used drone remote sensing techniques to detect trees with similar symptoms to trees infected with PWD.Drone remote sensing was employed because it yields high-quality images and can easily reach the locations of pine trees. To differentiate healthy pine trees from those with PWD, we produced a land cover(LC)map from drone images collected from the villages of Anbi and Wonchang by classifying them using two classifier methods, i.e., artificial neural network (ANN) and support vector machine (SVM).Furthermore, compared the accuracy of two types of Global Positioning System (GPS) data, collected using drone and hand-held devices, for identifying the locations of trees with PWD. We then divided the drone images into six LC classes for each study area and found that the SVM was more accurate than the ANN at classifying trees with PWD.In Anbi,the SVM had an overall accuracy of 94.13%,which is 6.7%higher than the overall accuracy of the ANN, which was 87.43%. We obtained similar results in Wonchang, for which the accuracy of the SVM and ANN was 86.59% and 79.33%, respectively. In terms of the GPS data, we used two type of hand-held GPS device. GPS device 1 is corrected by referring to the benchmarks sited on both locations, while the GPS device 2 is uncorrected device which used the default setting of the GPS only. The data collected from hand-held GPS device 1 was better than those collected using hand-held GPS device 2 in Wonchang.However,in Anbi,we obtained better results from GPS device 2 than from GPS device 1.In Anbi, the error in the data from GPS device 1 was 7.08 m,while that of the GPS device 2 data was 0.14 m.In conclusion,both classifiers can distinguish between healthy trees and those with PWD based on LC data. LC data can also be used for other types of classification.There were some differences between the hand-held and drone GPS datasets from both areas.

1. Introduction

Pine trees are essential to Republic of Korea,both culturally and spiritually[1].There are vast pine forests(Pinus desiflora and Pinus thunbergii) in Republic of Korea, occupying a total area of 1 507 118 hm2or 23.5% of the total forested area [2,3]. In 1980,pine wilt disease (PWD) was detected for the first time in Busan,Republic of Korea. This resulted in severe losses and is considered a serious threat to Republic of Korea’s pine forests [1]. The damaged area now covers 7820 hm2and more than 60 cities in Republic of Korea [4]. PWD is caused by a type of nematode,Bursaphelenchus xylophilus, which is transmitted by insect vectors known as pine sawyer beetles (Monochamus spp.). The vector infects trees from early June to late July, which is the period of maturation feeding time for adult pine sawyers [4,5].

PWD originated in North America [6-8] and first spread to Japan, where it has caused severe problems for 100 years [8,9],then to China [10] and Republic of Korea [1,4] over the last few decades. PWD dissemination is closely related to environmental factors, such as the temperature and water content of the soil[11]. The pine wilt nematode (PWN) attacks pine trees, most of which die within three months of infection [1,12,13]. Proper handling, such as early detection, of this problem is required because fumigation, burning, and felling of trees are currently the only ways to control the spread of PWD [1,14].

Drone remote sensing is believed to have the potential to help solve this problem.It is flexible and low-cost,and drone platforms can be equipped with high-resolution remote sensing systems[15]. Drone, or unmanned aerial vehicle (UAV), images have been widely applied to overcome problems in forestry and agriculture[15-17], urban vegetation [18], and even disasters such as earthquakes [19,20], landslides [21-23], and volcanic eruptions[24,25]. In the case of the early detection of PWD, pine trees with leaves that turn brown before autumn are categorized as PWDindicated trees. Drones are useful because it is difficult to identify infected pine trees from field studies [13] due to the locations of the trees, which are usually on mountains. The topographical characteristics of forests also make direct access very challenging.Furthermore, pine trees infected with PWD cannot always be identified by the naked eye because they are viewed from below.

After obtaining drone images,we used supervised classification to identify the locations of trees affected by PWD. Artificial neural networks (ANNs) and support vector machines (SVMs) are two widely used classifiers with proven classification accuracy[26,27].In this study,we used ANN and SVM classifiers to identify trees with signs of PWD.

Furthermore, in addition to using drone global positioning system(GPS)data,we also used two hand-held GPS devices to collect field data and analyzed the differences between the results from the drone and the hand-held GPS devices. Differences were seen in the results even though the data were collected from the same location. It is essential to be aware of the differences between the results from hand-held GPS devices and drone GPS devices to understand the error profiles. To summarize, the purpose of this study was to identify the most effective classifier for detecting PWD-indicated trees and elucidate the differences between results collected from hand-held GPS devices and drone GPS devices.

2. Materials and method

2.1. Materials

High definition drone images (DJI Phantom 4 V 1.0; DJI, China)and GPS data from two hand-held GPS devices (Garmin Oregon 750 T;Garmin,Switzerland)were used to collect data from the study area.We focused on two areas,namely Anbi and Wonchang Villages,which are located in Chuncheon City,Gangwon Province,Republic of Korea, at 37°50′16.08′′N, 127°34′59.88′′E and 37°48′21.24′′N,127°46′30.36′′E, respectively. The two locations were selected due to their previous track records of PWD, so we assumed that some PWNs remain even though experts have already eradicated the disease from these areas. The study areas in Wonchang and Anbi Villages are shown in Fig.1(a).

Fig.1. (a) Study area, i.e., Wonchang and Anbi Villages,located in Chuncheon City,Republic of Korea, shown in Landsat-8 images; (b) there are two PWD-indicated trees in Wonchang, this image shows the first candidate, and (c) the second PWDindicated tree in Wonchang.

Drone images were collected from Anbi on 16 September 2018 and from Wonchang on 23 September 2018. The GPS data were collected from both areas on 5 October 2018.A benchmark monument in Wonchang used to correct the results from GPS device 1 before identifying PWD-indicated trees (Figs. 1(b) and (c)). The benchmark monument also known as survey marker, is a sited object to spot the key survey on Earth’s surface which also indicate the elevation. The default settings were used for GPS device 2 which then the GPS 2 called as uncorrected GPS. There was only one PWD-indicated tree in Anbi and two PWD-indicated trees in Wonchang. We used the following software in this study: Agisoft Photoscan Professional to process the drone images, BaseCamp version 4.7.0 to process the GPS data, and ArcMap 10.4 and ENVI Classic 5.3 to classify and assess the accuracy of the classifiers.To classify the images, we divided Anbi into 13 land cover (LC)classes, and Wonchang into nine classes so that we could distinguish the indicated-PWD trees from other types of LC,such as normal pine trees,grass fields,other trees species,bare land,road,and buildings.However,some types of LC,such as bare land,were classified as PWD due to the similarities in color. Hence, it was challenging to analyze the drone images in the study area. The classification scheme and category definitions are listed in Table 1 for Wonchang and Anbi.

2.2. Method

When mapping PWD-indicated trees in the study area,we used two types of artificial intelligence (AI): ANN and SVM classifiers.Yuan et al. [16] used ANNs and SVMs to estimate an indicatorcalled the leaf area index (LAI) of soybean plants with high accuracy and precision.Thus,we assessed the ANNs and SVMs classification results to calculate the accuracy of both classifiers using the error matrix method, which evaluates the overall accuracy, kappa coefficient, producer accuracy, and user accuracy. In addition to evaluating the accuracy of both classifiers,we compared the drone GPS results to the hand-held GPS results from both study areas.To evaluate these differences, we compared the center points of each PWD-indicated tree identified using the drone to the results from the hand-held GPS device. In general, our method is divided into four phases: collection of drone images, classification of images,accuracy assessment, and comparison between the classification and GPS results, as shown in Fig. 2.

Table 1 Land cover (LC) classifications and definitions for Wonchang and Anbi areas.

2.2.1. Artificial neural network

An ANN is a mathematical framework that is intended to mimic human learning processes through a parallel process that reinforces the linkage between input and output data [28,29]. One of the most common implementations of ANNs, which we also used in this study,is a feedforward network(FFN).This is a nonparametric nonlinear model that consists of an input layer, a hidden layer,and an output layer[30].All ANNs have a specific number of nodes,which are linked to each other with specific weightings and biases in the subsequent layer. The FFN is defined as:

Fig.2. Flow chart of the methodology used in this study to determine the accuracy of the ANN and SVM classifiers, and compare the drone GPS data to data from two hand-held GPS devices.

where the subscripts i and j denote the previous and current layer,respectively; x is the nodal value; b and w are bias and weight values,respectively;n is the number of nodes in the previous layer;and f signifies a transfer function of the present layer. Linear and log-sigmoid functions were assigned to the output and hidden layers, respectively. This combination is known to be effective for enhancing the extrapolation ability of the ANN[29,31].To minimize the error between the output and input data, we used a backpropagation algorithm (BPA) to train the ANN.

“You poor little child,” said the old woman, “how did you manage to come all this distance into the wide world on such a rapid rolling stream?” And then the old woman walked in the water, seized the boat with her crutch, drew it to land, and lifted Gerda out

Hence, the input and output were repeatedly fed through the network, and the error was propagated from the output to the input layer [32]. The most critical parameter in the ANN model is the quantity of the neurons; using more neurons results in higher learning accuracy but weakens the generalizability of the ANN[28].In this study,we based our work on Kavzoglu’s network architecture and training patterns [33]. We used ENVI Classic 5.3 to train the ANN. A logistic activation method was employed and the training threshold contribution and momentum were set to 0.9, while the rate field, root mean squared (RMS) exit criterion,and number of training iterations were set to 0.2, 0.1, and 100,respectively.

2.2.2. Support vector machine

An SVM is a machine learning method based on statistical learning theory. SVMs, originally proposed by Cortes and Vapnik[34], are nonparametric algorithms based on structural risk minimization (SRM) rather than empirical risk minimization (ERM),which is employed by ANNs. ERM is designed to minimize the errors in the training data used by classifiers, while SRM is designed to maximize the margins between the data groups to be classified, which in turn maximizes the generalizability of the model [35]. When implementing an SVM, it is essential to use an appropriate kernel function,i.e.,one which reflects the similarities between data points [36]. We used the radial basis function (RBF)kernel due to its excellent performance in nonlinear classification algorithms [37,38]. In this study, the penalty parameter and gamma were set to 100 and 0.05,respectively,to generate the most accurate model possible.

3. Results

The drone images collected from both study areas were classified using ANN and SVM classifiers implemented in ENVI Classic 5.3. In addition to the drone image data, GPS data were also collected so that we could evaluate the differences between the GPS data collected from the drone and hand-held GPS devices. The images were grouped into six classes:PWD-indicated trees,normal pines,grass and trees,road,shadows,and bare land for Wonchang.In Anbi, we used the following classes: PWD-indicated trees, normal pines,grass and trees,buildings,roads,and shadows.The classification accuracy was assessed using a stratified sampling method on a pixel by pixel basis. Pixels in each class were randomly selected from drone images and then used as reference points, which we compared with the results of the SVM and ANN classifiers in both areas.

We used Microsoft Excel to calculate the matrices and evaluate the overall accuracy,kappa coefficient,producer accuracy,and user accuracy so that we could compare between the two reference error matrices. The kappa coefficient is a discrete multivariate technique that is often used to measure accuracy[39].Kappa coefficients are interpreted as a collective judgment,in which an agreement is reached after excluding chance agreement[40].Therefore,we calculated the kappa statistic of each error matrix obtained from the ANN and the SVM classification results to determine which classifier yielded the best results.

3.1. Land classification result in Anbi village

The drone images collected from Anbi were classified into six categories, which were merged from 13 base classes. Thirteen classes were specified due to differences in the colors of objects in the images,such as buildings,which can have many roof colors.Hence, dividing the images into 13 classes made it easier for the algorithm to classify the images. After obtaining the classification results,we merged several classes that represented different colors of the same objects. The six classes thus generated were: PWDindicated trees, normal pine, grass and trees, buildings, roads,and shadows,as shown in Fig.3.To classify the images,we trained five polygons for each group with similar colors. We used 65 samples in total to generate the LC map. In this case, the SVM(Fig. 3(a)) distinguished PWD-indicated trees better than the ANN(Fig.3(b)).Fig.3(c)shows that some parts of images,showing items such as buildings and roads that have similar colors to PWD-indicated trees.These were classified as having the same colors as PWD-indicated trees by both the ANN and SVM classifiers,but the SVM recognized the trees better.

3.2. Accuracy assessment from Anbi classification

In Tables 2 and 3,we summarize the error matrices of the PWD classification results from both the ANN and SVM algorithms.Both methods were based on analyzing 358 pixels:59 pixels from PWDindicated trees, 60 pixels from normal pines, 60 pixels from grass and trees, 60 pixels from buildings, 59 pixels from roads, and 60 pixels from shadows. All sample data were based on multinomial probability theory and stratified random sampling. The overall accuracy of the SVM was 94.13%, while that of the ANN was 87.43%. The higher percentage of the SVM classifier indicates that it was more accurate than the ANN. This conclusion is supported by the kappa coefficients of both results: 0.9296 or 92.96% for the SVM and 0.8492 or 84.92% for ANN.

3.3. Land classification result in Wonchang village

We initially classified drone images collected from Wonchang into nine classes due to some objects having different colors, then simplified the results into six classes by merging classes of the same objects.As shown in Figs.4(a)and(b),the land classification maps of Wonchang generated using SVM and ANN have six classes.The classes is divided based on its similarity as shown in the drone image in Fig. 4(c), there are many different types and colors of vegetation, including bush, grass, and tree species that can be grouped into one class.Nevertheless,to enable classifiers to distinguish the classes quickly, we classified nine categories using the SVM and ANN.

Fig.3. Land classification map of Anbi,generated using:(a)an SVM and(b)an ANN from(c)drone image data.The images were classified into six categories:PWD-indicated trees (orange), normal pine trees (light green), grass and trees (green), buildings (blue), roads (light grey), and shadows (grey).

Table 2 Error matrix for Anbi classified using an ANN.

Table 3 Error matrix for Anbi classified using an SVM.

As in the case of Anbi,we merged the SVM and ANN categories based on their similarities.For example,grass,bush,and trees were merged into one class. Overall, the classification results generated by both classifiers can distinguish PWD-indicated trees, but the SVM classifier yielded better results than the ANN classifier, as shown in Figs. 4(a) and (b), respectively. In Wonchang, the SVM misclassified some areas but not as many as the ANN that classified some areas as PWD-indicated trees, such as bare land and other trees species, which were similar in color to PWD-indicated trees due to the arrival of peak fall season.

3.4. Accuracy assessment from Wonchang classification

We assessed the accuracy of the classification results for the Wonchang area using 358 sample pixels: 59 pixels from the PWD-indicated trees, 60 pixels from normal pines, 60 pixels from grass and trees, 59 pixels from roads, 60 pixels from shadows,and 60 pixels from bare land. The results of the accuracy assessments of the ANN and SVM classifiers for Wonchang are shown in Tables 4 and 5, respectively. The SVM was more accurate(86.59%)than the ANN(79.33%),which means that the SVM distinguished among the classes, especially the PWD-indicated trees,better than the ANN. This conclusion is supported by the kappa coefficient, which was 0.8391 or 83.91% for the SVM and 0.7520 or 75.20% for the ANN. According to the Landis and Koch (1997)characteristic, this indicates stronger agreement.

3.5. Differences between GPS datasets

There were some differences between the coordinates collected from the hand-held GPS device and the drone GPS device,as shown in Tables 6 and 7. There were several causes of these differences,such as GPS-signal propagation errors, which are modified as the signal travels through the ionosphere, and the effect of relativity.The two types of GPS devices provided different coordinates for the PWD trees in Anbi and Wonchang: the coordinate data from hand-held GPS device 1 was corrected in the field, while that of hand-held GPS device 2 was not corrected as we used the default settings of the GPS device. There were larger gaps between the drone GPS results and the results collected from hand-held GPS device 2 (differences of 7.37 and 7.25 m) than from device 1 (differences of 9.75 and 1.22 m) in Wonchang. Hence, GPS device 1,which corrected the coordinate, yielded more similar results to the drone GPS device. However, in Anbi, the results from GPS device 2 were closer (differences of 0.14 and 2.49 m) to those obtained using the drone GPS device. Hence, in this case, the default data from the device yielded better results than the corrected data provided by GPS device 1 (differences of 7.08 and 3.37 m).

4. Discussion

After processing the drone image by using two classifiers, ANN and SVM, we generate classification result and accuracy assessment of Wonchang and Anbi as well as the differences between GPS dataset that already mentioned in result part. However, there are some findings that need to be discussed that will be addressed in the following section.

4.1. Land classification map from Anbi

From the classification result of SVM and ANN classifier in Anbi,it looks slightly different in the map,but so different in their accuracy percentages. However, if we take a look to both classification result(Figs.3(a)and(b))in details,there are some differences of its classification result. For example the PWD-indicated tree class in SVM result is not as many as the ANN, either for the PWDindicated tree itself or the misclassified region.Moreover,the other classes such as road, shadow, and building were detected in some areas that it should not be, especially for the ANN classification result.

Since the confusion matrix(error matrix)method,which correspondence between the classification result and reference, so it makes the SVM and ANN has differences for its accuracy even the LC map looks similar. In addition this method used the pixel number to determine the accuracy, either for its overall accuracy or kappa coefficient.So,even though the PWD-indicated tree shade looks similar each other, the accuracy result shown was for all classes which experienced misclassification also.

Fig.4. Land classification map of Wonchang generated using:(a)an SVM classifier and(b)an ANN classifier from(c)drone images. Images classified into six classes:PWDindicated trees (orange), normal pines (light green), grass and trees (green), Bare land (pink), roads (light grey), and shadows (grey).

Table 4 Error matrix for Wonchang, classified using the ANN.

4.2. Accuracy assessment from Anbi

The accuracy assessment represents the accuracy from the classification map generated. In assessing the accuracy, the confusion matrix was employed to calculate the pixel number and its comparison. As can be seen form Tables 2 and 3, the SVM generates a better result than the ANN for its overall accuracy and kappa coefficients.From each table,we can see each class’user accuracy that referred to as reliability. Some classes are shown a very high percentage such as grass and trees and shadows classes in Table 3 which means these classes will actually be present on the ground as often as it is when the user of this map go to the field.

4.3. Land classification map from Wonchang

As well as the Anbi village result, the classification result from Wonchang is also slightly different yet the accuracy result is so different. Since the same method was applied to calculate the accuracy and employed the pixel number to determine the overall accuracy and kappa coefficient, thus the differences betweenclassification and accuracy result that seems to be opposite can be explained. If we look through the maps carefully in Figs. 4(a) and (b), particularly the PWD-indicated tree class in the ANN have larger misclassified area than the SVM. In addition, the bare land and some grass that has brownish color also detected as PWD-indicated tree which make the ANN has a lower overall accuracy than the SVM in Wonchang result.

Table 5 Error matrix for Wonchang, classified using the SVM.

Table 6 Comparison of the three GPS datasets collected from Anbi.

Table 7 Comparison of the three GPS datasets collected from Wonchang.

4.4. Accuracy assessment from Wonchang

Similar to the Anbi, accuracy assessment in Wonchang were also used the same method. Error matrix method allows us to generate the overall accuracy and kappa coefficient. As shown in Tables 4 and 5, the overall accuracy for SVM is higher than the ANN, so with the kappa coefficient. This accuracy percentage is influenced by the classification result and the accuracy from each class. For bare land class that generate a low percentage for producer accuracy among the other classes which means only 35% of the bare land on the ground correctly shown on the classified map or we can said the probability of the bare land classified in the map is only 35%. In contrast, the user accuracy of this class generate a better percentage, which is 91.30% which means the bare land class from the classification result map is reliable even though its producer accuracy was low. However, since each class’accuracy affect the overall and kappa accuracy, this example can explained why the SVM more powerful than the ANN.

4.5. Differences between GPS datasets

The differences between hand-held GPS device 1 and GPS device 2 is only from its coordinate data.The hand-held GPS device 1 was corrected by logging the GPS to the point on the benchmark nearby Wonchang and Anbi. Meanwhile the hand-held GPS device 2 was not corrected to the benchmark.As mentioned in result part,the differences can be occurred due to some technically causes such as GPS-signal propagation errors and relativity effect, nevertheless workflow in processing the data and field technique also might be have a role in this study.

5. Conclusions

We used ANN and SVM algorithms to classify images from Anbi and Wonchang and successfully distinguished PWD-indicated trees from other types of LC observed in the drone images. We obtained better results from the SVM classifier,which had a higher overall accuracy (94.13% for Anbi and 86.59% for Wonchang) than the ANN classifier (87.43% for Anbi and 79.33% for Wonchang).Compared to the ANN classifier, the kappa coefficient indicated stronger agreement in the case of the SVM (0.9296 for Anbi and 0.8391 for Wonchang).This indicates that SVM classifiers can classify types of LC or specific trees that have similar symptoms to PWD better than ANNs.

There were some differences in the GPS datasets obtained from the three devices used. The corrected hand-held GPS device was more accurate in Wonchang, but in Anbi, results from the handheld GPS device that provided raw data were closer to those of the drone. Overall, both algorithms successfully distinguished between PWD-indicated trees and the other types of LC shown in drone images. These algorithms can be used by experts, governments, and other policyholders to facilitate the early detection of PWD,and will help them to find the best solution to this problem.Follow-up observations by forestry researchers or experts are required to confirm whether the PWD-indicated trees are indeed infected by PWD.

Acknowledgements

This research was supported by a grant from the National Research Foundation of Korea,provided by the Korean government(2017R1A2B4003258).

Compliance with ethics guidelines

Mutiara Syifa,Sung-Jae Park, and Chang-Wook Lee declare that they have no conflict of interest or financial conflicts to disclose.