Threshold Filtering Semi-Supervised Learning Method for SAR Target Recognition

Linshan Shen,Ye Tian,＊,Liguo Zhang,2,Guisheng Yin,Tong Shuai,Shuo Liang and Zhuofei Wu

1Harbin Engineering University,College of Computer Science and Technology,Harbin,150001,China

2Xidian University,Xi’an,710000,China

3The 54th Research Institute of CETC,Shijiazhuang,050000,China

4University of Bologna,Bologna,40100,Italy

Abstract:The semi-supervised deep learning technology driven by a small part of labeled data and a large amount of unlabeled data has achieved excellent performance in the field of image processing.However,the existing semisupervised learning techniques are all carried out under the assumption that the labeled data and the unlabeled data are in the same distribution,and its performance is mainly due to the two being in the same distribution state.When there is out-of-class data in unlabeled data,its performance will be affected.In practical applications,it is difficult to ensure that unlabeled data does not contain out-of-category data,especially in the field of Synthetic Aperture Radar(SAR)image recognition.In order to solve the problem that the unlabeled data contains out-of-class data which affects the performance of the model,this paper proposes a semi-supervised learning method of threshold filtering.In the training process,through the two selections of data by the model,unlabeled data outside the category is filtered out to optimize the performance of the model.Experiments were conducted on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset,and compared with existing several state-of-the-art semi-supervised classification approaches,the superiority of our method was confirmed,especially when the unlabeled data contained a large amount of out-of-category data.

Keywords:Semi-supervised learning;SAR target recognition;threshold filtering;out-of-class data

1 Introduction

In recent years,artificial intelligence technology has been widely used in various fields,such as image recognition[1],adversarial attacks[2],privacy protection[3,4],etc.Among them,data-driven deep supervised learning has made remarkable progress in various vision tasks in the field of computer vision and achieved remarkable results[5-8].However,training a deep learning model requires a lot of labeled data[9,10].When the labeled data in the training dataset is insufficient,performance becomes limited.In the field of Synthetic Aperture Radar(SAR)image processing,SAR images are different from ordinary optical images due to the imaging mechanism and speckle noise[11-13].These factors make the labeling and recognition of SAR images quite difficult[14].

In order to get rid of the dependence of deep learning technology on a large amount of labeled data,semi-supervised learning technology is proposed[15,16].It uses a large amount of unlabeled data that is easy to obtain,reduces the need for labeled examples in deep learning,and reduces the labor cost of labeled data.Semi-supervised learning optimizes the model by automatically labeling unlabeled data during the training process,and has received extensive attention from the machine learning and computer communities.

Existing semi-supervised learning SAR target recognition technologies mainly include generative adversarial networks,teacher-student networks,and consistency regularization technology.Gao et al.[17]proposed a semi-supervised learning method based on a deep convolutional generative adversarial network (GAN),which uses two discriminators to stabilize the training of the generator,thereby generating high-quality SAR images to optimize the model to obtain good recognition performance.In the training process,the teacher-student network uses the teacher model to mark unlabeled data to optimize the student model and is widely used in semi-supervised learning.Tian et al.[18]use the teacher-student network to divide the unlabeled data to improve the unlabeled data quality of pseudomarking,thereby optimizing the recognition model.Wang et al.[14]proposed a semi-supervised learning framework through self-consistent enhancement(SCA),combined with data enhancement,hybrid labeling,consistent regularization,and other technologies to obtain excellent recognition accuracy on SAR images.

The above semi-supervised learning SAR target recognition method improves the recognition accuracy of the model to a certain extent.The premise of these methods is that the unlabeled and labeled data used during training are in the same distribution.However,during actual model training and usage,unlabeled data is often mixed with a large amount of out-of-class data[19].This kind of out-of-category data is mixed into the unlabeled data,which will negatively affect the training of the model to a certain extent.Especially in the field of SAR image recognition,it is often difficult to clean and partition unlabeled data.

In order to solve the problem that the out-of-class data mixed with unlabeled data in the semisupervised learning SAR target recognition method affects the recognition accuracy of the model,Guan et al.[20]proposed a semi-supervised learning method based on Edit Nearest Neighbors(ENN).When most of the neighbors of a pseudo-labeled sample have different classes,D.Guan defines the sample as a noise instance and excludes this noise instance during the training process of the semisupervised learning method.Gao et al.[21]proposed an ensemble GAN consisting of a conditional GAN,an unconditional GAN,and a classifier to simultaneously achieve semi-supervised generation and recognition.However,the above two methods both need to train multiple classifiers at the same time,which consumes more resources.Unlike the above methods,our goal is to clean the unlabeled data and optimize the recognition efficiency of the model through semi-supervised learning while only using a single model.To this end,we propose a threshold filtering semi-supervised learning method,the framework of which is shown in Fig.1.

Figure 1:The framework of the proposed method

In the training process of the model,the unlabeled data is labeled by the current training model,and there is a certain difference in the probability of labeling results.Our method is implemented using it as an entry point.Before the start of each epoch of model training,we use the current model to pseudo-label all unlabeled data,sort the samples according to their pseudo-label prediction probability,and select some samples as unlabeled samples based on threshold filtering to optimize the recognition accuracy of the model.The proposed method is simple,effective,and flexible because it can be easily integrated into existing semi-supervised learning methods.Through a large number of experiments,we proved the effectiveness of our method on the mainstream SAR target recognition dataset Moving and Stationary Target Acquisition and Recognition(MSTAR)in a semi-supervised setting.

The rest of the article is organized as follows:Section 2 details the proposed method in this article.Based on the open dataset,Section 3 presents the experimental results and discussion.Finally,Section 4 gives a conclusion.

2 The Proposed Method

In this section,we introduce the details of the proposed method.The framework of the proposed method is shown in Fig.1,for unlabeled data,it is pseudo-labeled by training the model.The pseudolabels are sorted according to the maximum probability value from large to small,and the samples ranked in the top 50%are selected.Then judge the last 50%of the samples,screen out the samples with the largest prediction probability greater than 80%(the basket selection part in the figure),and perform semi-supervised learning training together with the first 50%of the data to optimize the model.

2.1 Pre-selection

In the existing semi-supervised learning method,given a labeled datasetand an unlabeled datasetwheren?m.The model optimization is performed under the condition that the two datasets have the same distribution.However,in actual SAR target recognition,unlabeled data is usually mixed with out-of-category data,as shown in Fig.2.Unlabeled data contains samples that are out-of-class in the labeled data(indicated with red bounding boxes).It can be seen that data outside these categories is not easy to find,and manual selection is time-consuming and laborious.For this reason,in the semi-supervised training process of the model,we use preprocessing to select some unlabeled data with high probability as samples to optimize the model.The specific preprocessing methods are as follows:

Figure 2:One example of class distribution mismatch

Based on SCA[14],we screen unlabeled samples during the training process in order to optimize the recognition performance of the model.In order to save calculation time and improve the efficiency of model training,we refer to the characteristics of the teacher-student semi-supervised model,and use the teacher model in the training process to filter and divide the unlabeled data.The characteristic of the deep learning training model is that the recognition performance of the model is gradually improved.Therefore,a large number of unlabeled samples will be incorrectly divided among the samples selected in the initial training stage,which cannot play the role of optimizing the model.To this end,at the beginning of each epoch of the model training process,we use the teacher model of the current training model to predict and label all unlabeled samples,and sort them according to the maximum value of the labeling probability.Sort out the unlabeled data that is considered to be data outside the category,and use the remaining data combined with labeled data to optimize the model of this epoch.

The method of screening unlabeled samples is as follows:

where Nsis a hyperparameter,which is the number of samples selected for training from the unlabeled sample datasetU.Through Eq.(2),we have selected the current training model and sorted the firstNsamples with the largest predicted probability of unlabeled samples.In the current situation,the proportion of out-of-class samples in these samples is lower than the proportion of all unlabeled datasets.Using the filtered unlabeled samples to train the model through Eq.(1),the recognition accuracy of the model can be further improved.

2.2 Threshold Filtering

However,in the model training process,it is unknown how many out-of-class labeled samples are mixed in the unlabeled data,so the choice of Nsneeds to be verified through experiments.To this end,we have further adjusted the method of selecting unlabeled samples:

whereτis a hyperparameter,which represents the sample prediction probability threshold.We use Eq.(3)to select samples with the maximum predicted probability of the model greater thanτto form an unlabeled datasetU′′,and the finally selected unlabeled sample set.

In this way,in the initial stage of model training,a certain number of samples can be selected to train the model,and at the same time,when the model has a certain performance,more intra-class data can be selected to optimize the recognition performance of the model.

Through the pre-selection rule,the unlabeled model can be filtered at the beginning of each epoch of semi-supervised training,and the unlabeled sample set for training can be selected.

Despite the pre-selection rules,there are still some out-of-class samples among the unlabeled samples.Especially in the initial training stage,when the sample set selected by Eq.(2) is used for training,the model has a large number of fake labels for the fake labels of the samples.This situation will weaken the training effect of the model and affect the final recognition accuracy of the model.Therefore,referring to the pseudo-labeling method[14],we introduce a prediction threshold to decide whether to use pseudo-labeled samples:

whereτ′is the threshold hyper-parameter used to ensure the accuracy of pseudo-labeled samples during the training process.

Through threshold filtering,in the training process of the model,pseudo-labeled data can be automatically processed in real-time,and pseudo-labeled data with a larger prediction probability is used to optimize the recognition accuracy of the model.

2.3 Loss Function

3 Experiments and Results

This section demonstrates the effectiveness of our proposed method by way of experimental comparison.First,we introduced the experimental dataset,including the division of training set and test set,the division of labeled data and unlabeled data in the training set,and the selection of in-class samples and out-of-class samples.Then we introduced the implementation details and experimental environment of the proposed method.Finally,by comparing with existing methods,the effectiveness of our proposed method is confirmed.

3.1 Dataset Description

To evaluate the effectiveness of the method,the MSTAR dataset is used in the experiments,which is a public dataset created by the U.S.Air Force Laboratory and widely used in SAR target recognition.The MSTAR dataset is divided into two sub-datasets,a training dataset obtained at a 17°depression angle,and a test dataset captured at a 15°depression angle.In each data subset,there are ten types of SAR images of military vehicles with ground targets.The ten types contained in the dataset are 2S1,BMP2,BRDM2,BTR60,BTR70,D7,T62,T72,ZIL131,and ZSU234.

In our experiment,we randomly select 10 samples from each of the six categories 2S1,BMP2,BRDM2,BTR60,BTR70 and D7,a total of 60 samples in 17°depression angle is used as labeled samples.Then 1260 samples are selected as unlabeled samples from all samples.We control the distribution of labeled samples and unlabeled samples by controlling the proportions of the first six categories in the unlabeled samples.For example,when the proportion of in-class samples of unlabeled samples is 50%,630 out of 1260 samples are samples in the first 6 categories,and the rest are samples in the last four categories of T62,T72,ZIL131 and ZSU234.When the within-class sample ratio of unlabeled samples is 100%,all 1,260 samples are the first 6 class samples.

Then,we selected 50 SAR target images with 15°depressions for each category in six categories,including 2S1,BMP2,BRDM2,BTR60,BTR70 and D7,a total of 300 images are used as the validation set.Finally,in each category,145 images that are different from the previous 50 images are selected,a total of 870 images are used as the test set.

3.2 Implementation Details

For our experiments,we use the“Wide-ResNet-28-2”architecture in Wide Residual Networks[23]as our backbone network,and a batch of 32 images and 200 batch as an epoch.The model is trained for 120 epochs by Adam solver with a learning rate of 0.002.Weight decay decaying weights by 0.02 at each update for the training model is used as a regularization method.We set the number of training samples in the unlabeled sample datasetNsto 630,the sample prediction probability thresholdτis set to 0.8,and the threshold hyper-parameterτ′is set to 0.9.The main configuration of the employed computer is:Graphics Processing Unit(GPU):GeForce RTX 2080Ti;operating system:Ubuntu 18.04;running software:Python 3.7.

3.3 Metrics

In order to quantitatively evaluate the proposed method,we use accuracy as a performance indicator,which is defined as:

where TP,FN,TN,and FP represent the number of true positives,false negatives,true negatives,and false positives.

3.4 Experimental Results and Analysis

In this section,we compare our proposed method with several state-of-the-art semi-supervised learning methods,including∏-model,mean teacher,mixmatch and SCA,to confirm the performance of our proposed method.∏-model uses consistency regularization to use unlabeled data,and encourages the interpolation between the prediction of the unlabeled instance and the prediction enhanced by the random data to tend to zero.Mean teacher performs an exponential moving average of parameters on the label predicted by the model on the unlabeled data on the basis of∏-model to optimize the model.Mixmatch and SCA are interpolated through mix-up to mix labeled data and pseudo-labeled unlabeled data to optimize the model.

For the above methods,we use PyTorch to re-implement them and apply them to the same model(wide-resnet-28-2)to ensure fair comparison.At the same time,we randomly selected 5 sets of labeled data,unlabeled data,validation set and test set.All methods are tested on these 5 sets of data,and the model that achieves the best recognition effect for each method on the validation set is selected,and the average value of the recognition accuracy of the model on the test set is used as the final result.Moreover,we also compare with the supervised learning method that simply trains a deep neural network on the small labeled dataset as the baseline method.In the case of only 10 labeled samples in each category,using the same model and data to continue training,supervised learning achieved recognition accuracy of 70.39%.

We conducted experiments under the condition that the proportion of samples in the category to the unlabeled dataset is 0.5,0.6,0.7,0.8,0.9 and 1,and the recognition results are shown in Tab.1.

Table 1:Comparison of the proposed method with the other methods

As can be seen from Tab.1,our method achieves the best results under all in-class sample ratios.The experimental results show that the semi-supervised method proposed in this paper can achieve the best results in the case of unlabeled samples with out-of-class data and no out-of-class data.At the same time,the recognition accuracy of all methods is higher than the case of using only labeled data,which also shows that semi-supervised learning has certain advantages.

However,the recognition accuracy rate decreases as the proportion of unlabeled data mixed with out-of-class samples increases.When the mixing ratio of out-of-class samples is 50%,the recognition accuracy obtained by the∏-model method is only 5.06% higher than the recognition accuracy of supervised learning.The recognition progress of other methods is also below 90%,which shows that out-of-class samples have a greater impact on the recognition accuracy of the semi-supervised learning model.

In addition,when only a small number of out-of-class samples are included,our method has a huge advantage.When the proportion of in-class samples in unlabeled samples exceeds 60%,the recognition accuracy of our method is higher than 90%.Compared with other methods,it shows a huge problem.When the ratio of in-class samples to unlabeled samples is 0.9,our method achieves a recognition accuracy of 95.46%.This recognition accuracy rate is much higher than the recognition accuracy rates of other methods in the same situation.In actual SAR target recognition applications,it is difficult to ensure that unlabeled samples are not mixed with out-of-class samples.The semi-supervised learning method we proposed can achieve higher recognition accuracy in this case.Therefore,our method is effective.

At the same time,our method also obtains the highest recognition accuracy when all samples are within-class samples.This shows that the simple screening of samples in the training process of our proposed method can also improve the recognition accuracy of the semi-supervised learning model.

In order to show the recognition of each category by our method more clearly,Fig.3 shows the confusion matrix of supervised learning method using only 10 labeled samples per class and Fig.4 shows the confusion matrix of the method proposed in this paper with 10 labeled samples per class and 1260 unlabeled samples containing 80%of the in-class samples.

Figure 3:Confusion matrix of supervised learning method on the test set of the MSTAR dataset

Figure 4:Confusion matrix of our proposed method on the test set of the MSTAR dataset

It can be seen from Figs.3 and 4 that our method can greatly improve the model’s recognition accuracy of the SAR target.For the categories with low model recognition accuracy obtained by the supervised learning method,such as 2S1,BRDM2,BTR60,BTR70,etc.,the accuracy is greatly improved.The recognition accuracy of 2S1 was increased from 65%to 91%,the recognition accuracy of 2BRDM2 was increased from 68%to 97%,the recognition accuracy of 2BTR60 was increased from 62%to 93%,and the recognition accuracy of 2BTR70 was increased from 66%to 93%.

3.5 Time Analysis

In this section,we discuss the time required to train and test our method,other semi-supervised learning methods,and supervised learning methods.Tab.2 records the training and testing time of supervised learning method,∏-model,Mean teacher,Mixmatch,SCA,and the proposed method in our experimental environment.

Table 2:Time analysis of the proposed method and other methods

It can be seen from Tab.2 that from the training time of each epoch,the supervised learning method takes the shortest time.The training time for a single epoch is about 7 s.Other semi-supervised learning methods have little difference in time.The training time of a single epoch of the semisupervised learning method∏-model is 16.68 s,and the training time of a single epoch of our method is 24.69 s.

We believe that the time gap between supervised learning and semi-supervised learning comes from the fact that consistency regularization requires pseudo-labeling of unlabeled data,which takes a certain amount of time.Although our method consumes the most time,in all methods in this experiment,the model was trained for 120 epochs.So overall,compared with the supervised learning method,the training time of our method is only half an hour longer than the supervised learning to train the model.Compared with the huge improvement in recognition accuracy,a small amount of time is acceptable.

For the test time of each image,since all methods use the same model,the SAR image recognition method is the same,so the test time is almost the same,and the test time of all methods for each image is within 1 s.In actual use,more attention is paid to the recognition time of a single SAR image,and semi-supervised learning does not increase the time required for testing.This also proves that the semi-supervised learning method has practical value in improving the accuracy of SAR recognition.

4 Conclusion

In this paper,we propose a threshold-filtered semi-supervised learning SAR target recognition method to improve the recognition accuracy of the semi-supervised learning method when there are out-of-class samples in SAR unlabeled data.Our method uses pre-selection rules and threshold filtering to filter unlabeled data in real-time during the training process,eliminate unlabeled samples considered to be out-of-class data,and use the remaining samples to train and optimize the SAR recognition model.Experimental results prove that when there are out-of-class samples in SAR unlabeled samples,compared with other semi-supervised learning,our method has higher recognition accuracy under the same model conditions.In the future research work,we will try to use the filtered out-of-class unlabeled data to carry out the SAR out-of-class target recognition technology based on semi-supervised learning technology under the condition of open set.

Funding Statement:Our research is funded by National Key R&D Program of China(2021YFC3320302),Fundamental Research(JCKY2020210B019),Natural Science Foundation of Heilongjiang Province(No.F2018006),and Network threat depth analysis software(KY10800210013).

Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.