• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    High Performance Classification of Android Malware Using Ensemble Machine Learning

    2022-08-24 12:57:48PagnchakneatOukandWooguilPak
    Computers Materials&Continua 2022年7期

    Pagnchakneat C.Oukand Wooguil Pak

    1Department of Computer Engineering, Keimyung University, Daegu, 42601, Korea

    2Department of Information and Communication Engineering, Yeungnam University,Gyeongsan, Gyeongbuk, 38541, Korea

    Abstract: Although Android becomes a leading operating system in market,Android users suffer from security threats due to malwares.To protect users from the threats, the solutions to detect and identify the malware variant are essential.However, modern malware evades existing solutions by applying code obfuscation and native code.To resolve this problem, we introduce an ensemble-based malware classification algorithm using malware family grouping.The proposed family grouping algorithm finds the optimal combination of families belonging to the same group while the total number of families is fixed to the optimal total number.It also adopts unified feature extraction technique for handling seamless both bytecode and native code.We propose a unique feature selection algorithm that improves classification performance and time simultaneously.2-gram based features are generated from the instructions and segments, and then selected by using multiple filters to choose most effective features.Through extensive simulation with many obfuscated and native code malware applications, we confirm that it can classify malwares with high accuracy and short processing time.Most existing approaches failed to achieve classification speed and detection time simultaneously.Therefore, the approach can help Android users to keep themselves safe from various and evolving cyber-attacks very effectively.

    Keywords: Android malware classification; family grouping; native code;obfuscation; unified feature extraction

    1 Introduction

    With over 2 billion active Android devices worldwide [1,2], Android is considered as the worldly most popular mobile operating system.The rising popularity of Android system is accompanied by the increasing number of malwares that targets this system.As a result, there are many different kinds of malware variance available on vast Android landscape.Android malware becomes one of the biggest critical issues in Android system.According to [3], there are almost 8,400 malware applications were found daily and they target on Android device in order to steal the private information for malicious activity.Although Google uses its own scanning tool,i.e.,Play Play Protect security suite for years, malware issues still exist.Recently,McAfee lab found out several obfuscated malwares in Google Store from the same publisher [4].Furthermore, ESET analyzer also found out that there are three specific malware families can invade from Google Store detection [5].Finally, there is also native malware that invade Google Store [6] found by Fortinet analyzer.

    To keep smartphone users safe from the threat of the malware, the solution to detect and identify the malware variant is needed.Thus, many various approaches using static or dynamic analysis have been proposed [7-12].In traditional ways, malware developers use flaws of system permission [7-9,13-17] to load malicious function in the malware.Recently, malware is evolving very fast in terms of the total number and the sophistication level [1,4-6].More seriously, modern malware evades detection from most of the existing solutions by applying code obfuscation and native code [10,11].Such techniques cause increased detection time or even detection failure formost existing anti-malware solutions [8,9,14,15,18,19].Thus, we need a new approach to overcome modern malwares.

    Additionally, using the obfuscation technique, the malware developer easily generates multiple variants of malware that can cause misdetection in most existing systems [10,11].Malware developers also manually make variants of original malwares and incapacitate anti-malware programs.Such variants and original malwares can be categorized as the same family.However, most previous literature only focuses on malware detection to inform the user whether it is benign or malware without informing the malware family name.

    Malware classification identifies the belonging malware family of the malware.Malware classification has many advantages compared to malware detection [8,20].First, it helps existing solutions to detect a malware with higher accuracy.Existing detection technique extracts unique signatures from each malware and use the signatures to detect it; thus, they have a high failure probability to detect malware variants created manually or with code obfuscation[10,11].If we know the family that existing malwares belong to, we can build signature based on common features among malwares in the same family.Such signatures are very robust against obfuscation or manual variations.Second, a family name from classification is very helpful to decide how to deal with unknown malwares.From other known malwares in the same family, we can use the same remedy to remove the malware and recover the damaged system.Third, it also helps to prevent the system from malwares.It is very difficult to configure security policies one-by-one for every single malware to protect the system for all malwares.Instead, if we configure them according to each family, the configuration cost can be low and scalable.

    However, most existing anti-malware approaches still focus on malware detection rather than classification.To cope with the issue, we present a new solution based on unified static feature extraction that can classify the malware application, which is not incapacitated to obfuscated and native malwares.This study optimizes the feature set to the small size of the feature dimension.We also suggest an efficient grouping technique for malware families.Additionally, we propose an ensemble learning technique to classify malwares using results from different groups of malware families to achieve high scalability in terms of accuracy and processing time.

    In summary of our proposed approach, this study makes the following contribution to state of the art malware classification for Android.

    ?High accurate group-based classification: This study introduces an efficient way to group the malware families by applying Boosted Random Forest (BRF) to a dataset and recursively combining the largest family with the smallest one.Such a grouping technique greatly helps to increase the classification performance and reduce processing time.

    ?Ensemble machine learning: We thoroughly analyze 28 malware families including various obfuscated and native code malwares.Based on the result, we determine the size of groups and design the optimal ensemble learning technique with multiple Random Forests (RFs).It provides high classification accuracy compared to latest solutions.

    ?Lightweight and unified feature selection: This study unifies the feature extraction to seamlessly process both bytecode and native code.After obtaining initial features, it selects only best features by intersecting the initial features.Since the classification results depend on feature selection, we propose a very practical feature extraction and selection algorithm, and therefore,achieving a good performance with a large scale dataset.

    ?Robust malware classification: Although many existing works fail to detect or classify modern malwares using code obfuscation and native code, our algorithm provides classification results with high accuracy.To achieve this performance.We gathered various malware applications from many available sources that include traditional, obfuscation, and native malware for analysis.Based on the result, we elaborately design our algorithms such as predefined irrelevant features and 2-gram based features to achieve high classification accuracy regardless of malware types.

    The rest of our paper is organized as follows.We present some selected related work in Section 2.We introduce and describe our proposed system in Section 3.We analyze evaluation results in Section 4 and conclude it in Section 5.

    2 Related Work

    In the state-of-art Android malware analysis, it can be classified into two categories: Malware detection and Malware classification.Malware detection is the technique that focuses only on two target classes whether the application is benign or malware.On the other hand, Malware classification looks deeper into each malware characteristics to identify belonging family of malware application.As the result, malware classification provides a more scalable solution to malware analysis on specific malware over malware detection.

    For better understanding, this study, we will first discuss related work on the techniques that only focus on detecting malware in Android.We also discuss on the signature-based and machine learning based techniques that use to detect malware application.We then cover about the techniques that focus on malware classification.We also look at ensemble classification and random feature ensemble technique that are used to classify the malware.

    2.1 Malware Detection

    There are two popular techniques to detect the malware.The first technique is signature-based malware detection; the technique solely depends on a specific pattern such as Control Flow Graph(CFG) [9,21,22] or known malicious functions used in malicious applications.The second technique is machine learning [13,15,16] based malware detection that trains feature set extracted from malware applications with recent machine learning algorithms to detect the malware.

    2.1.1 DroidNative

    DroidNative [9] is the system that focuses on detecting native malware application.DroidNative generates the Annotated Control Flow Graph (ACFG) [23] and Sliding Window of Difference(SWOD) using Malware Analyze Intermediate Language (MAIL) [24].It compares generated ACFG with the built model to identify the malware.Furthermore, it reduces the comparison time using similarity comparison based on machine learning.DroidNative can effectively detect native malware and traditional malware applications.However, DroidNative has no mention of obfuscated malwares;thus, the system may be vulnerable to obfuscated malwares.Furthermore, DroidNative can only perform the detection not classification.Since DroidNative uses CFG, it can detect only pre-existing malwares in the trained model.

    2.1.2 DroidSieve

    DroidSieve [13] is one of unique systems that can detect obfuscated malware.It uses only static analysis to extract the feature from the application.DroidSieve extracts all possible features for malware analysis, including bytecode, strings, and resource files.Then, it selects only fine-grained features for malware detection.It can easily detect obfuscated malware as well as other traditional malware.It also argues that it can handle native malware.However, it only relies on metadata to detect native one, and therefore it cannot robustly detect native malware applications.

    2.2 Malware Classification

    The fundamental techniques for classifying malware are literally the same with detecting malware but using the technique without any consideration for classification characteristics achieves usually low accuracy.To increase the classification accuracy, many approaches have been proposed: the random feature set ensemble [25], ensemble different classification algorithms [7], and ensemble supervised model with un-supervised model [8].However, each literature has its own limitation on malware classifications.

    2.2.1 Ensemble Clustering and Classifier

    To mitigate the classification issues, ensemble clustering and classifier (EC2) [8] proposed a way to ensemble the results from clustering and classifying processes.EC2 aims to understand the characteristics of the malware family, making more robust on malware’s signatures.It uses static and dynamic analyses to extract the feature for the malware classification.Static analysis in EC2 extracts activities, services, and other metadata, whereas dynamic analysis extracts read and write states, SMS,and network.It ensembles six different algorithms including three clustering algorithms and three classification algorithms to obtain the final result.However, EC2 does not include any information about malware variants using obfuscation and native code.Therefore, it can be vulnerable to obfuscated and native malwares.Furthermore, clustering can cause huge overheads to the processing performance of the overall system.

    2.2.2 RevealDroid

    RevealDroid [7] extracts all possible features from malware applications using only static analysis method.RevealDroid extracts application programming interface (API), native code executable and linkable format (ELF), and Android metadata as feature sets from malwares.Thus, it is able to detect obfuscated or native malwares.RevealDroid builds multiple classifiers according to the number of targets.For example, if the model contains five target families, then RevealDroid will generate five classifiers to detect each family whether it corresponds to the target family.The system focuses on selecting the fine-grained features from the dataset for better scalability and performance.

    2.2.3 IagoDroid

    IagoDroid [25] is the extended study of RevealDroid that focuses on malware classification.IagoDroid selects random features from the original feature set, and then uses multiple RevealDroid classifiers to classify the random malware on each random feature set.Each classifier uses each own disjoint feature set, and IagoDroid ensembles the results from the classifiers to generate the final result.Then, it compares the ensembled result with the original RevealDroid result to finalize the classification.However, IagoDroid only focuses on reducing misclassification more than increasing the accuracy of the overall system.Thus, it is limited for practical deployment in the real system.

    3 Proposed Algorithm

    They still have much room for improvement in terms of classification detection and run-time although many malware classification algorithms have been proposed so far.Currently, no systems are available to classify obfuscated malwares.For native malwares, only a few systems exist but they also fail to achieve a high classification accuracy.We note that most approaches do not rely on application code but non-application code such as metadata.The code of the application contains plenty of information which can be directly used for classifying malwares.However, extracting features from the application code is time-consuming [25,26].Moreover, native malware contains bytecode and native code together, we need to process bytecode and native code, separately.Such an approach makes the classification process more complicated and slower.Therefore, in this study, we focus on unifying separate feature extractions for both bytecode and native library code by converting everything into a native binary.Android Runtime (ART) [25,27] makes it possible to achieve unifying feature extraction since ART can convert Android bytecode into native binary code; thus, we can perform feature extraction using only native code.It will greatly simplify the feature extraction process, and it makes malware classification more efficient in terms of speed and accuracy.

    For better classification, some previous literatures on malware classifications use ensemble techniques [7, 8,14].The ensemble is the process that combines the many algorithms or model to find out the best result for each feature set.However, it has some issues such as insufficient accuracy or long processing time.To cope with the issues, this study proposes a different way to generate the ensemble model for boosting the performance of the classification system.We build groups of the malware based on each label using BRF for the entire dataset then train it using BRF for each group separately.We will explain each procedure in more detail.

    3.1 System Overview

    Fig.1 shows the overview of the proposed approach.It consists of four phases: Data Preprocessing, Feature Engineering, Family Grouping, and Ensemble Model Generation.In data preprocessing,application bytecode and native library are converted into native binary code using ART and disassembly [11,16,28].In Feature Engineering, we perform feature extraction from the applications converted in the pre-processing.We then remove the irrelevance features and perform multiple feature selections to choose common features from the feature sets.We intersect the multiple filter-method based-feature selections together to obtain the finely chosen features set.Then, we categorize malware families into small groups with different characteristic based on the result of BRF.Lastly, we ensemble each result of BRF for each group to obtain the result for the target family.Now, we will explain each step in detail.

    Figure 1: Overview of building the classification model for our proposed approach

    3.2 Data Preprocessing

    In the preprocessing stage, we first extract bytecode and native library from the Android malware applications.Starting from Lollipop, Android introduced a new function called ahead of time (AOT)compilation at the installation time [26].Using Android runtime (ART), we convert extracted bytecode into native binary code.As a result,ART makes it possible for the system to unify the process of feature extraction.The system disassembles native code obtained from both bytecode and native library into a readable assembly text file for feature extraction.

    In this study, we use the disassembly technique proposed by [29].First, the system decodes everything on the native binary into an assembly including data as well as code.We should note that it may generate incorrect instructions especially when we try to disassemble data.Second, we decode everything on the native binary into assembly without the data.This process can sometimes fail to find valid embedded machine instructions in data.Thus, we combine the first and the second assembly techniques to remove the data by comparing the instructions between both assemblies.

    3.3 Feature Engineering

    We remove the irrelevance instructions [9] from the dataset obtained from the data preprocessing.Irrelevance instructions mean instructions used too widely or used for only specific purposes, so they are not regarded as ones used for malware operation.For example, INVD is the instruction used to invalidate internal caches, consequently, inappropriate for malware analysis.During the preprocessing process,we remove such irrelevance instructions to increase the performance and reduce overall feature dimension in feature extraction.

    In this study, the proposed system uses 2-gram, i.e.,‘pair of segment and instruction’or‘consecutive instructions’to generate the feature set.In addition, we use all loadable code segments [30] to generate the features.Using all the segments is essential to analyze all operations of a target application.

    Tab.1 shows all segments used for feature extraction in this study.All loadable code segments include .rodata, .oatexec, and .oatdata.In original ELF, .rodata contains read-only data without any machine instruction; however, .rodata in Android ELF contains two sub-segments: .oatexec and.oatdata [30], which are able to embed machine instructions.Thus, we should include segments such as .rodata in our feature extraction process.

    Table 1: List of segments used to generate features on the proposed system

    Let us take an example as Fig.2.Assuming that we have .rodata and .text segments in disassembled binary first, our system selects .rodata and removes irrelevance instructions from this segment, i.e.,wbinvd.Then, the system starts to count the instruction in .rodata segment; there are two‘a(chǎn)dd’instruction in this segment, and it counts that instruction by combining it with the segment as‘rodata_add:2’.After counting all the instruction in the segment, it starts to combine instruction with instruction; if the subsequence instruction of previous‘a(chǎn)dd’is‘pop’, the system counts sequence of the combined instructions as‘a(chǎn)dd_pop:1’.Once all the subsequence instructions in .rodata are counted,the system continues the same process with the next segment .text.The last instruction from .rodata is used to combine with the very first instruction of .text segment.From previous work, it is known that there are instruction combinations mainly used only in malicious applications.By using information about these combinations as features, normal apps and malicious apps can be distinguished.Long n-gram are mainly used in existing studies, but short n-grams are used in the proposed method.When code obfuscation is applied to malicious applications, the order of the instructions is changed severely,which makes features based on long n-gram not working well.On the other hand, the proposed method uses short n-gram,making the features based on short n-grams robust even for code obfuscation.Fig.2 illustrates the feature extraction in our proposed system.

    Figure 2: Example of 2-gram based feature generation using segments and instructions

    To increase the classification accuracy and reduce the overall feature dimension size, the system applies feature selection to the obtained dataset [31].We decided to use a filter-method to select the features since it chooses features depending on the general characteristics of the data independent of any dataset without involving any training model [32].Thus, it leads to a faster feature selection pipeline.Each filter-method selects a different dimension of feature set; thus, to obtain the final feature set, we intersect the feature sets together to get the final feature set.

    Assume that we have two feature sets A and B selected from an original feature set using filtermethods A and B, respectively.Both filter-methods can result in different feature dimension sizes.For example, feature set A contains“pushf_add, add_add, add_pop, pop_pushf, pushf_add, pop_add”while feature B contains“add_add, add_pop, pop_add, mov_call, call_lret”.The system starts to intersect two feature sets by keeping only the same features that exist in both feature sets.After intersection, final features, i.e.,“add_add, add_pop, pop_add”are used to train the model.Fig.3 shows how the feature set is intersected.

    Figure 3: Filter-method based feature set intersection process

    3.4 Family Grouping

    To group malware families, we can consider two different techniques.The first technique is to cluster the whole instance usingk-means algorithm.The algorithm uses unsupervised learning to group the instance with similar characteristics together using centroid.However, this technique causes the performance overhead to the classification system, thus we do not consider this clustering technique.

    The second technique is to group the families based on both the family size and the misclassified results of each family from BRF.We group the malware dataset based on family size since it has less overhead compared to the clustering technique.Before the grouping process, we obtain classification results using BRF model trained with all families, and initially, each family corresponds to each group.In the grouping process, the system selects the largest and smallest groups.For the largest group, the system checks whether the most misclassified group is the same as the smallest in the BRF applied classification results.If it is the same, it tries to the same procedure with the next smallest group.If it differs, the two groups are merged into a larger group.This process is repeated until the total family size in the largest group reaches to the predefined maximum family size in a group.If it reaches to the maximum size, the system selects the next largest group and repeats the grouping procedure.Fig.4 shows the overall grouping process.

    Figure 4: Malware family grouping process, where fiis the set of the i-th malware family that includes all malwares belonging to the family as elements, Gmaxis the maximum number of families in a group,and |·| is the cardinality of the set

    For a better explanation for the grouping process, we assume that we have eight malware families as shown in Tab.2, where the maximum family size in a group is four.The system selectsgeinimiandboxeras the largest and the smallest groups, respectively.However, the most misclassified family ofgeinimiisboxer, and thus, the system cannot mergegeiminiwithboxer.The system chooses the next smallest group, i.e.,Fakerunand merges it withgeinimi.On the other hand,boxerwill be merged with the next largest group,i.e.,Fakeinst.This step is repeated until each group contains four families.Tab.3 shows the result of the example dataset in Tab.2.

    Table 2: Example of malware families for grouping

    Table 3: Final grouping results for the example dataset in Tab.2

    Through experiments, we can see that the classification performance of BRF can be changed according to the number of families in the training dataset.Thus, the number is an important factor of the proposed algorithm.In addition, which families are included in the training dataset can affect performance.If the dataset is composed of very similar families, the classification accuracy will be inevitably low.To avoid this situation and improve the classification performance, the proposed method allocates malware families which have low mis-classification rate into the same training dataset, called group.

    3.5 Ensemble Model Generation

    We train an RF model with each group.RF is a well-known ensemble model of decision tree where every tree is built with randomly selected features of the dataset [33].RF has a unique capability to train a large-scale dataset with a high training speed.Therefore, we choose RF to trains each group malware families to build the model.For simplicity, we use the same settings for allmodel.The features and the number of trees for training model in BRF are the same for each model.We use 10-fold crossvalidation in every model.

    Finally, we ensemble trained group models to finalize the result.The ensemble approach has some variations.Some approaches ensemble the results from different learning algorithms while others ensemble results from different random feature sets.In this study, we ensemble results from each model of groups with the same algorithms and the same feature set.We apply an application to all group models, and then choose the family with the highest scores as the classification result.Fig.5 shows integrated procedures of building our classification machine-learning model.

    Figure 5: Flowchart of the proposed system

    4 Performance Evaluation

    4.1 Evaluation Environment

    To evaluate the performance of classification for our system, we conducted extensive experiments to compare with existing work.First, we briefly explain the evaluation dataset for our experiment.Second, we discuss metrics used to evaluate the system.Finally, we compare the result of our system with previous literature.All evaluations were conducted in a desktop equipped with one Intel Core i7-4790 K 4 GHz, 16 GB RAM, and two 256 GB SSDs configured to Redundant Array of Independent Disks (RAID) 0.

    The dataset for the experiment was gathered from many different sources: Marvin, Information Security Centre of Excellence (ISCX), Drebin, and PRAGuard.There are three types of malware in our dataset: Un-obfuscated, obfuscated, and native ones.The dataset contains originally 43 malware families.However, very small families with one or two malwares only were removed since they have too few samples to be applied for cross-validation.Finally, 28 families are contained in the dataset.The whole dataset consists of 15,890 malware applications, where the sizes of obfuscated and native malware are 7,560 and 1,390, respectively.We retained 70% of each dataset for training and 30% for the testing procedures.

    We compare the classification model in term of F-Score and run-time per sample.F-Score is defined as the harmonic mean of precision and recall as shown in (1).The number of malware application has been growing very fast; thus, it is critical to perform high-speed malware classification with a large scale malware.Therefore, we also evaluate our system in term of run-time per sample.Run-time per sample is the time that system spent from feature extraction to identify the target, so it is very important metric since it indicates the scalability of the system.

    4.2 Evaluation Results

    4.2.1 Classification Accuracy

    The main challenge for our proposed algorithm to increase the performance of malware classification is to determine that how many and which filter-method algorithms should be used.It is also important to find how many families should belong to one group.To find such optimal values,we tried on filter-method combinations as many as we can.We also measured the performance as the number of families in a group increases.Through these experiments, we can find the best filtermethods and the family number in a group.For filter-method, we considered six algorithms such as Pearson Correlation,Spearman correlation, Mutual information,Kendall correlation[34],Fisher,and Chi Square.

    We measured the classification accuracy using F-Score as mentioned previously.The F-Score is highest when the number of filter-methods and the number of families in a group are four and three,respectively, as depicted in Fig.6.The highest score is 97.16% and it is very high compared to existing classification algorithms.

    Figure 6: Comparison of F-Scores according to the number of filter-methods (NF) with the maximum number of families in a group increasing (Gmax)

    To ensure our system against malware classification in Android, we compared this study to two previous literatures on malware classification: RevealDroid and IagoDroid.We decide to choose RevealDroid and IagoDroid in our comparison because they are only practically available malware classification systems that can detect and classify obfuscated and native malware [35].

    RevealDroid is the system that supports both malware detection and classification.RevealDroid focuses on the manifest,library,APIcall,Reflective,and native call to extract all possible features from every malware application.RevealDroid produces many classifiers to perform malware classification.For example, if there aren-target families in the dataset, RevealDroid will producenclassifiers for each target family.Thus, using our dataset, RevealDroid generates 28 classifiers based on a regression tree for each family.However, RevealDroid was designed to focus on malware detection more than classification thus it achieves moderate F-Score.Through the experiment, our system outperforms RevealDroid in terms of F-Score, achieving around 16% higher score.Fig.7 show the F-Score comparison between our proposed and RevealDroid.

    Figure 7: Comparison of the classification accuracy between our proposed algorithm and related work

    IagoDroid is the extended version of RevealDroid; it was designed to support better malware classification than RevealDroid.IagoDroid generates many RevealDroid classifiers by randomly selecting the features among original features, and ensembles the result from multiple classifiers to obtain the result.Since IagoDroid shows different F-Scores according to the number of classifiers,we configured the number as the best value, i.e., four to achieve the highest F-Scores.However, we outperform IagoDroid with the best number of classifiers roughly 10% in term of F-Score.Fig.7 also shows the F-Score comparison results between our proposed and IagoDroid.

    In addition, there are 4,767 obfuscated apps included in the test dataset.For obfuscated apps,RevealDroid and IagoDroid have 77% and 81% on F-score, whereas the proposed method has 96%.Therefore, it shows a tendency similar to the performance for the total test dataset.

    Tab.4 shows the best combination of filter-methods for intersection.For highest performance,we choose four filter-method algorithms: Chi Square, Mutual information, Kendall correlation, and Spearmen correlation.Tab.5 also shows the name of families belong to each group when we achieve the best score.

    Table 4: Intersected filter-methods when the F-Score is highest according to the number of filtermethods

    Table 5: Grouping result when the highest score is achieved

    4.2.2 Classification Time

    The classification time is a very important metric since it determines whether the algorithm can be used in a real environment.We also measured the classification time according to the number of filter-methods and the number of families in a group, where filter-methods with the highest F-Score are selected.As shown in Fig.8, the time generally increases with the number of filter-methods and the number of families in a group.It achieves moderate classification time when it achieves the best F-Score, i.e., four filter-methods and three families in a group.

    We also compared our algorithm with RevealDroid and IagoDroid in terms of the classification time.Our system shows 6.5 times faster than RevealDroid, and 11 times faster than IagoDroid.Since IagoDroid internally utilizes many RevealDroid classifiers and ensembles the result from multiple classifiers, it shows the worst classification time.For our algorithm, it achieves fastest classification though efficient feature selection algorithm.Fig.9 show the classification time results between our proposed algorithm and related work.

    Figure 8: Comparison of average classification time according to the number of filter-methods (NF)with the maximum number of families in a group (Gmax) increasing

    Figure 9:Comparison of average classification time between our proposed algorithm and related work,where the total number apps is 15,890

    Fig.10 shows the total run-time according to the various filter-method numbers and the total family numbers in a group.The run-time includes consumed time from feature selection, grouping,learning, and classification with 15,890 applications.The run-time increases in proportion to the number of families in a group.With three families in a group, we can achieve the best F-Score and very short run-time.

    Figure 10: Comparison of the total run-times according to the number of filter-methods (NF) with the number of families in a group (Gmax) increasing, where the run time includes learning and classification time

    5 Conclusion

    Although Android malware is one of the most serious threats in Android landscape, it becomes more difficult to classify recent obfuscate and native code malwares even with the latest solutions.To overcome this issue, we developed malware type independent unified feature extraction, small but effective feature selection, fast BRF based grouping, and accurate ensemble machine learning with multiple RFs.These techniques are integrated into one high-performance classification algorithm and it can classify Android malwares regardless they are obfuscated or native code-based malwares.Through extensive simulation with many types of malware, we could prove its high performance such as high accuracy and fast classification speed.Malware classification is known as more difficult than malware detection.Thus, many existing works have not been able to improve the detection accuracy and fast classification speed simultaneously.Due to such a high performance of the algorithm,we expect that our approach can help Android users to keep them from various and evolving cyber-attacks.

    Funding Statement:This work was supported by the National Research Foundation of Korea (NRF)grant funded by the Korea government (MSIT)(NRF-2019R1F1A1062320).

    Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.

    中国国产av一级| 欧美性猛交╳xxx乱大交人| 人妻制服诱惑在线中文字幕| 日韩一区二区三区影片| 在现免费观看毛片| а√天堂www在线а√下载| 91久久精品国产一区二区三区| 国内少妇人妻偷人精品xxx网站| 麻豆一二三区av精品| 在线播放国产精品三级| 一级黄片播放器| 少妇裸体淫交视频免费看高清| 自拍偷自拍亚洲精品老妇| 99热精品在线国产| 女人被狂操c到高潮| 亚洲aⅴ乱码一区二区在线播放| 国产精品人妻久久久影院| 我的老师免费观看完整版| 欧美日韩国产亚洲二区| 啦啦啦观看免费观看视频高清| 久久国内精品自在自线图片| 久久精品影院6| av天堂在线播放| 欧美+日韩+精品| 国内久久婷婷六月综合欲色啪| 精品一区二区三区视频在线| 狂野欧美激情性xxxx在线观看| 18禁黄网站禁片免费观看直播| 亚洲av二区三区四区| 天堂网av新在线| 色视频www国产| 综合色丁香网| 少妇丰满av| 国产午夜福利久久久久久| 欧美潮喷喷水| 国产精品伦人一区二区| 成人无遮挡网站| 成年女人永久免费观看视频| av免费观看日本| 熟女电影av网| 欧美一级a爱片免费观看看| 夜夜夜夜夜久久久久| 18禁在线无遮挡免费观看视频| 国产精品免费一区二区三区在线| 欧美又色又爽又黄视频| 此物有八面人人有两片| 伦理电影大哥的女人| 国产精品人妻久久久久久| 成人亚洲精品av一区二区| 一边摸一边抽搐一进一小说| 成人午夜高清在线视频| 97在线视频观看| 国产一区亚洲一区在线观看| 国产精品人妻久久久影院| 日本五十路高清| 网址你懂的国产日韩在线| 国产探花极品一区二区| 人人妻人人看人人澡| 欧美极品一区二区三区四区| 国产精品一区www在线观看| 校园人妻丝袜中文字幕| 久久精品国产清高在天天线| 91久久精品电影网| 亚洲经典国产精华液单| 亚洲三级黄色毛片| 51国产日韩欧美| 免费搜索国产男女视频| 色噜噜av男人的天堂激情| 婷婷亚洲欧美| 亚洲在线自拍视频| 最近中文字幕高清免费大全6| 国产真实伦视频高清在线观看| 精品99又大又爽又粗少妇毛片| 毛片一级片免费看久久久久| АⅤ资源中文在线天堂| 国产 一区精品| 人妻夜夜爽99麻豆av| 久久久久免费精品人妻一区二区| 午夜精品国产一区二区电影 | 精品久久久噜噜| a级毛色黄片| 最近手机中文字幕大全| 偷拍熟女少妇极品色| 久久中文看片网| 少妇丰满av| 在线观看免费视频日本深夜| 国产日韩欧美在线精品| 99久久人妻综合| 亚洲久久久久久中文字幕| 国产精品蜜桃在线观看 | 国产一级毛片在线| 国产淫片久久久久久久久| 日日干狠狠操夜夜爽| 嫩草影院精品99| 国产私拍福利视频在线观看| 免费看a级黄色片| 此物有八面人人有两片| 国产黄片美女视频| 久久久久久久久久久丰满| 美女高潮的动态| 日本欧美国产在线视频| 在线观看午夜福利视频| 久久久久久久午夜电影| 免费电影在线观看免费观看| 97人妻精品一区二区三区麻豆| 日韩av不卡免费在线播放| 免费看光身美女| 啦啦啦韩国在线观看视频| 九九爱精品视频在线观看| 一进一出抽搐动态| av卡一久久| 国产一区二区三区在线臀色熟女| 99久久精品热视频| 国产精品无大码| 边亲边吃奶的免费视频| 国产精品一区二区三区四区免费观看| 亚洲人成网站在线观看播放| 国产 一区 欧美 日韩| 又爽又黄无遮挡网站| 最近2019中文字幕mv第一页| 成人无遮挡网站| 99热网站在线观看| 国产一区二区三区av在线 | 久久综合国产亚洲精品| 久久久久久久午夜电影| 中文精品一卡2卡3卡4更新| 国产蜜桃级精品一区二区三区| 婷婷色综合大香蕉| 男人舔奶头视频| 一进一出抽搐gif免费好疼| 精品人妻偷拍中文字幕| 黄色日韩在线| 小说图片视频综合网站| 欧美+日韩+精品| 搞女人的毛片| 熟妇人妻久久中文字幕3abv| 少妇熟女aⅴ在线视频| 国语自产精品视频在线第100页| 日本黄色视频三级网站网址| 22中文网久久字幕| 三级国产精品欧美在线观看| 国产老妇女一区| 午夜福利高清视频| 人人妻人人澡欧美一区二区| 日产精品乱码卡一卡2卡三| 亚洲欧美日韩无卡精品| 少妇被粗大猛烈的视频| 久久久成人免费电影| 国产私拍福利视频在线观看| 97超视频在线观看视频| 国产一区二区激情短视频| 淫秽高清视频在线观看| 国产精品日韩av在线免费观看| 中文字幕熟女人妻在线| 免费看美女性在线毛片视频| 欧美在线一区亚洲| 国产一区二区激情短视频| 久久久久久久久久黄片| 美女黄网站色视频| 亚洲一级一片aⅴ在线观看| 日韩欧美一区二区三区在线观看| 亚洲欧美成人综合另类久久久 | 亚洲av.av天堂| 亚洲av男天堂| 久久精品国产清高在天天线| 久久精品国产清高在天天线| 亚洲av.av天堂| 99热只有精品国产| 91麻豆精品激情在线观看国产| 日日撸夜夜添| 97超视频在线观看视频| 国产一区二区三区在线臀色熟女| 欧美性感艳星| 狂野欧美激情性xxxx在线观看| 在线观看美女被高潮喷水网站| 国产成人a∨麻豆精品| 久久久色成人| 日韩一区二区三区影片| 国产大屁股一区二区在线视频| 中文字幕免费在线视频6| 99热这里只有精品一区| 观看免费一级毛片| 欧美日本视频| 国产黄片美女视频| 日本-黄色视频高清免费观看| 成人毛片a级毛片在线播放| 大型黄色视频在线免费观看| 国产极品天堂在线| 免费观看a级毛片全部| 色噜噜av男人的天堂激情| 国产成人91sexporn| 国产一区二区亚洲精品在线观看| 综合色av麻豆| 青青草视频在线视频观看| 亚洲精品久久国产高清桃花| 熟女人妻精品中文字幕| 少妇人妻精品综合一区二区 | 91精品国产九色| 蜜桃久久精品国产亚洲av| 联通29元200g的流量卡| 看片在线看免费视频| 国产在线男女| 一个人看视频在线观看www免费| 在线a可以看的网站| 国产乱人视频| 男女做爰动态图高潮gif福利片| 成人漫画全彩无遮挡| 超碰av人人做人人爽久久| 天美传媒精品一区二区| 春色校园在线视频观看| 啦啦啦啦在线视频资源| 中文字幕人妻熟人妻熟丝袜美| 日韩在线高清观看一区二区三区| 久久鲁丝午夜福利片| 成人亚洲欧美一区二区av| 日韩欧美 国产精品| 亚洲成人中文字幕在线播放| 丰满的人妻完整版| 久久久久久久午夜电影| 少妇裸体淫交视频免费看高清| 久久韩国三级中文字幕| 国产片特级美女逼逼视频| 精品日产1卡2卡| 99精品在免费线老司机午夜| 欧美潮喷喷水| 色5月婷婷丁香| 黄色日韩在线| 人妻夜夜爽99麻豆av| 最好的美女福利视频网| 26uuu在线亚洲综合色| 在线免费观看的www视频| 99在线人妻在线中文字幕| 亚洲最大成人手机在线| 99热6这里只有精品| 亚洲人与动物交配视频| 女人被狂操c到高潮| 99久久成人亚洲精品观看| 成人一区二区视频在线观看| 国产精品99久久久久久久久| 久久鲁丝午夜福利片| 欧美日韩一区二区视频在线观看视频在线 | www日本黄色视频网| av免费在线看不卡| 色哟哟·www| 成人午夜高清在线视频| 给我免费播放毛片高清在线观看| 少妇人妻精品综合一区二区 | 久久久久久久久中文| 天堂中文最新版在线下载 | 老女人水多毛片| 麻豆成人av视频| ponron亚洲| av免费在线看不卡| 欧美成人一区二区免费高清观看| 最近手机中文字幕大全| 久久99热6这里只有精品| 国产 一区 欧美 日韩| 少妇熟女aⅴ在线视频| 欧美又色又爽又黄视频| 99热只有精品国产| 精品免费久久久久久久清纯| 亚洲精品亚洲一区二区| 亚洲精品日韩av片在线观看| 亚洲内射少妇av| 日韩欧美精品免费久久| 午夜激情欧美在线| 亚洲在线观看片| 网址你懂的国产日韩在线| 边亲边吃奶的免费视频| 3wmmmm亚洲av在线观看| or卡值多少钱| 最近手机中文字幕大全| 女的被弄到高潮叫床怎么办| 久久久精品大字幕| 国产探花极品一区二区| 亚洲一区二区三区色噜噜| 国产 一区精品| 亚洲av成人av| 日日摸夜夜添夜夜爱| 精品人妻熟女av久视频| 亚洲精品乱码久久久v下载方式| 国内精品久久久久精免费| 看免费成人av毛片| 午夜福利高清视频| 99久久精品热视频| 黄色视频,在线免费观看| 最近手机中文字幕大全| 性欧美人与动物交配| 国内精品久久久久精免费| 黄色配什么色好看| 久久热精品热| 久久久成人免费电影| 欧美性猛交╳xxx乱大交人| 午夜福利视频1000在线观看| 久久99热6这里只有精品| 亚洲在线观看片| 成年版毛片免费区| 2022亚洲国产成人精品| 丰满乱子伦码专区| 亚洲欧美清纯卡通| 欧美丝袜亚洲另类| 精品久久久噜噜| 亚洲第一区二区三区不卡| 久久久久久久久大av| 夜夜爽天天搞| 国产精品蜜桃在线观看 | 欧美激情国产日韩精品一区| 国产成人精品久久久久久| 在线观看一区二区三区| 日本免费a在线| 久久久久性生活片| 亚洲丝袜综合中文字幕| 欧美三级亚洲精品| 日韩欧美一区二区三区在线观看| 最近视频中文字幕2019在线8| 国产精品美女特级片免费视频播放器| 国产 一区 欧美 日韩| 免费av不卡在线播放| 深夜a级毛片| 99九九线精品视频在线观看视频| 亚洲人与动物交配视频| 国产亚洲av片在线观看秒播厂 | 精品人妻偷拍中文字幕| 久久精品国产亚洲av香蕉五月| av在线天堂中文字幕| 成人av在线播放网站| 乱码一卡2卡4卡精品| 国产精品国产高清国产av| 午夜福利在线在线| 99在线人妻在线中文字幕| 能在线免费看毛片的网站| 成人二区视频| 18+在线观看网站| 一区二区三区高清视频在线| 一边亲一边摸免费视频| 毛片女人毛片| 国产久久久一区二区三区| 一个人看视频在线观看www免费| 亚洲成人精品中文字幕电影| 国产黄片视频在线免费观看| 国产亚洲欧美98| 日日摸夜夜添夜夜添av毛片| 午夜福利高清视频| 精品久久久久久久久av| 国产精品一区www在线观看| 小蜜桃在线观看免费完整版高清| 亚洲乱码一区二区免费版| 国产视频首页在线观看| 日本撒尿小便嘘嘘汇集6| 99精品在免费线老司机午夜| 成人毛片60女人毛片免费| 亚洲精品粉嫩美女一区| 秋霞在线观看毛片| 日韩成人伦理影院| 美女大奶头视频| 美女xxoo啪啪120秒动态图| 成人av在线播放网站| 久久这里有精品视频免费| 午夜福利高清视频| 激情 狠狠 欧美| 久99久视频精品免费| 欧美潮喷喷水| 国产精品久久久久久亚洲av鲁大| 少妇的逼好多水| 成人av在线播放网站| 欧美最黄视频在线播放免费| 身体一侧抽搐| 国产精品日韩av在线免费观看| 一进一出抽搐动态| 国产探花在线观看一区二区| 永久网站在线| 国产一区亚洲一区在线观看| 最近手机中文字幕大全| 国产又黄又爽又无遮挡在线| 国产老妇女一区| 91麻豆精品激情在线观看国产| 毛片一级片免费看久久久久| 欧洲精品卡2卡3卡4卡5卡区| 婷婷亚洲欧美| avwww免费| 丝袜美腿在线中文| 日韩强制内射视频| 久久午夜亚洲精品久久| 国产成人a区在线观看| 免费观看在线日韩| 爱豆传媒免费全集在线观看| 18禁裸乳无遮挡免费网站照片| 又爽又黄a免费视频| 亚洲人成网站在线播放欧美日韩| 亚洲av男天堂| 午夜免费激情av| 亚洲av成人精品一区久久| 不卡一级毛片| 国产国拍精品亚洲av在线观看| 在线免费十八禁| 赤兔流量卡办理| 久久99精品国语久久久| 国产精品女同一区二区软件| 国产69精品久久久久777片| 国产伦一二天堂av在线观看| 久久国产乱子免费精品| 亚洲av熟女| 99热这里只有精品一区| 变态另类成人亚洲欧美熟女| 免费电影在线观看免费观看| 免费观看的影片在线观看| 91精品一卡2卡3卡4卡| av天堂在线播放| 听说在线观看完整版免费高清| 又爽又黄无遮挡网站| 国产精品一区二区在线观看99 | 久久精品综合一区二区三区| 男人狂女人下面高潮的视频| 不卡一级毛片| 国产视频首页在线观看| 99热这里只有精品一区| 免费观看在线日韩| 亚洲无线在线观看| 91精品国产九色| 欧美日韩一区二区视频在线观看视频在线 | 精品人妻熟女av久视频| 免费av观看视频| 日韩中字成人| 久久国内精品自在自线图片| 日韩欧美在线乱码| 亚洲国产精品sss在线观看| videossex国产| 免费人成在线观看视频色| 亚洲乱码一区二区免费版| 成人午夜高清在线视频| 精品久久久久久久久久免费视频| 亚洲精品日韩在线中文字幕 | 成人午夜精彩视频在线观看| 99riav亚洲国产免费| 成年av动漫网址| 你懂的网址亚洲精品在线观看 | 国产精品av视频在线免费观看| 日韩高清综合在线| 欧美潮喷喷水| 久久人人精品亚洲av| 男女视频在线观看网站免费| 成人美女网站在线观看视频| 久久欧美精品欧美久久欧美| 91在线精品国自产拍蜜月| 国产大屁股一区二区在线视频| 一进一出抽搐gif免费好疼| 18禁在线播放成人免费| 欧美高清性xxxxhd video| 国产午夜福利久久久久久| or卡值多少钱| 国产精品久久电影中文字幕| 成年女人看的毛片在线观看| 国产一区二区亚洲精品在线观看| 国产淫片久久久久久久久| 亚洲三级黄色毛片| 18禁黄网站禁片免费观看直播| 男人的好看免费观看在线视频| 欧美激情久久久久久爽电影| 亚洲最大成人av| 亚洲欧美日韩高清在线视频| 国产av在哪里看| 老司机影院成人| 婷婷亚洲欧美| 精品午夜福利在线看| 成熟少妇高潮喷水视频| 寂寞人妻少妇视频99o| 狠狠狠狠99中文字幕| 国内精品久久久久精免费| 免费观看a级毛片全部| 国产老妇伦熟女老妇高清| 天天一区二区日本电影三级| 国产成年人精品一区二区| 亚洲成a人片在线一区二区| 国产亚洲精品久久久久久毛片| 久久久精品94久久精品| 一本精品99久久精品77| 天天躁夜夜躁狠狠久久av| 久久韩国三级中文字幕| 亚洲丝袜综合中文字幕| 少妇丰满av| 可以在线观看的亚洲视频| 白带黄色成豆腐渣| 国产一区二区激情短视频| 精品99又大又爽又粗少妇毛片| 成人毛片a级毛片在线播放| 一个人观看的视频www高清免费观看| 国产在线男女| 久久精品国产清高在天天线| 夜夜夜夜夜久久久久| 日本成人三级电影网站| 狂野欧美激情性xxxx在线观看| 亚洲国产精品久久男人天堂| 亚洲激情五月婷婷啪啪| 国产精品久久视频播放| 97超碰精品成人国产| 久久精品国产清高在天天线| 青春草国产在线视频 | 亚洲精品乱码久久久久久按摩| 亚洲五月天丁香| 日韩欧美在线乱码| 色吧在线观看| 男人狂女人下面高潮的视频| 日本黄大片高清| 亚洲aⅴ乱码一区二区在线播放| 国产探花在线观看一区二区| 26uuu在线亚洲综合色| 国产麻豆成人av免费视频| 少妇熟女欧美另类| 99视频精品全部免费 在线| 美女xxoo啪啪120秒动态图| 日本一本二区三区精品| 亚洲精品成人久久久久久| 熟女人妻精品中文字幕| 亚洲成人av在线免费| 超碰av人人做人人爽久久| 久久久精品大字幕| 男女下面进入的视频免费午夜| 最近的中文字幕免费完整| av天堂中文字幕网| 一级毛片久久久久久久久女| 日韩大尺度精品在线看网址| 亚洲欧洲国产日韩| a级毛色黄片| 色哟哟哟哟哟哟| 天天躁日日操中文字幕| 中文资源天堂在线| 色吧在线观看| 校园春色视频在线观看| 神马国产精品三级电影在线观看| 亚洲精品国产av成人精品| 国内精品一区二区在线观看| 色综合站精品国产| 久久久a久久爽久久v久久| 99热网站在线观看| 国产精品永久免费网站| 18禁黄网站禁片免费观看直播| 精品无人区乱码1区二区| 成人毛片a级毛片在线播放| 99久久精品国产国产毛片| 婷婷亚洲欧美| 国产黄片视频在线免费观看| 日韩在线高清观看一区二区三区| 国产亚洲91精品色在线| 中文在线观看免费www的网站| 少妇丰满av| 晚上一个人看的免费电影| 亚洲自拍偷在线| 国产成人aa在线观看| 亚洲经典国产精华液单| 一个人免费在线观看电影| 97超碰精品成人国产| 国产一区二区亚洲精品在线观看| 亚洲av成人精品一区久久| 亚洲av男天堂| 久久精品国产鲁丝片午夜精品| 少妇人妻一区二区三区视频| 免费在线观看成人毛片| 91久久精品电影网| 日韩亚洲欧美综合| 日本-黄色视频高清免费观看| 亚洲精品影视一区二区三区av| 久久精品国产亚洲av涩爱 | 亚洲精品乱码久久久v下载方式| 深夜a级毛片| 免费在线观看成人毛片| 欧美精品一区二区大全| 亚洲最大成人手机在线| 亚洲人成网站在线播放欧美日韩| av卡一久久| 国产精品.久久久| 色播亚洲综合网| 看免费成人av毛片| 最近中文字幕高清免费大全6| 国产伦在线观看视频一区| 中国国产av一级| 国产亚洲精品av在线| 淫秽高清视频在线观看| 国产激情偷乱视频一区二区| 亚洲av成人av| 日韩大尺度精品在线看网址| 免费不卡的大黄色大毛片视频在线观看 | 亚洲人成网站高清观看| 国产精品av视频在线免费观看| 成人亚洲精品av一区二区| 哪个播放器可以免费观看大片| 国产日本99.免费观看| 波多野结衣高清无吗| 美女 人体艺术 gogo| 亚洲精品乱码久久久久久按摩| 能在线免费观看的黄片| 成人毛片a级毛片在线播放| 国产老妇伦熟女老妇高清| 国产一级毛片七仙女欲春2| 美女cb高潮喷水在线观看| 高清午夜精品一区二区三区 | 久久精品国产亚洲av涩爱 | 免费一级毛片在线播放高清视频| 国产av不卡久久| 99精品在免费线老司机午夜| 国产国拍精品亚洲av在线观看| 大又大粗又爽又黄少妇毛片口| 长腿黑丝高跟| 日韩一区二区视频免费看| 97在线视频观看| 在线国产一区二区在线| 噜噜噜噜噜久久久久久91| 日韩在线高清观看一区二区三区| 中文字幕熟女人妻在线| 日日摸夜夜添夜夜添av毛片| av在线天堂中文字幕| 国产精品99久久久久久久久| 搡老妇女老女人老熟妇| 成人高潮视频无遮挡免费网站| 久久6这里有精品| 一个人看的www免费观看视频|