Method of military software security and vulnerability testing based on process mutation*

JIN Li-ya(金麗亞), WANG Rong-hui(王榮輝), XIAO Qing(肖 慶)

(1. Department of Information Engineering, Academy of Armored Force Engineering, Beijing 100072, China;2. Research Department, Academy of Armored Force Engineering, Beijing 100072, China)

Method of military software security and vulnerability testing based on process mutation*

JIN Li-ya(金麗亞)1, WANG Rong-hui(王榮輝)2, XIAO Qing(肖 慶)1

(1. Department of Information Engineering, Academy of Armored Force Engineering, Beijing 100072, China;2. Research Department, Academy of Armored Force Engineering, Beijing 100072, China)

To solve the problems caused by military software security issues, this paper firstly introduces a new software fault injection technique, namely main static fault injection method: program mutation. And then source code for testing this algorithm is put forward. On this basis buffer overflow testing based on program mutation is conducted. Finally several military software source codes for buffer overflow testing are tested using deficiency tracking system (DTS) tool, Experimental results show the effectiveness of the proposed algorithm.

military software testing； fault injection； buffer overflow； source code scanning

0 Introduction

Military software refers to software system which carries out battle operations or business management. With the development of information technology and new military revolution all over the world, a large number of military softwares are widely used in the army. Military software presents the features of diversification, complication and intelligent. The most remarkable characteristic that military software differs from commercial software is that the malfunction or security flaws would cause huge economic losses or casualties. In a sense, the quality and security of military software have a pivotal role for winning over a battle.

Practice has proved that software testing is an effective method for mining security flaws of the military software. However, military software security flaws, such as buffer overflow, format string and competitive conditions continuously occur in the military software. Program mutation technology can test security flaws by inserting fault function into source code so as to cause exceptions, and observe their effects on system security[1].

This paper puts forward an algorithm for buffer overflow testing based on program mutation analysis. This algorithm calculates function abstract for C/C++ string operations, memory operations, and so on. Then the record of information in function invocation points is compared and judged so as to test the vulnerability of military software buffer overflow potential.

1 Software fault injection

1.1 Summary of software fault injection

As a testing technique, software fault injection method produces faults in a conscious way according to the fault model and subsequently applies them to the testing system so as to accelerate the system to produce error and failure. According to injection mode, software fault injection can be divided into dynamic fault injection and static fault injection.

The dynamic fault injection refers to trigger fault injection through a certain sort of trigger in a particular state or condition by changing the executed instructions and data, etc.[2]. The merit of dynamic fault injection is that it can realistically simulate the operation of the tested system, while the shortcoming of dynamic fault injection is that it needs a monitor to monitor the status of the target system. Thus the monitor perhaps preempts the other tasks running and occupies too much resource of the target system, which will affects the normal operation of the system.

The static fault injection refers to changing the code by pressing pile load and executing the object program memory. The merits of static fault injection include taking up less system resources and no need for monitor, therefore it can better keep the original timing system[3]. In addition, this method has the platform neutrality only in view of the code itself without considering the code which requires the operating platform.

1.2 Program mutation testing

Program mutation testing is a kind of test technology based on static software fault injection, and its history can be traced back to 1971[4]. This method simulates software failure depending on the type of software failure and distribution to change source codes. The general principle of program mutation testing is that the variation of implied fault represents mistakes that programmers often make. Such mistakes are carefully put into original program after being simply changed, and then fault sets are created, which are called mutation program set. Process of program mutation analysis is shown in Fig.1[5].

Fig.1 Flow chart of mutation analysis

Step 1: Submit test procedure to the automated test system.

Step 2: Test system creates mutations according to the test program.

Step 3: Test and eliminate the equivalent mutation as much as possible through equivalent mutation analysis program.

Step 4: Test system automatically generates test case sets. The test case sets firstly run in the original program, and then in all mutations. If the output of mutation is different from the output of the original program, the mutation is thought to be killed. Killed mutation will no longer be tested by the rest of the test cases.

Step 5: Delete the invalid test cases which can not kill mutations.

Step 6: Calculate the mutation point. If the mutation point is 1.0, it means that all mutations are tested.

Step 7: Whether the mutation point achieves the acceptable limit point. If not, go to step 4, or else go to step 8.

Step 8: Complete the testing. Testers check the expected output of the effective test cases. If there are mistakes, modify the original program, and if necessary, return to the first step for the next round of mutation testing.

2 Algorithm of buffer overflow testing based on process mutation

At present, most of the military software security vulnerabilities are buffer overflows, and their attack is aiming at disrupting the normal execution of program which has certain privileges. This can make the attacker take control of the program, change the program execution flow, overflow a buffer without border inspection, skip system inspection and jump to the attack codes.

Testing the buffer overflow vulnerabilities can scan source codes again. Principle of the algorithm can be summarized as follows: Firstly, mark the places in the program which need to be checked, then run the program by selecting the test sets (for example, legal, illegal or special character input data). If regular error occurs when program runs, create a data set depending on the data state when the program executes the selected position, and then modify one single variable in the data set, and make the program execute on the new data set. If output of the program meets the abnormal judgment condition, record the time that security issues occur, modify the variables in data set and execute the program on the new set until all the input test sets are completed, record the total number of tests n and the number of security issues nf, use 1-nf/n as the security index that the program is in the place.

Set P as the source program to check; T as the place to check in P; D0as the legal input data set; D1as the illegal input data set; D2as the special character input data set; n as the number of input test case sets and threshold as the abnormal judgment condition. Flow chart of the algorithm is shown in Fig.2.

Fig.2 Flow chart of buffer overflow testing algorithm based on program mutation

3 Experimental results and analysis

Testing tool DTS is a Java and C/C++ source defect detection tool. This paper tests buffer overflow vulnerabilities for five military software source codes with DTS by use of the proposed method. The experimental results are shown in Table 1.

Table 1 lists the size of the project, analysis time, number of the inspection point(IP), number of the actual defects which are artificial confirmed, number of false alarm, and finally lists the percentage of the performed mutation operators which are used in the source program.

The experiment results show that not all the marked locations have buffer overflows, only referring to those they need software developers to check more carefully. And determine whether they will overflow by input data or need to strengthen the robustness.

Table 1 Experimental results

4 Conclusion

According to the principle of fault injection and program mutation, the military software program buffer overflow testing algorithm is put forward based on program mutation. Five military software source codes are tested with DTS tool using the algorithm. Experimental results show that the algorithm has good practicability, high testing efficiency and low rate of non-response.

[1] Chess B, West J. Secure programming with static analysis. Addison-Wesley Professional, Boston, USA, 2007.

[2] Cowan C, Barringer M, Beattie S, et al. Format guard: automatic protection from printf format string. In: Proceedings of the 10th onference on USENIX Security Symposium, Washington, USA, 2001: 15-20.

[3] ZHANG Nan, ZHANG Jian-hua, FU Chun-chang, et al. Dynamic detection and prevention of buffer overflow attacks. Journal of Southwest University for Nationalities, 2007, 33(5): 1172-1174.

[4] Schuler D, Dallmeier V, Zeller A. Efficient mutation testing by checking invariant violations. In: Proceedings of the 18th International Symposium on Software Testing and Analysis, New York, USA, 2009: 69-80.

[5] Ammann P, Offutt J. Instruction to software testing. Cambridge University Press, UK, 2008: 180-181.

date： 2013-03-11

Innovation Fund of Academy (No.2012CJ066)

JIN Li-ya (jinliya@sina.com)

CLD number： TP311.562 Document code： A

1674-8042(2013)03-0228-03

10.3969/j.issn.1674-8042.2013.03.006