Automated Software Testing Using Covering Arrays

Abstract

Modern society is increasingly dependent on the quality of software systems. Software testing is a widely used approach to ensure software quality. Since exhaustive testing is impractical due to resource constraints, it is necessary to strike a balance between test efforts and quality assurance. Interaction testing is one approach to marrying these two qualities. It characterizes the system under test by a set of parameters and the respective test values (domain size) for each parameter. Instead of testing all possible combinations of values for all the parameters, interaction testing constructs a covering array as a test set to cover all the t-way combinations (i.e., combinations involving t parameters, where t is referred to as the degree of interaction and is usually small). Each combination of values in a set of parameters represents a possible interaction among those parameters. The rationale of interaction testing is that not every interaction contributes to every fault, and many faults can be exposed by the interactions among a small number of parameters. Empirical studies have shown that interaction testing can significantly reduce the number of tests while still detecting faults effectively. This thesis mainly describes the GUI of an interaction testing tool called FireEye. FireEye constructs covering arrays that provide multi-way coverage for up to 6-way testing. We focus on the design and implementation of the GUI for FireEye. The GUI is developed using Java Swing. Software testing demands a great deal of time and money, so it is necessary to save these resources wherever possible. One way to meet time and resource constraints is to develop a user friendly GUI that can save the tester some time while also being able to rapidly generate test cases. Various goals such as ease of use, interactivity and portability were kept in mind while designing the GUI paradigm (model of interaction) for FireEye. Other features of FireEye's GUI included the following: decreasing the time necessary for the tester to analyze the test configuration, minimal hardware requirements, easy installation, and no high end software requirements.