Warning: Parameter 1 to Language::getMagic() expected to be a reference, value given in /usr/share/mediawiki/includes/StubObject.php on line 58
Dissertationsthema Stephan Weißleder – GRK-Wiki

Dissertationsthema Stephan Weißleder

Aus GRK-Wiki

Wechseln zu: Navigation, Suche

Test Models and Coverage Criteria for Automatic Model-Based Test Generation with UML State Machines

Topic of the Thesis

Testing is one of the most important means to validate the correctness of systems. The costs of testing are put at 50% of the overall project costs. There are many efforts to decrease the costs for testing, e.g. by introducing automation. There are many different testing techniques, processes, scopes, and targets. This thesis is focused on functional model-based testing. Functional testing consists of comparing the system under test (SUT) to a specification. A functional test detects a failure if the observed and the specified behavior of the SUT differ. Model-based testing is about using models as specifications. Several modeling languages have been applied to create test models, e.g. B, Z, the Unified Modeling Language (UML), or the Object Constraint Language (OCL). Model-based testing allows to derive test suites automatically from formal test models. This thesis is focused on automatic model-based test generation with UML state machines and OCL expressions. Although testing with state charts, state diagrams, or state machines has been investigated for several decades, there are still many unexplored aspects and issues left to be solved. Testing is often incomplete, i.e. cannot cover all possible system behaviors. There are several heuristic means to measure the quality of test suites, e.g. fault detection, mutation analysis, or coverage criteria. These means of quality measurement can also be used to decide when to stop testing. This thesis is concentrated on coverage criteria. There are many different kinds of coverage criteria, e.g. focused on data flow, control flow, transition sequences, or boundary values. In this thesis, we will present new approaches, e.g., to combine test models or to simulate and combine coverage criteria.

Contribution of the Thesis

This thesis is focused on automatic model-based test generation with UML state machines as test models and coverage criteria that are applied to them. The inputs of the model-based test generation process are a test model and coverage criteria to satisfy. The application of a coverage criterion to the test model results in a set of test model-specific test goals. The test goals and the test model are used to automatically generate the test suite.

The contributions of this thesis are focused on the combination of different test models, the relations of test models and coverage criteria, and the combination of coverage criteria. This thesis contains five contributions

First, we introduce a novel test generation algorithm based on UML state machines and class diagrams with OCL expressions. The advantage of this algorithm is the combination of abstract test case creation and boundary value analysis. The major contribution is the combination of the corresponding, e.g., transition-based and boundary-based, coverage criteria.

As the second contribution, we investigate the mutual dependency of state machines and coverage criteria in model-based testing. We transform state machines and evaluate the impact of the applied coverage criteria. The major contribution is that the application of any feasible coverage criterion to the transformed state machine can have the same effect as the application of almost any other feasible coverage criterion to the original state machine. We present an experience report of an industrial cooperation that shows the importance of model transformations for model-based test generation. We define simulated coverage criteria satisfaction and present corresponding model transformations. The most important effect is that the satisfaction of a strong coverage criterion on the original state machine can be simulated by satisfying a weak coverage criterion on a transformed state machine. We also show new ways to combine and define coverage criteria. They can also be simulated with existing coverage criteria. This second contribution can be used together with the afore presented test generation approach.

The third contribution is the combination of different test models, which can be used together with the two previously presented contributions. We present the combination of UML state machines with structural models like UML class diagrams and with behavioral models like UML interaction diagrams. Since automatic test generation depends on the provided test models, this combination is advantageous for automatic test generation. Both proposed combinations of test models have advantages that go beyond the separate application of the corresponding single test models. New coverage criteria are presented that are focused on combined test models.

Fourth, we investigate the application of coverage criteria on test models and focus on the resulting set of test-model-specific test goals. The test cases are generated based on the order of the test goals. The contribution is an empirical evaluation of the impact of the test goal order on the efficiency of the generated test suite, e.g. the average number of test cases to execute until detecting a failure. Since this is also a general contribution to automatic model-based test generation, its advantages can be combined with the advantages of the afore three contributions.

These four contributions are interrelated and all support automatic model-based test generation. They are furthermore substantiated by the developed tool support and corresponding case studies. We work on two Eclipse plugins based on EMF and UML 2.1: The model-based test generation tool ParTeG (Partition Test Generator) implements the novel test generation approach described as the first contribution. It partly supports the transformation and the combination of test models, and performs the ordering of test goals. The tool Coverage Simulator is currently under development. Its goal is to provide a wide range of test model transformations to support the simulated satisfaction of coverage criteria as presented in the second contribution. ParTeG has been used to generate tests for standard examples as well as academic and industrial test models.

Persönliche Werkzeuge