A tertiary study on model-based testing areas, tools and challenges: Preliminary results

Abstract

Context: Model-based testing (MBT) is one of the most studied approaches by secondary studies in the area of software testing. A tertiary study that aggregates knowledge from secondary studies on MBT can be useful to both academia and industry. Objective: The goal of this study is to identify and characterize secondary studies in model-based testing, in terms of the areas, tools and challenges they have investigated. Method: We conducted a tertiary study in MBT. Our systematic mapping of secondary studies included 12 literature surveys and 10 systematic reviews over the period 1996–2016.Results: We found that the two most studied areas of MBT are UML models and Transition-based notations. We also found that only 5 studies compared and classified MBT tools. The main challenges and limitations found were related to the need for more empirical evidence that supports the selection of MBT approaches and tools. Conclusions: Not many systematic reviews on MBT were found, consequently some areas still lack secondary studies: test execution aspects, language types, model dynamics, and some model paradigms and generation methods. We thus encourage the MBT community to perform further systematic reviews and mapping studies, following known protocols and reporting procedures, in order to increase the quality and quantity of empirical studies in MBT.