Automated Testing of Mobile Applications: A Systematic Map and Review

Abstract

Context: Software testing is a costly and time-consuming activity. Automated testing of mobile applications is considered complex and difficult. Indeed, several factors such as a variety of inputs (user, context, and environment) that a mobile application normally requires, and the heterogeneity of the technologies make automated testing not a trivial task. Two of the major challenges for automated testing are creation of the appropriate test cases scenarios and to decide in which devices to perform the testing. Objective: This paper reports on a systematic map and review. Automated testing approaches for mobile applications, testing techniques, and empirical assessment are identified, mapped, and characterized. We investigate the major challenges in automated testing of mobile applications. An analysis and synthesis of these studies is conducted. Method: A systematic mapping and systematic literature review research method has been conducted for identifying and aggregating evidence about automated testing of mobile applications. Results: A total 83 studies were identified. The results were tabulated and synthesized to provide recommendations to practitioners about automated testing of mobile applications. The main approaches identified were model-based testing (30%), capture/replay (15.5%), model-learning testing (10%), systematic testing (7.5%), fuzz testing (7.5%), random testing (5%) and scripted based testing (2.5%). Conclusions: In recent years, the number of proposals for automated software testing of mobile applications has increased. In 40% of the studies, the testing techniques use GUI-based models of the application. Further research is needed, in order to improve the creation of effective and efficient models for automated testing of mobile applications.