Mutation Testing to Evaluate Android Applications

According to International Data Corporation (IDC), Android accounted for 81% of all smartphone shipments in the third quarter of 2013. With the increase in smartphones, mobile application installs from Google Play have reached more than 48 billion app installs as revealed by Sundar Pichai, senior vice president, Chrome and Apps at Google I/O 2013. Thus, Android application development has become increasingly popular with the need of an effective software solution for application development and testing. While most Android applications are developed in Java, the Android framework is quite different fromm typical Java frameworks. Thus, while development or testing of Android applications, the user needs to be aware of all the terms and concepts specific to the Android framework such as activity, views and the Android SDK in general.

Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, few test cases explicitly specify the assignments of valid values to the preferences, or configurations , under which they should be executed, and few existing mobile testing tools take the impact of preferences into account or provide help to testers in identifying and setting up the configurations for running the tests. This article presents the Prefest approach to effective testing of Android apps with preferences. Given an Android app and a set of test cases for the app, Prefest amplifies the test cases with a small number of configurations to exercise more behaviors and detect more bugs that are related to preferences. In an experimental evaluation conducted on real-world Android apps, amplified test cases produced by Prefest from automatically generated test cases covered significantly more code of the apps and detected seven real bugs, and the tool’s test amplification time was at the same order of magnitude as the running time of the input test cases. Prefest ’s effectiveness and efficiency in amplifying programmer-written test cases was comparable with that in amplifying automatically generated test cases.

Mutation testing is a fault based testing technique which examines the adequacy of test cases by generating faulty programs, called mutants, to be run against them. However, one of the pitfalls of mutation testing is its high computational cost, rendering it impractical for industry use. In this survey, we study mutation generation techniques, cost reduction methods and test-data generation approaches found in the literature, before presenting some tools and applications for mutation testing.

Mutant reduction based on dominance relation for weak mutation testing

Traditionally, mutation testing is used as test set generation and/or test evaluation criteria once it is considered a good fault model. This paper uses mutation testing for evaluating an automated static analyzer. Since static analyzers, in general, report a substantial number of false positive warnings, the intention of this study is to define a prioritization approach of static warnings based on their correspondence with mutations. On the other hand, knowing that Mutation Test has a high application cost, another possibility is to try to identify mutations of some specific mutation operators, which an automatic static analyzer is not adequate to detect. Therefore, this information can be used to prioritize the order of incrementally applying mutation operators considering, firstly, those with no correspondence with static warnings. In both cases, contributing to the establishment of incremental strategies on using automatic static analysis or mutation testing or even a combination of them. We used mutation operators as a fault model to evaluate the direct correspondence between mutations and static warnings. The main advantage of using mutation operators is that they generate a large number of programs containing faults of different types, which can be used to decide the ones most probable to be detected by static analyzers. We provide evidences on the correspondence between mutations and some types of static warnings. The results obtained for a set of 19 open-source programs indicate that: 1) static warnings may be prioritized based on their correspondence level with mutations; 2) specific set of mutation operators and their mutations may be prioritized based on their correspondence level with warnings. It is possible to provide an incremental testing strategy aiming at reducing the cost of both static analysis and mutation testing using the correspondence information between these activities/artifacts.

Android applications have experienced an outstanding growth in recent years with a projected figure of 378 billion in the number of application downloads by 2020. As a result, verification and validation of Android apps and their graphical user interface through testing is crucial for quality assurance. One way to assess quality assurance is through the use of mutation testing. Mutation testing modifies the code of the System Under Test in small ways in order to produce faulty versions of the program called mutants and compare them to the original version. However, the cost of mutation testing can be expensive and time consuming. We propose a graphical user-friendly Android mutation tool called Edroid whose primary purpose is to mutate Android's main components such as activities, services, content providers and broadcast receivers using the source code of XML files. We introduce fourteen Android mutation operators, where ten of them are new, that are derived from operators used in different fields such as mutation testing for Graphical User Interface and XML schema. We evaluate Edroid on five real world Android applications using testing methodologies such as random testing and black box testing. Results from these experiments demonstrate that Edroid facilitates the generation of mutants. In addition and equally important, the set of proposed mutation operators are effective in revealing errors that improve the quality of a given test suite.

Android applications are considered the most popular and desirable applications due to their friendly interface, various categories and available options, and supporting most of the available hardware in the market. Complexity to understand, design, develop, implement, and test android applications have been raised. Maintainability is a very important quality attribute that we should consider it seriously. However, it is one of the most difficult and costly attributes that can be achieved. The software metrics are used to predict and estimate the software maintainability value. There are several metrics and formulas that are used to measure and estimate the maintainability value. These metrics and formulas are derived from the different usages of the maintainability. This paper focus on analyzing and measuring the maintainability of Android mobile applications using Object Oriented metrics and Android Metrics. The purpose of this paper is to assess the impact all of these metrics on the maintainability of Android applications and choose the metrics that has the highest impact.

In the software Industry Android applications are the most demanded software it is possibly because of the advancement of society and technology. The Technological development of society and the dependence of the people getting bigger day by day on the android application from alarms to fitness routines, or some industrial software development management application tool depends on these Applications. Considering the growing prevalence of smartphones while travelling with an incorporated map application in it, the information related to weather report with respect to a location can be represented through an android application. The purpose of this research paper is to study all software requirements, technologies and the development process involved in development of an android application using kotlin programming language which implements functionalities like location access of the user, current weather report, date etc. Analyzing the technologies involved in development and also why such an android application is important for the people? This paper discusses all assessed features and enhancements of kotlin language according to its implementation in app development.

Mutation testing is a well known technique for estimating and improving the quality of test suites. Given a test suite T for a system S, mutation testing systematically creates mutants of S and executes T to measure how many mutants T detects. If T does not detect some (non-equivalent) mutants, T can be improved by adding test cases that detect those mutants. Mutants are created by applying mutation operators. Mutation operators are important because they define the characteristics of the system that are tested as well as the characteristics that are improved in the test suite. While mutation operators are well defined for a number of programming paradigms such as sequential or multi-threaded, to the best of our knowledge, mutation operators have not been defined for the actor programming model. In this paper, we define and classify mutation operators that can be used for mutation testing of actor programs.

In software testing process, test data generation represents an important step for high quality software, even for mobile devices. As proposed in previous works, a potential source for random data generation is represented by the UI layout files that are used for almost all mobile platforms (Android, iOS, Windows Phone/Mobile). This paper continues the previous work and presents a test data generation system based on Android layout files. The test data generator uses DSL files as input and generates test data that conform to several testing principles. The generated test data could be stored in XML files or any format required by the testing frameworks.

Mutation testing has a widespread reputation of being a rather powerful testing technique. However, its practical application requires the detection of equivalent mutants. Detecting equivalent mutants is cumbersome since it requires manual analysis, resulting in unbearable testing cost. To overcome this difficulty, researchers have proposed the use of mutant classification, an approach that helps isolating equivalent mutants. From this perspective, the present paper establishes and assesses possible mutant classification strategies. The conducted study suggests that while mutant classification can be useful in isolating equivalent mutants, it fails to kill some mutants. Indeed, the experimental results show that the proposed strategies achieve to kill approximately 95% of the introduced killable mutants.

The ratio of equivalent mutants: A key to analyzing mutation equivalence

Mutation testing can be used to measure the quality of a given test suite. But two flaws prevent wide acceptance. First, there are equivalent mutants - mutants that are semantically equivalent to the unmodified version and therefore unkillable. Manually identifying those mutants is time-consuming and error-prone. Second, initially there are often too few test cases. Mutation testing detects missing cases. But it is too time-consuming to manually write all the required test cases. This paper shows how to use symbolic execution to tackle both problems, i.e., to detect equivalent mutants and exclude them from further analysis, and to automatically generate test cases that kill the remaining mutants. Our evaluation uses a set of 252 publicly available mutants for which it is known that they are hard to classify. Despite the fact that detecting equivalent mutants is an undecidable problem in general, our fully automatic tool MutantDistiller correctly classifies all of them (13 equivalent, 239 non-equivalent). MutantDistiller also generates test cases that kill the nonequivalent mutants.

A probabilistic framework for mutation testing in deep neural networks

Testing every member of a product line individually is often impracticable due to large number of possible product configurations. Thus, feature models are frequently used to generate samples, i.e., subsets of product configurations under test. Besides the extensively studied combinatorial interaction testing (CIT) approach for coverage-driven sample generation, only few approaches exist so far adopting mutation testing to emulate faults in feature models to be detected by a sample. In this paper, we present a mutation-based sampling framework for fault-based product-line testing. We define a comprehensive catalog of atomic mutation operators on the graphical representation of feature models. This way, we are able (1) to also define complex mutation operators emulating more subtle faults, and (2) to classify operators semantically, e.g., to avoid redundant and equivalent mutants. We further introduce similarity-based mutant selection and higher order mutation strategies to reduce testing efforts. Our implementation is based on the graph transformation engine Henshin and is evaluated concerning effectiveness/efficiency trade-offs.