Accelerating Automated Android GUI Exploration with Widgets Grouping

Android Apps are frequently updated, every couple of weeks, to keep up with changing user, hardware and business demands. Correctness of App updates is checked through extensive testing. Recent research has proposed tools for automated GUI event generation in Android Apps. These techniques, however, are not efficient at checking App updates as the generated GUI events do not prioritise updates, and instead explore other App behaviours. We address this need in this paper with CAT (Change-focused Android GUI Testing). For App updates, at the source code or GUI level, CAT performs change impact analysis to identify GUI elements affected by the update. CAT then generates GUI event sequences to interact with the affected GUI elements. Our empirical evaluations using 21 publicly available open source and 2 commercial Android Apps demonstrate that CAT is able to automatically identify GUI elements affected by App updates, generate and execute GUI event sequences focusing on change-affected GUI elements. Comparison with two popular GUI event generation tools, DroidBot and DroidMate, revealed that CAT was more effective at interacting with the change-affected GUI elements. Finally, CAT was able to detect previously unknown change-related bugs in two open source Apps. Developers of the commercial Apps found CAT was more effective than their in-house GUI testing tool in interacting with changed elements and faster at detecting seeded bugs.

Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, few test cases explicitly specify the assignments of valid values to the preferences, or configurations , under which they should be executed, and few existing mobile testing tools take the impact of preferences into account or provide help to testers in identifying and setting up the configurations for running the tests. This article presents the Prefest approach to effective testing of Android apps with preferences. Given an Android app and a set of test cases for the app, Prefest amplifies the test cases with a small number of configurations to exercise more behaviors and detect more bugs that are related to preferences. In an experimental evaluation conducted on real-world Android apps, amplified test cases produced by Prefest from automatically generated test cases covered significantly more code of the apps and detected seven real bugs, and the tool’s test amplification time was at the same order of magnitude as the running time of the input test cases. Prefest ’s effectiveness and efficiency in amplifying programmer-written test cases was comparable with that in amplifying automatically generated test cases.

Evidence from empirical studies suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular, GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI testing fragility, i.e., GUI test classes failing or needing interventions because of modifications in the AUT or in its GUI arrangement and definition. The objective of our study is to examine the diffusion of test classes created with a set of popular GUI Automation Frameworks for Android apps, the amount of changes required to keep test classes up to date, and the amount of code churn in existing test suites, along with the underlying modifications in the AUT that caused such modifications. We defined 12 metrics to characterize the evolution of test classes and test methods, and a taxonomy of 28 possible causes for changes to test code. To perform our experiments, we selected six widely used open-source GUI Automation Frameworks for Android apps. We evaluated the diffusion of the tools by mining the GitHub repositories featuring them, and computed our set of metrics on the projects. Applying the Grounded Theory technique, we then manually analyzed diff files of test classes written with the selected tools, to build from the ground up a taxonomy of causes for modifications of test code. We found that none of the considered GUI automation frameworks achieved a major diffusion among open-source Android projects available on GitHub. For projects featuring tests created with the selected frameworks, we found that test suites had to be modified often – specifically, about 8% of developers’ modified LOCs belonged to test code and that a relevant portion (around 50% on average) of those modifications were induced by modifications in GUI definition and arrangement. Test code written with GUI automation fromeworks proved to need significant interventions during the lifespan of a typical Android open-source project. This can be seen as an obstacle for developers to adopt this kind of test automation. The evaluations and measurements of the maintainance needed by test code wrtitten with GUI automation frameworks, and the taxonomy of modification causes, can serve as a benchmark for developers, and the basis for the formulation of actionable guidelines and the development of automated tools to help mitigating the issue.

The Android Application Programming Interface provides the necessary building blocks for app developers to harness the functionalities of the Android devices, including for interacting with services and accessing hardware. This API thus evolves rapidly to meet new requirements for security, performance and advanced features, creating a race for developers to update apps. Unfortunately, given the extent of the API and the lack of automated alerts on important changes, Android apps are suffered from API-related compatibility issues. These issues can manifest themselves as runtime crashes creating a poor user experience. We propose in this paper an automated approach named CiD for systematically modelling the lifecycle of the Android APIs and analysing app bytecode to flag usages that can lead to potential compatibility issues. We demonstrate the usefulness of CiD by helping developers repair their apps, and we validate that our tool outperforms the state-of-the-art on benchmark apps that take into account several challenges for automatic detection.

Android GUI testing is an important research field to maintain software quality of Android apps. Although many GUI testing schemes have been investigated in the past, the discussion of exploration event scheduling has not been investigated comprehensively for the event replay process. Based on the component priority relationship, this study proposes a novel exploration event scheduling scheme called CPR to reduce the number of the testing events in the replay process. Compared with the breadth-first traversal scheme, the proposed CPR scheme can reduce up to 62% of testing events for achieving the same component coverage, and up to 69% of testing events for layout traversal. Compared with the depth-first traversal scheme, CPR can reduce up to 15% of testing events for achieving the same component coverage, and up to 42% of testing events for layout traversal. With respect to the testing time, CPR can achieve the best performance for most of the AUTs in the empirical study. The results of the empirical experiments show that the proposed CPR scheduling scheme can have the benefits in improving the testing performance.

Systematic exploration of Android apps is an enabler for a variety of app analysis and testing tasks. Performing the exploration while apps run on actual phones is essential for exploring the full range of app capabilities. However, exploring real-world apps on real phones is challenging due to non-determinism, non-standard control flow, scalability and overhead constraints. Relying on end-users to conduct the exploration might not be very effective: we performed a 7-use study on popular Android apps, and found that the combined 7-use coverage was 30.08% of the app screens and 6.46% of the app methods. Prior approaches for automated exploration of Android apps have run apps in an emulator or focused on small apps whose source code was available. To address these problems, we present A3E, an approach and tool that allows substantial Android apps to be explored systematically while running on actual phones, yet without requiring access to the app's source code. The key insight of our approach is to use a static, taint-style, dataflow analysis on the app bytecode in a novel way, to construct a high-level control flow graph that captures legal transitions among activities (app screens). We then use this graph to develop an exploration strategy named Targeted Exploration that permits fast, direct exploration of activities, including activities that would be difficult to reach during normal use. We also developed a strategy named Depth-first Exploration that mimics user actions for exploring activities and their constituents in a slower, but more systematic way. To measure the effectiveness of our techniques, we use two metrics: activity coverage (number of screens explored) and method coverage. Experiments with using our approach on 25 popular Android apps including BBC News, Gas Buddy, Amazon Mobile, YouTube, Shazam Encore, and CNN, show that our exploration techniques achieve 59.39--64.11% activity coverage and 29.53--36.46% method coverage.

Android apps can be effectively tested by randomly generating inputs and triggering corresponding events. Most test generators focus on user-triggered events, such as button clicks. However, the state of an app is not only determined by the interactions with a user, but also inputs from the system and other apps, which are called intents in Android. Intent fuzzing, that is, the automated generation of randomized intents as test inputs, has been demonstrated to be an effective means for identifying crashes in apps. However, the behavior of intent handlers is influenced by the state of the app, which may depend on the user’s interactions with the app that triggers corresponding events. Recent test generators have therefore started integrating some of both types of events, leaving open questions about the best way to combine and balance UI inputs and intents. In this paper, we describe a general framework for integrating user events and intents for testing Android apps. We study empirically how to best combine these two types of events, and evaluate the effectiveness of the combination. Our experiments suggest that combining UI inputs and intents reveals substantially higher code coverage as well as more unique crashes (844 on 500 F-Droid apps) than sending only user-events (762) or only intents (511): The combined approach achieves a magnitude higher activity coverage (78.07%) than using only user inputs (70.45%) and sending only intents (58.23%). Furthermore, 121 unique crashes were found only through the combination of UI inputs and intents. Although intent crashes and UI crashes result from similar exception types, they are distinct, which is relevant when comparing test generators.

This work aims at defining a procedure and a set of mechanisms able to improve the quality of the code coverage in automated software reverse engineering processes, and specifically in automated GUI-driven testing of Android apps. Existing automated model-based testing techniques, based on reverse engineering, generate test cases which can be executed directly on the software's GUI. We propose to augment the code coverage of these techniques, by exploiting information from patterns, defined at different levels (application design, state-based model, interaction with Android services), and generating additional test cases that may increase the coverage capability of GUI-Ripping based testing technique. The generation of the additional test cases is accomplished by defining an automatable procedure which exploits an existing GUI testing approach and a pattern based approach used in a different context.

Android apps are developed using a Software Development Kit (SDK), where the Android application programming interface (API) enables app developers to harness the functionalities of Android devices by interacting with services and hardware. However, API frequently evolves together with its associated SDK. The mismatch between the API level supported by the device where apps are installed and the API level targeted by app developers can induce compatibility issues. These issues can manifest themselves as unexpected behaviors, including runtime crashes, creating a poor user experience. Recent studies investigated API evolution to ensure the reliability of the Android apps, however, they require improvements. This work aims to establish novel methodologies that will improve the state-of-the-art compatibility issue detection and testing approaches.

The purpose of this research is to help the Department of Transportation in supplying emission testing tool which achievable and can used by citizens which mean as portable emission testing tools. This research using experimental method by measuring carbon monoxide (Co), hydrogen (Hc), and nitrogen oxide (Nox) using sensor which connected to smartphone. The result of the research is prototype vehicle emission testing tools L3 category or motorcycle category. The problem is the expensive price to get the vehicle emission testing tools. Then, comparison between emission testing tools with vehicle no comparable each other. There for, it necessary to develop alternative emission testing tools. This prototype vehicle emission testing tools using approach same parameter with existing testing tools. This tool is expected to assist the public in testing the emissions of private vehicles, before being tested by Department of Transportation.

Availability was considered a critical factor of Network Function Virtualization(NFV) platforms. And many existing testing tools with fault injection features for NFV, only aim at availability, while now more and more attention have been paid to the performance degradation when failure happens. Moreover, performance data on hardware/software resources, like CPU consumption, memory usage, are required to collect in performance and fault-tolerant testing. To fulfill the requirements above, this paper proposes a service-based testing framework for NFV platform performance evaluation, under workloads and fault loads. By taking advantage of the Microservice Architecture, the proposed framework will integrate some existing open-source testing frameworks, including performance testing tools, fault injection tools, and monitoring tools, and build a more comprehensive testing scenario. A case study is conducted on Clearwater, a widely used open-source NFV application, to validate the efficiency and usability of the proposed framework.

Background. Energy awareness in the ICT has become an important issue: ICT is both a key player in energy efficiency, and a power drainer. Focusing on software, recent work suggested the existence of a relationship between power consumption, software configuration and usage patterns in computer systems. Aim. The aim of this work was collecting and analysing power consumption data of a general-purpose computer system, simulating common usage scenarios, in order to extract a power consumption profile for each scenario. Methods. We selected a desktop system running Windows XP as a test machine. Meanwhile, we developed 11 usage scenarios, classified by their functionality, and automated by a GUI testing tool. Then, we conducted several test runs of the scenarios, collecting power consumption data by means of a power meter. Results. Our analysis resulted in an estimation of a power consumption value for each scenario and software application used, obtaining that each single scenario introduced an overhead from 2 to 11 Watts, corresponding to an increase of about 12%. Conclusions. We determined that software and its usage patterns impacts consistently on the power consumption of computer systems. Further work will be devoted to evaluate how power consumption is affected by the usage of specific system resources, like processors, disks, memory etc.KeywordsGreen SoftwareEnergy AwareEnergy ProfilingPower Consumption

The debugging for mobile software has been became supported easily and strongly by diversified technology such as performance analysis, tracing, and memory debugging. However, testers who want to find and locate knotty defects between the various components of embedded software feel that just using debugging tools is not enough. Although, using testing tools can achieve the target, but not all the testing tools can be deployed on the real device. So in this paper, we describe a new user-friendly Hook-based testing approach for embedded software. Besides introducing the concept of Hook-based testing, we show how to use AOP to implement components testing on the target easily. Meanwhile, we will provide the testing result by process tracing, and memory checking. Finally we compare with famous debugging tools to prove our approach much better.

Despite the proliferation of Android testing tools, Google Monkey has remained the de facto standard for practitioners. The popularity of Google Monkey is largely due to the fact that it is a black-box testing tool, making it widely applicable to all types of Android apps, regardless of their underlying implementation details. An important drawback of Google Monkey, however, is the fact that it uses the most naive form of test input generation technique, i.e., random testing. In this work, we present Deep GUI, an approach that aims to complement the benefits of black-box testing with a more intelligent form of GUI input generation. Given only screenshots of apps, Deep GUI first employs deep learning to construct a model of valid GUI interactions. It then uses this model to generate effective inputs for an app under test without the need to probe its implementation details. Moreover, since the data collection, training, and inference processes are performed independent of the platform, the model inferred by Deep GUI has application for testing apps in other platforms as well. We implemented a prototype of Deep GUI in a tool called Monkey++ by extending Google Monkey and evaluated it for its ability to crawl Android apps. We found that Monkey++ achieves significant improvements over Google Monkey in cases where an app’s UI is complex, requiring sophisticated inputs. Furthermore, our experimental results demonstrate the model inferred using Deep GUI can be reused for effective GUI input generation across platforms without the need for retraining.

Graphical User Interface (GUI) designing and testing of different applications is significant from a user point of view. GUI testing ensures that the user interface of apps is visually appealing, user-friendly, and enhances user satisfaction. Pattern-Based GUI Testing (PBGT) is a new model-based testing approach that aims to enhance reusability while reducing the workload associated with modeling user interfaces and testing applications. The objective of conducting a Systematic Literature Review (SLR) on pattern-based GUI testing is to consolidate existing knowledge and provide insights for improving the effectiveness of PBGT methods. The objectives of conducting an SLR are the identification of current state-of-the-art approaches and tools in the targeted research context. To accomplish the mentioned objectives, A hybrid methodology combining Kitchenham’s SLR and PRISMA guidelines was used to conduct this SLR. We performed a keyword-based search on well-known databases IEEE, ACM, Science Direct, Wiley, and Springer Link to determine the potential studies published in the last 10 years. Following a comprehensive systematic analysis, we identified a total of twenty-four studies employing varied approaches for pattern-based GUI testing. Additionally, we identified 11 tools that have been used and proposed in PBGT for Android and web apps and made a taxonomy of tools. We also identified challenges and research directions. The implications of this study are significant. The current study is helpful for researchers intending to work in pattern-based GUI testing. The comprehensive knowledge about the current state-of-the-art Tools, techniques, and approaches will help practitioners to effectively apply them in industry and enable researchers to extend their work in this context in future research.