Data loss detector: automatically revealing data loss bugs in Android apps

Most of today's mobile apps rely on the underlying networks to deliver key functions such as web browsing, file synchronization, and social networking. Compared to desktop-based networks, mobile networks are much more dynamic with frequent connectivity disruptions, network type switches, and quality changes, posing unique programming challenges for mobile app developers. As revealed in this paper, many mobile app developers fail to handle these intermittent network conditions in the mobile network programming. Consequently, network programming defects (NPDs) are pervasive in mobile apps, causing bad user experiences such as crashes, data loss, etc. Despite the development of network libraries in the hope of lifting the developers' burden, we observe that many app developers fail to use these libraries properly and still introduce NPDs. In this paper, we study the characteristics of the real-world NPDs in Android apps towards a deep understanding of their impacts, root causes, and code patterns. Driven by the study, we build NChecker, a practical tool to detect NPDs by statically analyzing Android app binaries. NChecker has been applied to hundreds of real Android apps and detected 4180 NPDs from 285 randomly-selected apps with a 94+% accuracy. Our further analysis of these defects reveals the common mistakes of app developers in working with the existing network libraries' abstractions, which provide insights for improving the usability of mobile network libraries.

SummaryThis article presents the techniques and results of a novel model‐based test case generation approach that automatically derives test cases from UML state machines. The main contribution of this article is the fully automated fault‐based test case generation technique together with two empirical case studies derived from industrial use cases. Also, an in‐depth evaluation of different fault‐based test case generation strategies on each of the case studies is given and a comparison with plain random testing is conducted. The test case generation methodology supports a wide range of UML constructs and is grounded on the formal semantics of Back's action systems and the well‐known input–output conformance relation. Mutation operators are employed on the level of the specification to insert faults and generate test cases that will reveal the faults inserted. The effectiveness of this approach is shown and it is discussed how to gain a more expressive test suite by combining cheap but undirected random test case generation with the more expensive but directed mutation‐based technique. Finally, an extensive and critical discussion of the lessons learnt is given as well as a future outlook on the general usefulness and practicability of mutation‐based test case generation. Copyright © 2014 John Wiley & Sons, Ltd.

ABSTRACTCode generation for users' intent has become increasingly prevalent with the large language models (LLMs). To automatically evaluate the effectiveness of these models, multiple execution‐based benchmarks are proposed, including specially crafted tasks, accompanied by some test cases and a ground truth solution. LLMs are regarded as well‐performed in code generation tasks if they can pass the test cases corresponding to most tasks in these benchmarks. However, it is unknown whether the test cases have sufficient test adequacy and whether the test adequacy can affect the evaluation. In this paper, we conducted an empirical study to evaluate the test adequacy of the execution‐based benchmarks and to explore their effects during evaluation for LLMs. Based on the evaluation of the widely used benchmarks, HumanEval, MBPP, and two enhanced benchmarks HumanEval+ and MBPP+, we obtained the following results: (1) All the evaluated benchmarks have high statement coverage (above 99.16%), low branch coverage (74.39%) and low mutation score (87.69%). Especially for the tasks with higher cyclomatic complexities in the HumanEval and MBPP, the mutation score of test cases is lower. (2) No significant correlation exists between test adequacy (statement coverage, branch coverage and mutation score) of benchmarks and evaluating results on LLMs at the individual task level. (3) There is a significant positive correlation between mutation score‐based evaluation and another execution‐based evaluation metric () on LLMs at the individual task level. (4) The existing test case augmentation techniques have limited improvement in the coverage of test cases in the benchmark, while significantly improving the mutation score by approximately 34.60% and also can bring a more rigorous evaluation to LLMs on code generation. (5) The LLM‐based test case generation technique (EvalPlus) performs better than the traditional search‐based technique (Pynguin) in improving the benchmarks' test quality and evaluation ability of code generation.

Preferences, the setting options provided by Android, are an essential part of Android apps. Preferences allow users to change app features and behaviors dynamically, and therefore their impacts need to be considered when testing the apps. Unfortunately, few test cases explicitly specify the assignments of valid values to the preferences, or configurations , under which they should be executed, and few existing mobile testing tools take the impact of preferences into account or provide help to testers in identifying and setting up the configurations for running the tests. This article presents the Prefest approach to effective testing of Android apps with preferences. Given an Android app and a set of test cases for the app, Prefest amplifies the test cases with a small number of configurations to exercise more behaviors and detect more bugs that are related to preferences. In an experimental evaluation conducted on real-world Android apps, amplified test cases produced by Prefest from automatically generated test cases covered significantly more code of the apps and detected seven real bugs, and the tool’s test amplification time was at the same order of magnitude as the running time of the input test cases. Prefest ’s effectiveness and efficiency in amplifying programmer-written test cases was comparable with that in amplifying automatically generated test cases.

A large-scale study on the adoption of anti-debugging and anti-tampering protections in android apps

Android apps often have a “dark mode” option used in low-light situations, for those who find the conventional color palette problematic, or because of personal preferences. Typically developers add a dark mode option for their apps with different backgrounds, text, and sometimes iconic forms. We wanted to understand the actual provision of this dark mode in real-world Android apps through an empirical study of posts from Stack Overflow and real-world Android app analysis. Using these approaches, we identified the aspects of dark mode that developers implemented as well as the key difficulties they experienced in implementing it. We performed a quantitative analysis using open-coding of more than 300 discussion threads to create a taxonomy regarding the aspects discussed by developers with respect to dark mode in Android. Our quantitative analysis of over 6,000 Android apps highlights which dark mode features are typically provided in Android apps and which aspects developers care about during dark mode design. We also examined four app development support tools to see how well they aid Android app development for dark mode. From our analysis, we distilled some key lessons to guide further research and actions in aiding developers with supporting users who require such assistive features. For example, developers should be aware of the potential risks in using unsuitable dark mode design schema and researchers should take dark mode features into consideration when developing app development support tools.

With the increasing demand for power system stability and resilience, effective real-time tracking plays a crucial role in smart grid synchronization.However, most studies have focused on measurement noise, while they seldom think about the problem of measurement data loss in smart power grid synchronization.To solve this problem, a resilient fault-tolerant extended Kalman filter (RFTEKF) is proposed to track voltage amplitude, voltage phase angle and frequency dynamically.First, a threephase unbalanced network's positive sequence fast estimation model is established.Then, the loss phenomenon of measurements occurs randomly, and the randomness of data loss's randomness is defined by discrete interval distribution [0, 1].Subsequently, a resilient fault-tolerant extended Kalman filter based on the real-time estimation framework is designed using the timestamp technique to acquire partial data loss information.Finally, extensive simulation results manifest the proposed RFTEKF can synchronize the smart grid more effectively than the traditional extended Kalman filter (EKF).

Internet of Things (IoT) is the growing invention in the current development of different domains like industries, e-health, and education, etc. Semantic web of things (SWoT) is an extension of IoT that enhance the communication by behaving intelligently. SWoT comprises 7 layered architecture. The perception layer is an important layer for collecting data from devices and to communicate with its associated layer. The data loss at the perception layer is very common due to inadequate resources, unpredictable link, noise, collision, and unexpected damage. To address this problem, we propose a method based on Compressive Sensing which recovers and estimates sensory data from a low-rank structure. The contribution of this paper is three folds. Firstly, we determine the problem of data acquisition and data loss at semantic sensory nodes in SWoT. Secondly, we introduce a compressive sensing based framework for SWoT that recovers the data accurately using low-rank features. Thirdly, the data estimation method is utilized to reduce the volume of the data. Proposed Compressive Sensing based Data Recoverability and Estimation (CS-RE) method is evaluated and compared with the existing reconstruction methods. The simulation results on real sensory datasets depict that the proposed method significantly outperforms existing methods in terms of error ratio and data recoverability accuracy.

The traditional architecture of an urban intelligent transportation data network has a hierarchical distribution. In this architecture, organizations independently manage and upload data, thereby making data sharing difficult to achieve. To solve this problem, a big data platform of urban intelligent transportation was built in this paper by using blockchain technology. In this platform, block data are used as the core, thereby removing the centralized data management of each organization. This method also completely changes the modes of data acquisition, processing, analysis, and storage and fully realizes platform-based large data sharing, decentralization, and distributed computing of the multi-source system of urban intelligent transportation. The big data platform of urban intelligent transportation based on blockchain technology should also solve several key problems, including the unification of data from different sources, the unified supervision and operation of data, and the compatibility with other advanced technologies. To solve these problems, the key technologies for large data platforms were evaluated from a technical perspective. These technologies include the unified technology of each node, the supervision and operation of multi-technology sharing, and the technology compatibility with vehicle networking and vehicle-road collaboration. By taking the problem of traffic flow data loss in road networks under haze as a simulation case scenario, the application of block chain technology was described. By taking some road networks in Beijing as examples, the simulation results show that under the traditional intelligent transportation data network framework, given the limitations of equipment installation layouts and subordinate management organizations, the coverage of road networks for data acquisition is greatly affected by the objective environment, which may lead to data loss. The data acquisition system should also run independently, thereby preventing data sharing at the bottom of the network. The big data platform architecture of urban intelligent transportation based on blockchain technology addresses the limitations of various organizations, realizes data sharing in urban intelligent transportation, and solves the data loss problem under the traditional network architecture.

Automated-test-generation tools generate test cases to enable dynamic analysis of Android apps, such as functional testing. These tools build a GUI model to describe the app states during the app execution, and generate a script that performs actions on UI widgets to form a test case. However, when the test cases are re-executed, the apps under analysis often do not behave consistently. The major reasons for such limited reproducibility are due to (1) backend-service dependencies that cause non-determinism in app behaviors and (2) the severe fragmentation of Android platform (i.e., the alarming number of different Android OS versions in vendor-customized devices). To address these challenges, we design and implement Paladin, a novel system that generates reproducible test cases for Android apps. The key insight of Paladin is to provide a GUI model that leverages the structure of the GUI view tree to identify equivalent app states, since the structure can tolerate the changes on the UI contents for an app behavior performed in different test executions. Based on the model, Paladin can search the view tree to locate the desired UI widgets to trigger events and drive the app exploration to reach the desired app states, making the test cases reproducible. Evaluation results on real apps show that Paladin could reach a much higher reproduction ratio than the state-of-the-art tools when the generated test cases are re-executed across different device configurations. In addition, benefiting from the reproducible capability, Paladin is able to cover more app behaviors compared with the existing tools.

Android apps include third-party native libraries to increase performance and to reuse functionality. Native code is directly executed from apps through the Java Native Interface or the Android Native Development Kit. Android developers add precompiled native libraries to their projects, enabling their use. Unfortunately, developers often struggle or simply neglect to update these libraries in a timely manner. This results in the continuous use of outdated native libraries with unpatched security vulnerabilities years after patches became available. To further understand such phenomena, we study the security updates in native libraries in the most popular 200 free apps on Google Play from Sept. 2013 to May 2020. A core difficulty we face in this study is the identification of libraries and their versions. Developers often rename or modify libraries, making their identification challenging. We create an approach called LibRARIAN (LibRAry veRsion IdentificAtioN) that accurately identifies native libraries and their versions as found in Android apps based on our novel similarity metric bin2sim. LibRARIAN leverages different features extracted from libraries based on their metadata and identifying strings in read-only sections. We discovered 53/200 popular apps (26.5%) with vulnerable versions with known CVEs between Sept. 2013 and May 2020, with 14 of those apps remaining vulnerable. We find that app developers took, on average, 528.71 ±40.20 days to apply security patches, while library developers release a security patch after 54.59 ± 8.12 days---a 10 times slower rate of update.

WebView is a widely used Android component that augments a native app with web browser capabilities. It eases the interactions between an app's native code and web code. However, the interaction mechanism of WebView induces new types of bugs in Android apps. Understanding the characteristics and manifestation of these WebView-induced bugs ( $\omega \text{Bugs}$ for short) facilitates the correct usages of WebViews in Android apps. This motivates us to conduct the first empirical study on $\omega \text{Bugs}$ based on those found in popular open-source Android apps. Our study identified the major root causes and consequences of $\omega \text{Bugs}$ and made interesting observations that can be leveraged for detecting and diagnosing $\omega \text{Bugs}$ . Based on the empirical study, we further propose an automated testing technique $\omega \text{Droid}$ to effectively expose $\omega \text{Bugs}$ in Android apps. In our experiments, $\omega \text{Droid}$ successfully discovered 30 unique and previously-unknown $\omega \text{Bugs}$ when applied to 146 open-source Android apps. We reported the 30 $\omega \text{Bugs}$ to the corresponding app developers. Out of these 30 $\omega \text{Bugs}$ , 14 were confirmed and 7 of them were fixed. This shows that $\omega \text{Droid}$ can effectively detect $\omega \text{Bugs}$ that are of the developers' concern.

Although Android apps are becoming more and more popular, apps tend to contain defects which can ultimately manifest as failures (or crashes) to end-users. In order to improve software quality, people have proposed different automated tools for testing Android applications. The automatic generation of test cases is an important means to improve the efficiency of software testing, but many tools related to test case generation do not seem to be fully adapted to the problem of Android test case generation. Therefore, this article aims to design and evaluate the automatic generation method of Android application test cases based on the multi-dimensional coverage. We conducted test experiments on 48 self-developed Android apps through the Gradle plugin, and the final average target coverage reached about 32%. Considering that the current component modules are not yet complete, we believe that tools will reach even higher coverage in the future.

While automated testing of mobile applications is very useful for checking run-time behaviours and specifications, its capability in discovering issues in apps is often limited in practice due to long testing time. A common practice is to randomly and exhaustively explore the whole app test space, which takes a lot of time and resource to achieve good coverage and reach targeted parts of the apps.In this paper, we present MAMBA1, a directed testing system for checking privacy in Android apps. MAMBA performs path searches of user events in control-flow graphs of callbacks generated from static analysis of app bytecode. Based on the paths found, it builds test cases comprised of user events that can trigger the executions of the apps and quickly direct the apps’ activity transitions from the starting activity towards target activities of interest, revealing potential accesses to privacy-sensitive data in the apps. MAMBA’s backend testing engine then simulates the executions of the apps following the generated test cases to check actual runtime behavior of the apps that may leak users’ private data. We evaluated MAMBA against another automated testing approach that exhaustively searches for target activities in 24 apps, and found that our graph-aided directed testing achieves the same coverage of target activities 6.1 times faster on average, including the time required for bytecode analysis and test case generation. By instrumenting privacy access/leak detectors during testing, we were able to verify from test logs that almost half of target activities accessed user privacy data, and 26.7% of target activities leaked privacy data to the network.

This work aims at defining a procedure and a set of mechanisms able to improve the quality of the code coverage in automated software reverse engineering processes, and specifically in automated GUI-driven testing of Android apps. Existing automated model-based testing techniques, based on reverse engineering, generate test cases which can be executed directly on the software's GUI. We propose to augment the code coverage of these techniques, by exploiting information from patterns, defined at different levels (application design, state-based model, interaction with Android services), and generating additional test cases that may increase the coverage capability of GUI-Ripping based testing technique. The generation of the additional test cases is accomplished by defining an automatable procedure which exploits an existing GUI testing approach and a pattern based approach used in a different context.