A Comparative Analysis of Quality Assurance of Mobile Applications using Automated Testing Tools

Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical evidence has been shown to clarify whether and how software testability is influenced by code visibility. We have performed an empirical study to shed light on this problem. Our study focuses on test code coverage, in particular that of automatic testing tools. Code coverage is commonly used for various purposes, such as evaluating test adequacy, assessing test quality, and analyzing testability. Our study uses code coverage as the concrete measurement of testability. By analyzing code coverage of two state-of-the-art tools, in comparison with that of developer-written tests, we have discovered that code visibility does not necessarily have effects on its code coverage, but significantly affects automatic testing tools. Low code visibility often leads to low code coverage for automatic tools. In addition, different treatments on code visibility can result in significant differences in overall code coverage for automatic tools. Using a tool enhancement specific to code visibility, we demonstrate the great potential to improve existing tools.

This paper studies the software test with automated test tool under cloud environment. The automated software test technology could speed up the test efficiency and less down the work of test engineers. On the cloud technology, the automated test tool should make some modification to adapt for cloud. The paper proposes the test integrated technology with automated test tool on cloud-based environment. This technology connects cloud-based environment and the automated test tool. The fully test procedure also are proposed to support test design, script develop and script execute. The cloud technology is used to test some applications with the automated test tool and archive the expected outcome.

Declarative layout constraints for testing web applications

The paper focuses on bugs in web applications that can be detected by analyzing the contents and layout of page elements inside a browser's window. Based on an empirical analysis of 35 real-world web sites and applications (such as Facebook, Dropbox, and Moodle), it provides a survey and classification of more than 90 instances of layout-based bugs. It then introduces Cornipickle, an automated testing tool that provides a declarative language to express desirable properties of a web application as a set of human-readable assertions on the page's HTML and CSS data. Such properties can be verified on-the-fly as a user interacts with an application.

Dynamic analysis is a technique that is used to fully understand the internals of a system at runtime. On Android, dynamic security analysis involves real-time assessment and active adaptation of an app’s behaviour, and is used for various tasks, including network monitoring, system-call tracing, and taint analysis. The research on dynamic analysis has made significant progress in the past years. However, to the best of our knowledge, there is a lack in secondary studies that analyse the novel ideas and common limitations of current security research. The main aim of this work is to understand dynamic security analysis research on Android to present the current state of knowledge, highlight research gaps, and provide insights into the existing body of work in a structured and systematic manner. We conduct a systematic literature review (SLR) on dynamic security analysis for Android. The systematic review establishes a taxonomy, defines a classification scheme, and explores the impact of advanced Android app testing tools on security solutions in software engineering and security research. The study’s key findings centre on tool usage, research objectives, constraints, and trends. Instrumentation and network monitoring tools play a crucial role, with research goals focused on app security, privacy, malware detection, and software testing automation. Identified limitations include code coverage constraints, security-related analysis obstacles, app selection adequacy, and non-deterministic behaviour. Our study results deepen the understanding of dynamic analysis in Android security research by an in-depth review of 43 publications. The study highlights recurring limitations with automated testing tools and concerns about detecting or obstructing dynamic analysis.

Software testing is an integral part of software development lifecycle. Lack of testing can often lead to disastrous consequences including lost of data, fortunes, and even lives. Despite its importance, current software testing practice lacks automation, and is still primarily based on highly manual processes from the generation of test cases up to the actual execution of the test. Although the emergence of helpful automated testing tools in the market is blooming, their adoptions are lacking as they do not adequately provide the right level abstraction and automation required by test engineers. JTst is a Java based automated unit testing tool that addresses some of the aforementioned issues. The main novel features are the fact that JTst automates the test preparation activities, facilitates the test data generation through recombination, and allows concurrent execution of test data, in order to encourage higher product quality at lower testing costs.

This paper describes the third round of the Java Unit Testing Tool Competition. This edition of the contest evaluates no less than seven automated testing tools! And, like during the second round, test suites written by human testers are also used for comparison. This paper contains the full results of the evaluation.

Due to incremental upgrade approach, growing number of enterprise systems present heterogeneous UIs, in which certain sections/applications are 32 bit whereas others are 64 bit. Typical business process use-case interactions in these enterprise systems span both these sections, simultaneously. However conventional test automation tools have been designed to test either 32 bit applications or 64 bit applications only, but not both simultaneously. The test tools cannot directly detect custom controls or third party components and hence rely on "hooking into these UI controls" to get access. These hooking components need to be managed for interoperability. Hooking allows test code to monitor messages received by the application under test. However Windows OS puts the following constraints: 1. If an application requires the use of hooks in other processes, it is required that a 32-bit application can inject a 32-bit DLL into 32-bit processes, and a 64-bit application can inject a 64-bit DLL into 64-bit processes. 2. The 32-bit and 64-bit DLLs must have different names. The first constraint impedes seamless interoperation of the 32 bit and 64 bit accessing sections of the test script. The second constraint implies separate management of the same injection code increasing maintenance effort. In this paper we report our experience of co-ordination between the hooking components as a mechanism to interact with these heterogeneous user interfaces. Our bit sensitive manager component, co-ordinates the 64-bit test environment, allowing 32-bit controls to be accessed seamlessly without maintaining separately named injection components. This facilitates efficient interaction even if the communication between the 32 and 64 bit sections are intense. Importantly, the mechanism does not cause instability in the application under test nor does it increase the application's memory footprint significantly. This mechanism is envisioned to drive the "simultaneous 32bit - 64 bit handling" capability in the commercial test automation tools and will be used widely as enterprise systems take an incremental upgrade approach to migrate towards fully 64 bit systems.

Gadgets have become indispensable attributes of our lives, but also applications. Using applications simplifies and speeds up users. It is the future computer science teachers who must be able to create their own applications for professional duties, because the possession of tools for creating software products is one of the software learning outcomes of future computer science teachers. The concept of "application" is analyzed in the article. An "application" is defined as a software product that can run on a PC and / or mobile device due to installation or opening by a browser without installation. Types of applications are presented: desktop applications, mobile, web applications. The existing classifications of applications are given, with their characteristics, namely: web application, rich Internet application, Internet application, mobile application, hybrid mobile application, composite web application (mashup). Describes the classification of Internet applications and composite web applications. Examples of web applications are given. Statistics on the use of mobile applications of social networks and messengers, according to which the most popular application is the YouTube application. The author's classification of applications is offered: desktop applications (applications that are installed on a PC), mobile applications (applications that are installed on mobile devices), Internet applications (web applications, applications that do not require installation, but are opened by a browser and a certain mostly use the Internet). The platforms for creating applications are given. The stages of application creation are characterized: selection of the target audience, definition of purpose and functionality, type of application interface, selection of tools for application creation, application creation, application testing, improvement of the developed application. Examples of applications that can be created by future computer science teachers for further use in professional activities are offered: own website, testing system, reference book, application with presentations to educational material and tasks, guest book, forum, etc. The creation and use of applications by future computer science teachers ensures the formation of pupils' interest in computer science lessons, because pupils often use applications for personal needs, rather than educational.

The agile approach grew dramatically over traditional approaches. The methodology focuses more on rapid development, quick evaluation, quantifiable progress and continuous delivery satisfying the customer desire. In view of this, there is a need for measurement of the agile development process. In this respect, the present research work investigates the inter-relationships and inter-dependencies between the identified quality factors (QF), thereby outlining which of these QF have high driving power and dependence power, working indirectly towards the success of agile development process. This paper proposes a new agile quality model, utilizing an interpretive structural modeling (ISM) approach and the identified factors are classifies using Matriced' Impacts Croise's Multiplication Applique´e a UN Classement (MICMAC) approach. The research findings can significantly impact agile development process by understanding how these QF related to each other and how they can be adopted.

In software testing process, test data generation represents an important step for high quality software, even for mobile devices. As proposed in previous works, a potential source for random data generation is represented by the UI layout files that are used for almost all mobile platforms (Android, iOS, Windows Phone/Mobile). This paper continues the previous work and presents a test data generation system based on Android layout files. The test data generator uses DSL files as input and generates test data that conform to several testing principles. The generated test data could be stored in XML files or any format required by the testing frameworks.

The primary objective of our project is to construct a fully functional voice based Home automation system that uses Internet of Things, Artificial Intelligence and Natural Language Processing (NLP) to provide a cost-effective, efficient way to work together with home appliances. There are many smart home solutions in the market that aim to automate the basic operations of these home appliances using various technologies such as GSM (Global System for Mobile), NFC (Near-Field Communication) etc. However, most of these systems focus on mimicking the basic operation of the electrical switch. Our project aims at providing a fully automated voice based solution that our users can rely on, to perform more than just switching on/off the appliances. The user sends a command through speech to the mobile device, which interprets the message and sends the appropriate command to the specific appliance. We plan on implementing four basic home appliances as a “Proof-of-Concept” for this project which includes Fan, Light, Coffee Machine and Door Alarms. The voice command given by the user is interpreted by the mobile device using Natural Language processing. The mobile device acts as a central console; it determines what operation must be completed by which appliance to fulfill the user's request. The central console might likewise be either a desktop application, web application or a smart phone application as nearly all of the data transferred can be processed by the cloud. However, for the convenience of the user and increased mobile capabilities we will be using a smart phone in this project. The appliances are associated with the mobile device through an Arduino Board that establishes the concept of Internet of Things. The Arduino Boards are interfaced with the appliances and programmed in a manner that they respond to mobile inputs.

The popularity of data warehouses for data analysis has grown tremendously. Data warehouse systems have emerged as the core of management information systems. Data warehouse is part of a larger infrastructure that includes legacy data sources, external data sources, data acquisition software, a repository, analytical tools, and user interface. The difficulties of data warehouse implementations have been widely cited in the literature but research on the factors for initial and ongoing data warehouse implementation success is rare and fragmented. Through a comprehensive review of the literature, 10 factors were found to be critical to DW implementation success — Organizational, Technical, Project, Environmental, Infrastructure, Information Quality, System Quality, Service Quality, Relationship Quality, and Net Benefits. A proposed research model is developed in this paper to determine the impact of quality and success factors on the implementation of data warehouse by adapting the updated (2003) DeLone and McLean Information System Success Model, new dimensions are proposed to the model.

Researchers have proposed automated testing tools to minimise the effort and resources spent on testing GUIs. A relatively simple strategy employed by the proposed tools thus far is the observe-select-execute approach, where all of a GUI’s actions on its current state are observed, one action is selected and the selected action is executed on the software. The strategy’s key function is to select an action that may achieve new and desirable GUI states. Due to difficulties in comparing actions, most existing test generators ignore this step and randomly select an action. However, a randomly selected action has limitations. It does not test most parts of a GUI within a reasonable amount of time and there is a high probability that the same actions are re-selected. This reduces code coverage, thereby resulting in undetected failures. To overcome this limitation, the Q-Learning algorithm was proposed by several researchers to minimise randomness. The idea was to change the probability distribution over the sequence space. Instead of making purely random selections, the least frequently executed action is selected so that the GUI can be further explored. Q-Learning showed better results than the random exploration strategy but it also presented a weakness. Q-Learning’s reward function assigns the highest value to the least frequently executed action without taking into consideration its potential ability in detecting failures. Furthermore, the proposed techniques based on the Q-Learning algorithm do not consider context-based actions. Thus, these techniques are unable to detect failures that occur due to the improper use of context data, which is becoming an increasingly common issue in mobile applications nowadays. We propose a tool, namely the Crash Droid, that allows the automation of testing context-aware Android applications. We utilise the Q-Learning algorithm to compare actions, including context-based actions, to effectively detect crashes and achieve a higher code coverage.

This paper introduces a testing tool for regular expressions (regexes) named AREXT. AREXT can automatically extract regexes from C++ source files and visually represent them as DFA graphs. Given a regex, AREXT can generate a set of positive and negative strings with 100% coverage of nodes, edges, and edge pairs. We leverage prior works of synthesizing regexes from natural language to create benchmarks for evaluating AREXT. Some current tools, i.e., EGRET and MUTREX, are also being evaluated and compared. Experiments show that AREXT can outperform EGRET as AREXT can detect more unexpected synthesized regexes in almost all benchmarks. The evaluation results indicate that strings with 100% coverage metrics (generated by AREXT) or strings with maximum mutation score (generated by MUTREX) are not enough to ensure the correctness of regexes under test. Experiments also show that combining AREXT, EGRET, and MUTREX can detect a majority of unwanted synthesized regexes (87–91%).