Managing Automated End to End Testing in Bitwise Heterogenous Environments

Use of mobile applications are trending these days due to adoption of handheld mobile devices with operating systems such as Android, iOS and Windows. Delivering quality mobile apps is as important as in any other web or desktop application. Simplification and ease of quality assurance or evaluation in mobile devices is achieved by using automated testing tools. These tools have been evaluated for their features, platforms, code coverage, and efficiency. However, they have not been evaluated and compared to each other for different quality attributes they can enhance in the apps under test. This research study aims to evaluate different testing tools focusing on identifying quality factors they aid to achieve in the apps under test. Furthermore, it aims to measure overall trends of essential quality factors achieved using automated testing tools. The findings of this study are beneficial to the practitioners and researchers. The practitioners need to look up for specific tools which aid them to assure the desired quality factors in the apps under test. The researchers may base their studies on the findings of this study to propose solutions or revise existing tools in order to achieve maximum number of critical quality attributes in the app under test. This study revealed that the trend of automated testing is high on usability, correctness and robustness. Moreover, the trend is average on testability and performance. However, for assurance of extensibility, maintainability, scalability, and platform compatibility, only a few tools are available.

This paper studies the software test with automated test tool under cloud environment. The automated software test technology could speed up the test efficiency and less down the work of test engineers. On the cloud technology, the automated test tool should make some modification to adapt for cloud. The paper proposes the test integrated technology with automated test tool on cloud-based environment. This technology connects cloud-based environment and the automated test tool. The fully test procedure also are proposed to support test design, script develop and script execute. The cloud technology is used to test some applications with the automated test tool and archive the expected outcome.

Cloud computing systems routinely process users' confidential data, but the underlying virtualization software in use today is not constructed to minimize the exposure of such data. For instance, virtual machine (VM) checkpointing can drastically prolong the lifetime and vulnerability of confidential data without users' knowledge by storing such data as part of a persistent snapshot. A key requirement for minimizing the exposure of any data is the ability to cleanly isolate such data for either exclusion or processing. Traditional mechanisms for memory taint tracking are expensive whereas those for isolating application footprint in VM-based sandboxes are not transparent. In this paper, we propose a transparent and lightweight mechanism for isolating a confidential application's memory footprint in a VM. The key idea is for a parent VM to spawn a child VM, called a Privacy-preserving Virtual Machine (PPVM) within which the confidential application executes. Hypervisor features, such as VM checkpointing, that need to exclude the memory of a confidential application can safely ignore the child VM's memory footprint. Alternatively, features such as checkpoint encryption or malware tracking can operate only on the child VM's memory. We implement memory isolation for PPVM through a lightweight VM fork operation that uses copy-on-write to reduce the memory and filesystem overhead of the PPVM. Transparency is achieved through a confidential shell that allows the parent VM to spawn the confidential application in the PPVM and exercise control over it during runtime. We demonstrate the effectiveness of PPVM through its use with VM checkpointing, which can safely checkpoint the parent VM while excluding or encrypting the associated PPVM. We show that our PPVM implementation achieves effective memory isolation with low overheads on memory, CPU, and network performance.

Software testing is an integral part of software development lifecycle. Lack of testing can often lead to disastrous consequences including lost of data, fortunes, and even lives. Despite its importance, current software testing practice lacks automation, and is still primarily based on highly manual processes from the generation of test cases up to the actual execution of the test. Although the emergence of helpful automated testing tools in the market is blooming, their adoptions are lacking as they do not adequately provide the right level abstraction and automation required by test engineers. JTst is a Java based automated unit testing tool that addresses some of the aforementioned issues. The main novel features are the fact that JTst automates the test preparation activities, facilitates the test data generation through recombination, and allows concurrent execution of test data, in order to encourage higher product quality at lower testing costs.

This paper describes the third round of the Java Unit Testing Tool Competition. This edition of the contest evaluates no less than seven automated testing tools! And, like during the second round, test suites written by human testers are also used for comparison. This paper contains the full results of the evaluation.

Dynamic analysis is a technique that is used to fully understand the internals of a system at runtime. On Android, dynamic security analysis involves real-time assessment and active adaptation of an app’s behaviour, and is used for various tasks, including network monitoring, system-call tracing, and taint analysis. The research on dynamic analysis has made significant progress in the past years. However, to the best of our knowledge, there is a lack in secondary studies that analyse the novel ideas and common limitations of current security research. The main aim of this work is to understand dynamic security analysis research on Android to present the current state of knowledge, highlight research gaps, and provide insights into the existing body of work in a structured and systematic manner. We conduct a systematic literature review (SLR) on dynamic security analysis for Android. The systematic review establishes a taxonomy, defines a classification scheme, and explores the impact of advanced Android app testing tools on security solutions in software engineering and security research. The study’s key findings centre on tool usage, research objectives, constraints, and trends. Instrumentation and network monitoring tools play a crucial role, with research goals focused on app security, privacy, malware detection, and software testing automation. Identified limitations include code coverage constraints, security-related analysis obstacles, app selection adequacy, and non-deterministic behaviour. Our study results deepen the understanding of dynamic analysis in Android security research by an in-depth review of 43 publications. The study highlights recurring limitations with automated testing tools and concerns about detecting or obstructing dynamic analysis.

Considerable amount of cloud adoption is proven in the area of software testing. There are several providers in the area of hosted testing tools and services like IBM and HP. The functional testing tools cover a variety of aspects in functionality testing for Web applications, Web services and mobile applications. Another set of tools in relation to cloud environments is in the area of performance testing from cloud providers such as BlazeMeter, SOASTA, LoadStorm and Keynote. This chapter discusses various options available for leveraging cloud for performance testing of software applications. Additionally, the chapter covers some of the key considerations when selecting the solution providers. The chapter also presents a conceptual reference framework for leveraging the readily available public cloud infrastructures for optimized cost and high-volume load simulations from different regions for efficient testing. Such a custom framework can help in making the enterprise system services more market ready, which in turn aids in improving the overall quality of the enterprise systems. Proposed reference framework can be used in performance testing, regression testing, benchmarking and product certification of any on-premise or cloud-deployed services. This helps in reducing the overall testing cost and test cycle duration, thereby achieving accurate capacity planning.

This paper introduces a testing tool for regular expressions (regexes) named AREXT. AREXT can automatically extract regexes from C++ source files and visually represent them as DFA graphs. Given a regex, AREXT can generate a set of positive and negative strings with 100% coverage of nodes, edges, and edge pairs. We leverage prior works of synthesizing regexes from natural language to create benchmarks for evaluating AREXT. Some current tools, i.e., EGRET and MUTREX, are also being evaluated and compared. Experiments show that AREXT can outperform EGRET as AREXT can detect more unexpected synthesized regexes in almost all benchmarks. The evaluation results indicate that strings with 100% coverage metrics (generated by AREXT) or strings with maximum mutation score (generated by MUTREX) are not enough to ensure the correctness of regexes under test. Experiments also show that combining AREXT, EGRET, and MUTREX can detect a majority of unwanted synthesized regexes (87–91%).

Quality is a key property of modern software systems. In addition, to quality improvement strategies such as CMM or CMMI and manual checking approaches such as reviews or inspections, software testing is the major quality assurance activity in most projects. Literally speaking, software testing is nearly as essential as writing code itself. Due to its prominent role there are quite a number of commercial and open-source test tools available. However, the question remains if an organisation should rely on a single tool or if it should follow a more general strategy using several tools? Practical experience has shown that the latter strategy is more promising by covering multiple quality aspects that cannot be covered by a single tool, although it requires a means of communication between tools (i.e., a tool independent test language). This paper introduces a XML-based and test tool independent test modeling language that support creating general test cases. The language separates test logic from test tool specific code. Thus, a test case only has to be specified once and can then be executed by several test tools.

Performance analysis of software reliability growth models with testing-effort and change-point

We discuss the potential for automatically prioritizing accessibility issues found by automated test tools in an application, based on 'accessibility impact' to the end user. We examine three influential factors in assessing the severity of an issue: criticality of the affected component, impact of the type of issue, and mitigating effect of the application context. We describe how these factors could be quantified and combined to yield severity measures consistent with those of a human expert, illustrating with a set of accessibility issues found in a real online web task.

The authors propose a software testing method for the detection of modification errors caused by functional specification changes. In the method, if inappropriate transformations are selected, the following problems may occur: (1) test cases cannot be obtained, (2) efficiency becomes worse, and (3) effectiveness declines.This paper identifies the transformations that the method can most effectively apply, employing Hatley's model as a software functional specification description model. It introduces the concept “change level” based on the analysis of actual software defects and shows that: (1) the transformations used in the change level are optimal for testing, and (2) when there exist two transformations in the change level whose restrictions are complementary, only one must be tested. The effectiveness of the proposed method is demonstrated by applying the method to an actual software product. The proposed testing method detected 90 percent of the defects due to functional specification changes in a software product. It can readily produce test cases even without automated testing tools when the restrictions consist of simple inequalities or equalities of the input parameters.

The use of an automated testing tool, the Circuit Switch Automated Test System (CSATS), is described. Use of this tool which has proven effective as a step in improving the quality of circuit switching systems. Benefits of automated testing, the CSATS concept and implementation, and example CSATS test scripts and their applications are summarized. The use of CSATS software in switching system analysis is illustrated, with emphasis on call processing testing. Associated capabilities and techniques are discussed. It is concluded that CSATS has proven effective in identifying requirements violations, minimizing analysis team size, shortening analysis intervals, maintaining test results for auditing, and consistently repeating tests to confirm fault removal. >

"Within the automotive product development cycle virtual and heterogeneous testing is becoming increasingly established through component, module and vehicle-level simulation. Though a number of standards in this field have been established, models are still mostly created in a fragmented manner: using domain-specific tools to create, manage and execute simulations without standardization of the content of the functional interfaces (FMI does only standardize the format) and limited scalability. This fragmentation leads to a lot of redundant effort as models of the same component or system are re-created several times. HIFI-ELEMENTS project addressed this fragmentation through two main mechanisms: Firstly, developing, validating and publishing a recommendation for standardization of model interfaces for common e-drive components (e-machine, inverter, battery, DC/DC converter, thermal management) and implementation of compliant versions of existing models. Secondly, implementing a seamless workflow linking extended versions of existing tools with effort-saving automated methods for model parameterization and test case generation. This seamless integration will substantially increase the number of integrations and test cases that can be early validated through simulation, leading to optimized efficiency designs and development effort reduction. The standardization also guarantees scalability among fidelity levels, from concept design to XiL through detailed modelling. In this paper we present the results of the Use Case C: Component co-optimization. The purpose of use cases is the demonstration of the advantages of the standardized models and workflow industry relevant scenarios. The work content performed in the use case is very extensive and multidisciplinary. In the first step, the high fidelity models from the expert components developers were validated independently with automated testing tools and later integrated to create a complete vehicle architecture integration. The standardization permitted to seamlessly test several component variants developed within the project for the same architecture, including tens of motor models with different technologies, inverters and high voltage converters with different IGBT technology and various battery packs. This possibility was exploited through co-optimization with multi objective Genetic Algorithm, permitting to select the optimal component combination, powertrain architecture (with and without high voltage DCDC converter) and components parametrization considering the trade-off of consumption and performances. The optimized and baseline variants were used to demonstrate the scalability of the models to different simulation objectives. The model was co-simulated with a traffic simulation environment in order to evaluate the impact of eco-driving recommendation algorithms in a realistic driving situation. The optimized solution was also validated against a wide database of driving conditions including real driving cycles, performance and vehicle dynamics. Finally, the integrated models were seamlessly transferred to real time simulation platform for Model-in-the-Loop testing with a simulated 3D environment aimed at ADAS testing. Real-time capability demonstrates that next steps such as Driver-in-the-Loop and Hardware-in-the loop can be achieved smoothly. The extensive simulation activities performed in this use case demonstrate the benefits of the standard in models exchangeability and effort reduction in model based development. This project received funding from the European Union’s (EU) Horizon 2020 Research and innovation program under grant agreement N 769935."

The accessibility of State universities and colleges (SUCs) websites is crucial for ensuring inclusivity and equal access to educational resources for all individuals, regardless of their abilities. This study evaluated the accessibility of SUCs in the Philippines websites using automated testing tools like WAVE and TAW, which analyzed web pages based on predefined accessibility criteria such as the Web Content Accessibility Guidelines (WCAG) 2.0. The evaluation process involved scanning and analyzing the websites to identify potential accessibility barriers or issues. The results of the accessibility evaluation reveal significant challenges in meeting WCAG guidelines across various aspects, such as Perceivable and Operable guidelines. Common issues include missing alternative text for images, unclear link descriptions, color contrast combinations, and inadequate form labeling. These findings emphasize the need for SUCs to prioritize web accessibility to ensure equal access to educational resources. Furthermore, it includes ranking SUCs' websites based on their accessibility scores, highlighting institutions that have demonstrated a strong commitment to accessibility, and identifying areas for improvement. MPSPC leads with a minimal mean error rate of 0.33, setting a benchmark for other institutions. PRMSU and GSU follow closely, demonstrating a commitment to enhancing web accessibility. The findings highlight the importance of adhering to accessibility guidelines to create an inclusive online learning environment for all users.