How to Specify It!

Avoiding concurrency errors remains one of the main challenges in day-to-day Erlang development. Errors can arise both from unexpected process and message interleaving, as well as concurrent clients or users interacting with the system in an unanticipated manner. To address this issue, many testing tools have been developed over the years. One approach is to pair a property-based testing tool with a tool that overrides the Erlang scheduler. This allows developers to focus on defining the properties of their system, and let the testing tools worry about testing these properties across different interleavings.This paper builds upon this approach and presents the Hansei testing framework. Hansei provides an approach to writing property-based tests that build on standard OTP abstractions, as well as a testing infrastructure designed to support test-driven development of concurrent systems. The Hansei infrastructure builds on the existing QuickCheck property-based testing tool, and supports both built-in message interleaving as well as a testing mode that works with existing process interleaving tools. Existing tools are more mature and feature-complete, while the built-in Hansei support has the unique property of being able to operate on code that spans multiple Erlang VMs.

Property-based testing (PBT), widely used in functional languages and interactive theorem provers, works by randomly generating many inputs to a system under test. While PBT has also seen some use in low-level languages like C, users in this setting must craft all their own generators by hand, rather than letting the tool synthesize most generators automatically from types or logical specifications. For low-level code with complex memory ownership patterns, writing such generators can waste significant amounts of time. CN, a specification and verification framework for C, features a streamlined presentation of separation logic that is specially tuned to present only "easy" logical problems to an underlying constraint solver. Prior work on the Fulminate testing framework has shown that CN's streamlined specifications can also be checked effectively at run time, providing an oracle for testing whether a memory state satisfies a pre- or postcondition. We show that the restricted syntax of CN is also a good basis for deriving generators for random inputs satisfying separation-logic preconditions. We formalize the semantics for a DSL describing these generators, as well as optimizations that reorder when values are generated and propagate arithmetic constraints. Using this DSL, we implement a property-based testing tool, Bennet, that generates and runs random tests for C functions annotated with CN specifications. We evaluate Bennet on a corpus of programs with CN specifications and show that it can efficiently generate bug-revealing inputs for heap-manipulating programs with complex preconditions.

The goal of software testing analysis is to validate that an implementation satisfies its specifications. Many errors in software are caused by generalizable flaws in the source code. Property-based testing assures that a given program is free of specified generic flaws. Property-based testing uses property specifications and a data-flow analysis of the program to guide evaluation of test executions for correctness and completeness.

We examine the behavior of dynamic value-ordering heuristics in a CSP under the requirement to generate a large number of diverse solutions as fast as possible. In particular, we analyze the trade-off between the solution search performance and the diversity of the generated solutions, and propose a general probabilistic approach to control and improve this trade-off. Several old/new learning-reuse heuristics are described, extending the survivors-first value-ordering heuristics family. The proposed approach is illustrated on a real-world set of examples from the Automatic Test Generation problem domain, as well as on several sets of random binary CSPs.KeywordsAutomatic Test GenerationLearning ReuseSolution DiversitySurvivors-FirstValue Ordering CSP Heuristic

We describe a generic approach to image interpretation, based on combining a general method of building flexible template models with Genetic Algorithm (GA) search. The method can be applied to a given image interpretation problem simply by training a Point Distribution Model (PDM), using a set of examples of the image structure to be located. A local optimisation technique, developed for use with PDMs, has been incorporated into the GA search with the aim of improving the speed of convergence and optimality of solution. We present results, from three practical applications, demonstrating that the new method offers significant improvements when compared to previously reported approaches to flexible template matching. The benefits include the ability to deal with different domains of application using a standard method, the ability to deal with complex multi-part models and improved search performance.

An overview of a general approach for mathematical modeling of evolving heterogeneous populations using a wide class of selection systems and replicator equations (RE) is presented. The method allows visualizing evolutionary trajectories of evolving heterogeneous populations over time, while still enabling use of analytical tools of bifurcation theory. The developed theory involves introducing escort systems of auxiliary “keystone" variables, which reduce complex multi-dimensional inhomogeneous models to low dimensional systems of ODEs that in many cases can be investigated analytically. In addition to a comprehensive theoretical framework, a set of examples of the method’s applicability to questions ranging from preventing the tragedy of the commons to cancer therapy is presented.

An approach to machine recognition, which attempts to overcome the problem of specifying suitable features for figures such as hand-printed characters, is suggested. Patterns or features may be considered as a set of regularly occurring elements. During the learning phase the machine discovers these elements (from their regularity) in the input, and from a set of examples of a class, forms a hierarchical description of each class. Patterns found in each category are sought in a new input. The unknown figure is recognized as a member of that class for which the match between the descriptive patterns in that category and input is most similar.

Quality is a key property of modern software systems. In addition, to quality improvement strategies such as CMM or CMMI and manual checking approaches such as reviews or inspections, software testing is the major quality assurance activity in most projects. Literally speaking, software testing is nearly as essential as writing code itself. Due to its prominent role there are quite a number of commercial and open-source test tools available. However, the question remains if an organisation should rely on a single tool or if it should follow a more general strategy using several tools? Practical experience has shown that the latter strategy is more promising by covering multiple quality aspects that cannot be covered by a single tool, although it requires a means of communication between tools (i.e., a tool independent test language). This paper introduces a XML-based and test tool independent test modeling language that support creating general test cases. The language separates test logic from test tool specific code. Thus, a test case only has to be specified once and can then be executed by several test tools.

Software Testing is known as an important part in software development as the product needs to meet the user's expectations and requirements. The testing tool is developed along with the development of automated software testing to act as a medium to test the software, and the existence of these tools eventually cut down the human's effort, energy, and time. Software testing tools are the key assets of a project that can improve productivity and software quality. As there are many tools in the market nowadays, the process of selecting the tool should be explained in detail and specific towards its functions and efficiency. This paper aims to help the authors in making the tool selection from the various versions of testing tools. This study finally chose the most suitable tool according to the research needs.

Software testing provides a means to reduce errors, cut maintenance and overall software costs. Numerous software development and testing methodologies, tools, and techniques have emerged over the last few decades promising to enhance software quality. While it can be argued that there has been some improvement it is apparent that many of the techniques and tools are isolated to a specific lifecycle phase or functional area. This paper presents a set of best practice models and techniques integrated in optimized and quantitatively managed software testing process (OptimalSQM), expanding testing throughout the SDLC. Further, we explained how can Quantitative Defect Management Model be enhanced to be practically useful for determining which activities need to be addressed to improve the degree of early and cost-effective software fault detection with assured confidence is proposed. To enable software designers to achieve a higher quality for their design, a better insight into quality predictions for their design choices, test plans improvement using Simulated Defect Removal Cost Savings model is offered in this paper.

Stochastic testing by fuzzing tools is one of the approaches to software vulnerability analysis. A testing process usually generates random input data for a tested program and takes a significant period of time. Reducing testing time is an important task. One of the areas of research for improving testing is to define only those sets of data sequences, which have an impact on the execution path of the tested program. Thus, a new approach of input data generation that reduces total testing time allows finding more program vulnerabilities. The paper suggests a modification of a genetic algorithm, which is used by fuzzer afl (American Fuzzy Lop). The promising positions model is introduced to improve the efficiency of input data generation. With this model, the most promising position in input data is chosen by the fuzzer genetic algorithm from the viewpoint of vulnerability analysis for next mutation steps. Compared to existing solutions, the suggested model pays attention to the perspective position of a data element to increase code coverage and directs the genetic algorithm to change it. The model was evaluated with the popular fuzzer afl and its modifications (aflfast, symfuzz, afl-rb). During the evaluation study, the suggested model reached 21 % more code coverage than existing solutions. Edge coverage between base program blocks is increased from 20897.3 up to 17267.4. The developed model can be used during software testing, which implies an input and processing of user data. The model can be integrated into stochastic testing tools. The modification should be done only, in the random generator component and does not require redesigning the whole testing tool.

It is imperative to safeguard computer applications and information systems against the growing number of cyber-attacks. Automated software testing can be a promising solution to quickly analyze many lines of code and detect vulnerabilities and possible attack vectors by generating function-specific testing data. This process draws similarities to the constrained adversarial examples generated by adversarial learning methods, so there could be significant benefits to the integration of these methods in testing tools. Therefore, this literature review is focused on the current state-of-the-art of constrained data generation methods applied for adversarial learning and software testing, aiming to guide researchers and developers to enhance software testing tools with adversarial testing methods and improve the resilience and robustness of their information systems. The found constrained data generation applications were systematized, and the advantages and limitations of approaches specific for white-box, grey-box, and black-box testing were analyzed, identifying research gaps and opportunities to improve automated testing tools with data generated by adversarial attacks.

Software testing plays a key role in Software Development. There are two approaches to software testing i.e. Manual Testing and Automated Testing which are used to detect the faults. There are numbers of automated software testing tools with different purposes, but it is always a problem to select a software testing tool according to the needs. Web service is broadly utilised idea these days due to quick promotion of Web services and less literature is accessible with respect to web administration's performance. Web applications are hard to test in contrast with customary applications particularly as far as Performance testing, for example, unpredictable load, response time and so on. In this investigation correlation of performances testing tools i.e. Apache JMeter, NeoLoad, and Loadster is done based on various parameters. Performance parameters results generated by these performance testing tools have been evaluated and assessed in terms of usability test parameters & performance parameters.

One of the main concerns in the software industry continues to be the development of high quality software. This concern will be exacerbated as software systems become more complex. The training of software developers continues to grow in academia since more institutions are offering software engineering (SE) courses. However, the list of topics that are expected to be covered in this course leaves little or no time for topics that focus on developing quality software, such as software testing and the use of testing tools.In this paper we describe an approach that non-intrusively integrates the use of software testing tools in SE courses. The cornerstone of our approach is the interaction students have with a Web-Based Repository of Software Testing Tools (WReSTT) that contains tutorials on testing concepts and testing tools. WReSTT employs both collaborative learning and social networking features that are attractive to students. We present the results of preliminary study performed in two SE courses that show how using the resources in WReSTT can potentially impact the students' understanding of software testing and the use of testing tools.

Application of Property-based Testing Tools for Metamorphic Testing