Selective Feature-Sensitive Coverage for Conformance Testing of Programming Languages

This paper presents the conformance and interoperability testing of tunnel protocols, which are widely used in the IPv4 to IPv6 transition. In the conformance testing, we use the traverse test method to test the tunnel implementation in the router and generate its test suite with good test coverage. An effective solution is proposed to realize the automatic configuration of the SUT (system under test) during the test process. And it also improves the readability of test cases. Besides, we report the test result of Linux. In the interoperability test, based on operational semantics of concurrent TTCN, we propose a test architecture to apply concurrent test process to test multiple implementations.

Function Block Diagram (FBD), one of the PLC programming languages, is a graphical modeling language which has been increasingly used to implement safety-critical software such as nuclear reactor protection software. With increased importance of structural testing for FBD models, FBD model-based test coverage criteria have been introduced. In this paper, we empirically evaluate the fault detection effectiveness of the FBD coverage criteria using mutation analysis. We produce 1800 test suites satisfying the FBD criteria and generate more than 600 mutants automatically for the target industrial FBD models. Then we evaluate mutant detection of the test suites to assess the fault detection effectiveness of the coverage criteria. Based on the experimental results, we analyze strengths and weaknesses of the FBD coverage criteria, and suggest possible improvements for the test coverage criteria.KeywordsFunction block diagrammutation analysistest coverage criteria

This paper deals with the testing of distributed systems. An implementation under test is checked for conformance with the properties defined by a reference specification. Since distributed systems usually have multiple interfaces, the reference specification will not define the order of all pairs of interactions taking place at different interfaces. Therefore a specification formalism supporting the definition of partial orders is required. Different such formalisms are compared in this paper, including MSC-Charts (or Interaction Overview Diagrams). A variation of this formalism, called Partial-Order-Charts (PO-Charts) is proposed which makes abstraction from the exchange of messages. It concentrates on the specification of partial orders between local actions in different system components. It is shown that the partial-order testing approach introduced for a single partial order specification can be adapted to testing PO-Charts which define various combinations of different partial orders which are sequenced by strict or week sequencing, including loops. Various examples are given to compare this testing approach with state machine testing methods which can be applied for bounded PO-Charts for which one can derive an equivalent state machine. The testing complexities and fault model assumptions of these two approaches are compared.

The aim of this study is to infer the performance assessment of sewage treatment plants (STPs) using the compliance index (CI). The methodology includes three steps, (1) estimation of performance indicators from the quality test, parameter test, and conformity test, (2) hypothesis testing of non-conformity, and (3) hypothesis testing using the non-parametric test. The CI integrates the results of the above nine tests through a scoring system based on individual performance. The study area, Navi Mumbai Municipal Corporation (NMMC), India, treats its domestic sewage through seven centralized STPs located in different zones. The influent and effluent parameters, such as flow rate, biochemical oxygen demand (BOD), chemical oxygen demand (COD), total suspended solids (TSS), pH, dissolved oxygen (DO), and temperature, were studied in this work. The removal efficiency (RE) analysis shows the efficiency of BOD, COD, and TSS varies between 96.67 and 97.30%, 84.55 and 89.70%, and 94.04 and 95.81%, respectively, and indicates that all STPs are performing well. The proposed CI infers that all STPs perform well and comply with NMMC standards, except one STP, namely Airoli. Interestingly, RE analysis shows that Airoli STP's BOD is the third-best compared to all other STPs. This study shows that the CI method can infer the performance of STPs better than RE and help the administrators to plan, execute, operate, and maintain STPs to achieve total sanitation in megacities.

Software transactions have received significant attention as a way to simplify shared-memory concurrent programming, but insufficient focus has been given to the precise meaning of software transactions or their interaction with other language features. This work begins to rectify that situation by presenting a family of formal languages that model a wide variety of behaviors for software transactions. These languages abstract away implementation details of transactional memory, providing high-level definitions suitable for programming languages. We use small-step semantics in order to represent explicitly the interleaved execution of threads that is necessary to investigate pertinent issues.

Modular language implementation in Rascal – experience report

Conformance testing is an operational way of determining whether an implementation conforms to the specification or not. It has a rich underlying theory wherein the specification and the implemen- tation under test (IUT) are each modeled by a timed automaton with inputs and outputs (TAIO), a variant of the classical timed automaton [1]. Test cases generated from the specification TAIO are symbolically executed against the implementation TAIO. Depending upon how test cases interact with the IUT, testing can be synchronous or asynchronous. In synchronous testing a test case interacts with the IUT directly, whereas in asynchronous testing a test case interacts with the IUT through a pair of first-in-first-out (FIFO) channels. Different approaches for synchronous testing of real-time systems have already been proposed [5],[7],[4],[8]. In this paper we propose an approach which is aimed at testing real-time systems asynchronously (i.e., remotely through some medium)

An approach for conformance testing of protocols specified as a collection of communicating finite state machines (FSMs) with two parts, pruning and a guided random walk procedure, is presented. First the protocol is pruned to various sets of machines; each set provides only one service. This significantly reduces the test sequence length. Then a guided random walk procedure that attempts to cover all transitions in the component FSMs is used. The results of applying the procedure to the full-duplex alternating bit protocol and the asynchronous transfer mode (ATM) adaptation layer convergence protocol are presented. For the ATM adaptation layer, 99% of component FSMs' edges can be covered in a test with 11692 input steps. Previous approaches cannot generate conformance tests for standard protocols (such as ATM adaptation layer) specified as a collection of communicating FSMs. >

Test coverage is an important step in software testing and a good indicator of software quality. It helps in evaluating the effectiveness of testing by providing data on different coverage items. It ensures that testing is effectively carried out without missing out all the key functional areas or the features of the software under development. We have analyzed several approaches to understand the existing test coverage framework. These approaches have been classified in different categories namely instrumentation based (source code and byte code), information retrieval (latent semantic indexing), genetic algorithm and UML based. Moreover, we have compared these approaches based on some criteria such as coverage types, data base support, software traceability support, regression testing support, coverage construction and coverage re-construction techniques. We are of the opinion, although much research effort has been put on how to get coverage information by either code based testing or requirement based testing, not much has been paid to measure the coverage by combining both to get better coverage.

FOZCIL (Framework for Object Z Conversion to Implementation Language) captures and implements language-independent features as the fixed part of a framework, and language-dependent properties as the extensible part of a framework. Once the framework has accepted the language-dependent features of a target OO language, it generates a FOZCIL tool instance. The tool instance is capable of accepting Object-Z specifications and of converting those specifications to skeletal code with dynamically-checkable design contracts written in that target language. This provides a significant improvement over existing tools that are custom built to provide automatic conversion from Object-Z only to a specific implementation language. FOZCIL streamlines the process of developing both the reusual language-independent portions and the language dependent portions of such tools.

Investigating the Quality and Conformity of Carbon Steel AISI 1035 under Varying Heat Treatment Conditions

Test case generation, selection and coverage from natural language

BZ‐TESTING‐TOOLS (BZ‐TT) is a tool set for automated test case generation from B and Z specifications. BZ‐TT uses boundary and cause–effect testing on the basis of the formal model. It has been used and validated on several industrial applications in the domain of critical software, particularly smart card and transport systems. This paper presents the test coverage criteria supported by BZ‐TT. On the one hand, these correspond to various classical structural coverage criteria, but specialized to the case of B abstract machines. The paper gives algorithms for these in Prolog. On the other hand, BZ‐TT introduces new coverage criteria for complex data structures, based on boundary analysis: this paper defines weak and strong state‐boundary coverage, input‐boundary coverage and output‐boundary coverage. Finally, the paper describes how BZ‐TT presents a unified view of these criteria to the validation engineer, and allows him or her to control the test case explosion on a coarse basis (choosing from a range of coverage criteria) as well as a fine basis (selecting options for each state or input variable). Copyright © 2004 John Wiley & Sons, Ltd.

Peer-to-peer (P2P) systems, characterized by their decentralized architecture, have become crucial to various applications such as file sharing, content distribution, and blockchain technologies. Due to their dynamic and heterogeneous nature, ensuring their reliability and robustness presents challenges. Such an issue has motivated researchers to develop testing techniques for these systems. Despite advances in research about software testing for distributed systems, there is a lack of data about their applicability and which features from P2P systems are highlighted by the proposed testing approaches. This paper provides a comprehensive overview of the state-of-the-art software testing for P2P systems. We systematically examine the inherent challenges and situations. Our findings show that some aspects of P2P systems can make the software testing activity difficult, such as validation of global properties and consensus, peer churn, a large volume of data, malformed data, reproducibility, and non-determinism. Several testing techniques have been used in this context, such as model-based testing, fault injection, fuzzing, stress testing, differential testing, conformance testing, and concurrency testing. We highlight fuzzing for testing malformed data, global properties, parallel events, and model-based testing as a powerful technique to abstract the complexities and varieties of P2P applications during the testing activities. Moreover, blockchain applications are the most addressed P2P application explored by the studies, where most papers were published in 2023, highlighting the theme’s relevance.

This paper presents a practical approach useful for assessing the coverage of test suites which are generated automatically from Extended Finite State Machine specifications. It is based on transition coverage, a variant of branch coverage which seems to correspond to the needs of test designers for conformance testing of communication protocols. It presents a tool for implementing this approach, and discusses the facilities needed for a qualitative analysis of coverage.