Quantum Concolic Testing

This work describes the working of white-box testing for the Service-Oriented Architecture (SOA) based Application. Now, a days it is very essential to perform code coverage testing to understand the quality of software. This paper deals with the measurement of branch coverage percentage for the BPEL architecture that orchestrates all the services, which are distributed geographically. Here, we are testing the code coverage of BPEL architecture, which actually shows the invocations of services when it is required. This work integrates the existing open source tools, to automate the computation of branch coverage for SOA based application. This paper shows a novel technique for generation of test cases and computing branch coverage through our proposed tool.

We present hybrid concolic testing, an algorithm that interleaves random testing with concolic execution to obtain both a deep and a wide exploration of program state space. Our algorithm generates test inputs automatically by interleaving random testing until saturation with bounded exhaustive symbolic exploration of program points. It thus combines the ability of random search to reach deep program states quickly together with the ability of concolic testing to explore states in a neighborhood exhaustively. We have implemented our algorithm on top of CUTE and applied it to obtain better branch coverage for an editor implementation (VIM 5.7, 150 K lines of code) as well as a data structure implementation in C. Our experiments suggest that hybrid concolic testing can handle large programs and provide, for the same testing budget, almost 4times the branch coverage than random testing and almost 2times that of concolic testing.

Recently, symbolic execution has been witnessed great success in applying symbolic execution to automatically generate high coverage test inputs for programs testing. While path explosion still a primary challenge in scaling concolic testing to large programs, meanwhile constraint solving is fundamental to concolic testing as a constraint solver is continuously invoked during analysis, while the main roadblock to performance of concolic testing engines is the time spent in constraint solving. To address this issue and achieve the purpose of selecting the path with simplest constraints to solve, this paper presents a new heuristic search strategy which determines the path priority based on the complexity of path constraints. We have implemented the strategy in angr, a binary-oriented concolic testing tools implemented by Shellphish, and evaluated this strategy on CTF binaries provided by famous institution. The results show that our strategy achieves significantly greater testing efficiency in larger programs while no apparent effect in minor programs.

ESBMC, a bounded model checking (BMC) verifier based on SMT solving, has demonstrated its effectiveness in bug detection in recent software verification competitions. We extend its capabilities to enable branch coverage analysis and test suite generation. Our contributions are twofold: (1) we define a branch coverage property and instrument the control flow graph (CFG) to compute branch coverage using SMT solving, and (2) we propose an incremental multi-property reasoning algorithm for efficient and sound test case generation. ESBMC is ranked 7th in the category of Test-Comp 2025.

Software testing refers to a process that improves the quality of software systems through bug detection, but software testing is one of the time and cost-consuming stages in software development. Hence, software test automation is regarded as a solution, which can facilitate heavy and laborious tasks of testing. Problem: Automatic generation of data with maximum coverage of program branches is regarded as an NP-complete optimization problem. Several heuristic algorithms have been proposed for this problem. Failure to maximise branch coverage, the poor success rate in optimal test data generation, and low stable results are the major demerits of the previous methods. Goal: Enhancing the branch coverage rate of the generated test data, enhancing the success rate in generating the test data with maximum coverage, and enhancing the stability and speed criteria are the main goals of this study. Method: In this study, a combination of grey wolf optimization algorithm and genetic algorithm have been used to automatically generate optimal test data. The proposed hybrid method (Fuzuli <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> ) tries to generate test data with maximum branch coverage at the software source code level. Results: The results obtained from the proposed algorithm were compared with those of the following algorithms: Shuffled Frog Leaping Algorithm (SFLA), Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), and Genetic Algorithm (GA).The results obtained from running a wide range of tests on standard benchmark programs showed that the proposed algorithm outperforms other algorithms with an average coverage of %99.98, a success rate of %99.97, and an average output of 2.86.

As quantum computing is rising in popularity, the amount of quantum programs and the number of developers writing them are increasing rapidly. Unfortunately, writing correct quantum programs is challenging due to various subtle rules developers need to be aware of. Empirical studies show that 40–82% of all bugs in quantum software are specific to the quantum domain. Yet, existing static bug detection frameworks are mostly unaware of quantum-specific concepts, such as circuits, gates, and qubits, and hence miss many bugs. This paper presents LintQ, a comprehensive static analysis framework for detecting bugs in quantum programs. Our approach is enabled by a set of abstractions designed to reason about common concepts in quantum computing without referring to the details of the underlying quantum computing platform. Built on top of these abstractions, LintQ offers an extensible set of ten analyses that detect likely bugs, such as operating on corrupted quantum states, redundant measurements, and incorrect compositions of sub-circuits. We apply the approach to a newly collected dataset of 7,568 real-world Qiskit-based quantum programs, showing that LintQ effectively identifies various programming problems, with a precision of 91.0% in its default configuration with the six best performing analyses. Comparing to a general-purpose linter and two existing quantum-aware techniques shows that almost all problems (92.1%) found by LintQ during our evaluation are missed by prior work. LintQ hence takes an important step toward reliable software in the growing field of quantum computing.

Unit testing is crucial for software development and maintenance. Effective unit testing ensures and improves software quality, but writing unit tests is time-consuming and labor-intensive. Recent studies have proposed deep learning (DL) techniques or large language models (LLMs) to automate unit test generation. These models are usually trained or fine-tuned on large-scale datasets. Despite growing awareness of the importance of data quality, there has been limited research on the quality of datasets used for test generation. To bridge this gap, we systematically examine the impact of noise on the performance of learning-based test generation models. We first apply the open card sorting method to analyze the most popular and largest test generation dataset, Methods2Test, to categorize eight distinct types of noise. Further, we conduct detailed interviews with 17 domain experts to validate and assess the importance, reasonableness, and correctness of the noise taxonomy. Then, we propose CleanTest, an automated noise-cleaning framework designed to improve the quality of test generation datasets. CleanTest comprises three filters: a rule-based syntax filter, a rule-based relevance filter, and a model-based coverage filter. To evaluate its effectiveness, we apply CleanTest on two widely-used test generation datasets, i.e., Methods2Test and Atlas. Our findings indicate that 43.52% and 29.65% of datasets contain noise, highlighting its prevalence. Finally, we conduct comparative experiments using four LLMs (i.e., CodeBERT, AthenaTest, StarCoder, and CodeLlama7B) to assess the impact of noise on test generation performance. The results show that filtering noise positively influences the test generation ability of the models. Fine-tuning the four LLMs with the filtered Methods2Test dataset, on average, improves its performance by 67% in branch coverage, using the Defects4J benchmark. For the Atlas dataset, the four LLMs improve branch coverage by 39%. Additionally, filtering noise improves bug detection performance, resulting in a 21.42% increase in bugs detected by the generated tests.

Coverage-Guided fuzzing is the type of fuzzing which focuses on the code or branch coverage. It is mainly efficient in detecting buffer overflow. For the code coverage, it is an important parameter of performance evaluation of the Coverage-Guided fuzzing tools, since the higher coverage means higher possibility of bug detection. However, the timeout set for the fuzzing also affects the efficiency, due to the fact that the growth rate of code coverage will gradually become slower as the running time becoming longer. Setting a timeout that is too long cannot help improving the coverage nor the bug detected, and it will be a complete waste of time. Since selecting the fuzzing time might be quite confusing for testers, in this paper, the author decided to discover the relationship of the growth of the coverage and the running time of the. AFL and FuzzFactory were used for the evaluation and the timeout were set as 1 second, 1 minute, 1 hour, 6 hours and 12 hours respectively. The results showed the relationship between fuzzing time and the performance, for all fuzzers and the effect between different fuzzers.

Efficient program path exploration is important for many software engineering activities such as testing, debugging, and verification. However, enumerating all paths of a program is prohibitively expensive. In this article, we develop a partitioning of program paths based on the program output. Two program paths are placed in the same partition if they derive the output similarly, that is, the symbolic expression connecting the output with the inputs is the same in both paths. Our grouping of paths is gradually created by a smart path exploration. Our experiments show the benefits of the proposed path exploration in test-suite construction. Our path partitioning produces a semantic signature of a program—describing all the different symbolic expressions that the output can assume along different program paths. To reason about changes between program versions, we can therefore analyze their semantic signatures. In particular, we demonstrate the applications of our path partitioning in testing and debugging of software regressions.

SummaryTo deliver reliable software, developers may rely on the fault detection capability of test suites. To evaluate this capability, they can apply code coverage metrics before a software release. However, recent research results have shown that these metrics may not provide a solid basis for this evaluation. Moreover, the fixing of a fault has a cost, and not all faults have the same impact regarding software reliability. In this sense, operational testing aims at assessing parts of the system that are more valuable for users. The goal of this work is to investigate whether traditional code coverage and code coverage merged with operational information can be related to post‐release bug detection. We focus on the scope of proprietary software under continuous delivery. We performed an exploratory case study where code branch and statement coverage metrics were collected for each version of a proprietary software together with real usage data of the system. We then measured the ability to explain the bug‐fixing activity after version release using code coverage levels. We found that traditional statement coverage has a moderate negative correlation with bug‐fixing activities, whereas statement coverage merged with the operational profile has a large negative correlation with higher confidence. Developers can consider operational information as an important factor of influence that should be analysed, among other factors, together with code coverage to assess the fault detection capability of a test suite.

Detecting concurrency bugs is becoming increasingly important. Many pattern-based concurrency bug detectors focus on the specific types of interleavings that are correlated to concurrency bugs. To detect multiple types of concurrency bugs, general detectors focus on multiple types of interleaving patterns, including data race pattern, atomicity violation pattern, and atomic-set violation pattern. Unfortunately, they suffer from redundant analysis due to repeated interleavings, which may affect the efficiency of concurrency bug detection. Hence, we propose an approach to identify the repeated interleavings. To the best of our knowledge, this is the first approach that can prune repeated interleavings to improve the efficiency of concurrency bug detection. We apply our approach to existing general detectors (PECAN and Maple) to avoid analyzing repeated interleavings. We evaluated the general detectors with and without our approach, respectively. The experimental results show that the bug detection results are not affected. With our approach, the bug detection time of PECAN and Maple are reduced by 40.0 % and 44.4 %, respectively. Additionally, our approach does not affect the overhead of bug detection, and consumes only a little memory.

Efficient program path exploration is important for many software engineering activities such as testing, debugging and verification. However, enumerating all paths of a program is prohibitively expensive. In this paper, we develop a partitioning of program paths based on the program output. Two program paths are placed in the same partition if they derive the output similarly, that is, the symbolic expression connecting the output with the inputs is the same in both paths. Our grouping of paths is gradually created by a smart path exploration. Our experiments show the benefits of the proposed pathexploration in test-suite construction.Our path partitioning produces a semantic signature of a program --- describing all the different symbolic expressions that the output can assume along different program paths. To reason about changes between program versions, we can therefore analyze their semantic signatures. In particular, we demonstrate the applications of our path partitioning in debugging of software regressions.

aSymbolic execution is an important software analysis technique, but it faces challenges such as path explosion, which leads to a reduction in efficiency. Existing path exploration strategies, such as Random State Search, typically exhibit poor adaptability to real-world programs and lack effective path selection strategies. To address these challenges, this paper proposes a Transfer Learning-based Symbolic Execution Path Exploration Strategy, TLS (Transfer Learning Search). We adopt a transfer learning method based on functional classification to optimize existing symbolic execution strategies. Specifically, real-world programs are classified according to their functional characteristics, and transfer learning is applied by freezing partial layers of existing neural networks with training sets from each program family that better reflect its features. Multiple models are trained based on different training sets to adapt to various program families. Experimental results show that this strategy solves the problem of insufficient training data for real-world programs. Compared to traditional heuristic methods such as random-path (rps) and random-state (rss) strategies, this approach significantly improves instruction coverage and branch coverage on specific program families. For example, in the Grep program test, branch coverage increased by approximately fifteen percentage points, generating more test cases. This approach provides a new and effective solution to the adaptability problem of symbolic execution for complex programs.

Dynamic symbolic execution or concolic testing has been proposed recently to effectively generate test inputs for real-world programs. Unfortunately, dynamic symbolic execution techniques do not scale well for large realistic programs, because often the number of feasible execution paths of a program increases exponentially with the increase in the length of an execution path. We believe that programmers usually have a good understanding of programs they write and could help concolic testing to guide path exploration effectively so that concolic testing can achieve the desired testing goal quickly. In this paper, we propose a simple annotation mechanism, called Guide SE, that enables a programmer to guide concolic testing by annotating the program under test. We have currently identified three mechanisms for providing these annotations: control annotations, equivalence annotations, and data annotations. Controls annotations allow a programmer to specify the paths of interest using regular expressions. Equivalence annotations enable a programmer to prune paths that result in user defined equivalent states. Data annotations allow a programmer to abstract portions of the program under test and to explore the reduced path space of the abstract program.

The storage and processing of quantum information are susceptible to external noise, resulting in computational errors. A powerful method to suppress these effects is quantum error correction. Typically, quantum error correction is executed in discrete rounds, using entangling gates and projective measurement on ancillary qubits to complete each round of error correction. Here we use direct parity measurements to implement a continuous quantum bit-flip correction code in a resource-efficient manner, eliminating entangling gates, ancillary qubits, and their associated errors. An FPGA controller actively corrects errors as they are detected, achieving an average bit-flip detection efficiency of up to 91%. Furthermore, the protocol increases the relaxation time of the protected logical qubit by a factor of 2.7 over the relaxation times of the bare comprising qubits. Our results showcase resource-efficient stabilizer measurements in a multi-qubit architecture and demonstrate how continuous error correction codes can address challenges in realizing a fault-tolerant system.