CP with Architectural State Lookup for Functional Test Generation

In this paper, a heuristic method based on a pre-calculated prediction of tests is presented. This method is for combinational test generation and applies to the combinational parts of sequential circuits equally wells. The proposed method is Directed Random Test Generation that is based on SCOAP observability and controllability parameters. RTG methods are usually evaluated based on fault coverage that grows exponentially with the number of test vectors. Instead of pure random testing with a massive number of test vectors, fewer test vectors are selected by Directed Random Test Generation, and more faults are detected in the early stages of test generation. As a result, fewer faults remain for the deterministic test stage, which is helpful where online testing is considered. A comparison between random testing and directed random testing is performed on combinational benchmarks, using MATLAB and ModelSim simulation environments. The focus is on the stuck-at fault models in random logic circuits. The simulation results show our method has a sharper increase in fault coverage when compared with pure random test generation.

In this paper, we extend the Constraint Programming (CP) based functional test generation framework with a novel concept of distribution constraints. The proposed extension is motivated by requirements arising in the functional validation field, when a validation engineer needs to stress an interesting architectural event following some special knowledge of design under test or a specific validation plan. In such cases there arises the need to generate a sequence of test instructions or a collection of tests according to user-given distribution requirements which specify desired occurrence frequencies for interesting events. The proposed extension raises the expressive power of the CP based framework and allows specifying distribution requirements on a collection of Constraint Satisfaction Problem (CSP) solutions. We formalize the notion of distribution requirements by defining the concept of distribution constraints. We present two versions of problem definition for CP with distribution constraints, both of which arise in the context of functional test generation. The paper presents algorithms to solve each of these two problems. One family of the proposed algorithms is based on CP, while the other one makes use of both CP and the linear programming (LP) technology. All of the proposed algorithms can be efficiently parallelized taking advantage of the multi core technology. Finally, we present experimental results to demonstrate the effectiveness of proposed algorithms with respect to performance and distribution accuracy.KeywordsConstraint Satisfaction ProblemPrecision ErrorProperty SubsetDistribution ConstraintConstraint Satisfaction Problem InstanceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Deterministic test pattern generation for combinational circuits is a time consuming process. So conducting a random test generation method at the initial stage and then using a deterministic test generation technique in order to generate test vectors for hard to detect faults is a common method. Generating random test patterns and then pruning them using a suitable heuristic is a viable solution that we are using in our paper. Our proposed method evaluates merit of a randomly generated test vector and decides to accept or reject it based on expected fault coverage. This expected value is dynamically updated as test vectors are chosen. The proposed method can be used in the initial phase of a test generation process instead of conventional pure random test generation method in order to reduce the number of test vectors in the final test set.

We investigated the influence of the random generation methods to the results of the functional delay test generation. There are the three possibilities: random generation of values 0 and 1 when the probability for the appearance of each value is 50% in every bit of test pattern; random generation of large integer values, which then are split into the sequences of 0’s and 1‘s according to the rules of the conversion of the decimal number to the binary number; anti-random generation when the presence of the earlier generated patterns is taken into account. We adopted the anti-random generation for the functional delay faults. The obtained results of the investigation indicate that the random generation has the direct influence to the results of the functional delay test generation. The largest coverage of transition fault is obtained when the anti-random generation is employed.

The authors define the notion of a lingering synchronisation effect. Such an effect occurs when a primary input cube (an incompletely specified primary input vector) determines the state of a circuit for several time units after it is applied. A primary input cube with a lingering synchronisation effect may prevent certain faults from being detected when it appears repeatedly in a test sequence. It should therefore be avoided when the goal is to achieve a high fault coverage. The authors demonstrate that benchmark circuits have primary input cubes with small numbers of specified values (typically one or two), which have lingering synchronisation effects. In some cases, the synchronisation effects linger for large numbers of time units. The authors define a ranking of primary input cubes based on the severity of their lingering synchronisation effects. They describe a random test generation process that avoids primary input cubes with lingering synchronisation effects, and achieves high fault coverage for benchmark circuits. The test generation process uses the severity of the lingering synchronisation effects of the primary input cubes to decide on the ones it should avoid.

Many software projects maintain automated GUI tests that are repeatedly executed for regression testing. Every test run executes exactly the same fixed sequence of steps confirming that the currently tested version shows precisely the same behavior as the last version. The confirmatory approach implemented by these tests limits their ability to find new defects. We therefore propose to combine existing automated regression tests with random test generation. Random test generation creates a rich variety of test steps that interact with the system under test in new, unexpected ways. Enhancing existing test cases with random test steps allows revealing new, hidden defects with little extra effort. In this paper we describe our implementation of a hybrid approach that enhances existing GUI test cases with additional, randomly generated interactions. We conducted an experiment using a mature, widely-used open source application. On average the added random interactions increased the number of visited application windows per test by 23.6% and code coverage by 12.9%. Running the enhanced tests revealed three new defects.

Multithreaded processors increase throughput by executing multiple independent programs on a single pipeline. Simultaneous Multithreaded (SMT) processors execute multiple threads simultaneously thus add a significant dimension to the design complexity. Dealing with this complexity calls for extended and innovative design verification efforts. This paper develops an analytic model based SMT random test generation technique. SMT analytic model parameters are applied to create random tests with high utilization and increased contention. To demonstrate the methodology, parameters extracted from the PPC ISA and sample processor configurations are simulated on the SMT analytic model. The methodology focuses on exploiting data/control and structural hazards to guide the random test generator to create effective SMT tests.

Multithreaded processors increase throughput by executing multiple independent programs on a single pipeline. Simultaneous Multithreaded (SMT) processors execute multiple threads simultaneously thus add a significant dimension to the design complexity. Dealing with this complexity calls for extended and innovative design verification efforts. This paper develops an analytic model based SMT random test generation technique. SMT analytic model parameters are applied to create random tests with high utilization and increased contention. To demonstrate the methodology, parameters extracted from the PPC ISA and sample processor configurations are simulated on the SMT analytic model. The methodology focuses on exploiting data/control and structural hazards to guide the random test generator to create effective SMT tests.

This paper presents a universal random test generator template for the design verification of microprocessors and system-on-chips (SOCs). The tool enables verification of the product in one continuous, integrated environment, from C model to behavioral RTL and gate to system-level integration, all in one self-contained chassis. Due to complexity of large designs, it has been a common practice to rely on the power of randomization, to bless us with the humanly not-conceivable corner cases that can arise in reality. There are lots of common features shared by random tools used for testing products with diverse functionalities. This paper proposes a template which captures the commonalities among the different random testing tools and enables the user to quickly design a random test generator by adding product-specific details and using most of the methods available in the template. This leads to high degree of code reuse, less debugging of the random tool and huge reduction in design-cycle time. In addition the template provides enough flexibility and interfaces to enable the execution of the generated tests on targets which may be a C model, RTL or the final chip. By this, one may test a software component, say a bootup code for the system-on-chip or microprocessor at all stages of its design, namely, the software prototype, the RTL at the pre-silicon level and finally the chip, at a post-silicon level. This satisfies the expectations out of a verification platform for a hardware-software codesign environment. The random test generator template was employed for testing a x86-compatible microprocessor both at RTL and post-silicon stage and a software model of a 802.11 MAC. The results are presented in the paper.

Multicore chips are expected to rely on coherent shared memory. Albeit the coherence hardware can scale gracefully, the protocol state space grows exponentially with core count. That is why design verification requires directed test generation (DTG) for dynamic coverage control under the tight time constraints resulting from slow simulation and short verification budgets. Next generation EDA tools are expected to exploit Machine Learning for reaching high coverage in less time. We propose a technique that addresses DTG as a decision process and tries to find a decision-making policy for maximizing the cumulative coverage, as a result of successive actions taken by an agent. Instead of simply relying on learning, our technique builds upon the legacy from constrained random test generation (RTG). It casts DTG as coverage-driven RTG, and it explores distinct RTG engines subject to progressively tighter constraints. We compared three Reinforcement Learning generators with a state-of-the-art generator based on Genetic Programming. The experimental results show that the proper enforcement of constraints is more efficient for guiding learning towards higher coverage than simply letting the generator learn how to select the most promising memory events for increasing coverage. For a 3-level MESI 32-core design, the proposed approach led to the highest observed coverage (95.81%), and it was 2.4 times faster than the baseline generator to reach the latter’s maximal coverage.

Functional verification of microprocessors is one of the most complex and expensive tasks in the current system-on-chip design process. A significant bottleneck in the validation of such systems is the lack of a suitable functional coverage metric. This paper presents a functional coverage based test generation technique for pipelined architectures. The proposed methodology makes three important contributions. First, a general graph-theoretic model is developed that can capture the structure and behavior (instruction-set) of a wide variety of pipelined processors. Second, we propose a functional fault model that is used to define the functional coverage for pipelined architectures. Finally, test generation procedures are presented that accept the graph model of the architecture as input and generate test programs to detect all the faults in the functional fault model. Our experimental results on two pipelined processor models demonstrate that the number of test programs generated by our approach to obtain a fault coverage is an order of magnitude less than those generated by traditional random or constrained-random test generation techniques.

We propose a functional fault model for delay faults in combinational circuits and describe a functional test generation procedure based on this model. The proposed method is most suitable when a gate-level description of the circuit-under-test, necessary for employing existing gate-level delay fault test generators, is not available or does not accurately describe the circuit. It is also suitable for generating tests in early design stages of a circuit, before a gate-level implementation is selected. In addition, it can potentially be employed to supplement conventional test generators for gate-level circuits to reduce the cost of handling large numbers of paths. A parameter called Δ is used to control the number of funtional faults targeted and thus the number of tests generated. If Δ is unlimited, the functional test set detects every robustly testable path delay fault in any gate-level implementation of the given ciruit. An appropriate subset of tests can be selected once the implentation is known. The test sets generated for various values of Δ are fault simulated on gate-level realizations to demonstrate their effectiveness. The experiments indicate that functional test sets may be able to identify functions whose realizations have low path delay fault coverage.

Researchers have proposed different methods for testing digital systems and circuits in the last couple of decades. The need for testing digital logic circuits has become more important than ever due to the growing complexity of such systems. During the design phase, testing is focusing on design defects, as well as manufacturing and wear out type of defects. Failures in digital systems could be caused, for example, by design errors, the use of inherently probabilistic devices, and manufacturing variability. As a way to test digital systems in a more efficient way, automated test generation has been proposed to automatically create tests that can quickly and accurately identify faulty components. Examples of such techniques are the sequential test generation, the scan path testing, and the random test generation techniques. With the research domain becoming more mature and growing, it is essential to systematically identify, analyze, and classify these contributions. We performed a systematic mapping study of automated test generation for digital circuits aimed at providing an overview of the application of these techniques. We focused on three of the most widely-used and well-supported hardware description languages (HDLs) for digital systems: Verilog, SystemVerilog, and VHDL. Our results suggest that the majority of the test generation methods for digital circuits are focused on the behavioral and register-transfer design levels. Fault-independent and fault-oriented test generation are the most frequently reported types of test generation methods, while HDL model simulation is the most common test generation technology used to search for test cases in these academic studies. While the results are suggesting a growing interest in this area, the majority of articles are published as conferences papers. Our results show that only 31% of the methods are implemented as software tools and only 63% of all contributions are actually generating executable test cases. This study makes three important contributions, (i) a state-of-the-art of test generation for digital system designs research is provided, (ii) the reported characteristics are identified in both the primary papers and experimental reports, (iii) gaps and opportunities for future test generation for digital system designs research are identified.

This paper presents a novel method for functional test generation aiming at exploring control state space of the design. The steady-state probabilities (SP's) of the abstract design's control FSM are used to guide test generation. The SP's of the states can reflect how hard the states can be reached, and the hard-to-reach states are assigned with high priority to be exercised. Experimental results show that our method has better performance in test generation in comparison with constrained random simulation, and demonstrate that SP's provide good guidance on traversing hard-to-reach states of the design under validation.

In order to reduce the time-to-market and simplify gate-level test generation for digital integrated circuits, GA-based functional test generation techniques are proposed for behavioral and register transfer level designs. The functional tests generated can be used for design verification, and they can also be reused at lower levels (i.e. register transfer and logic gate levels) for testability analysis and development. Experimental results demonstrate the effectiveness of the method in reducing the overall test generation time and increasing the gate-level fault coverage.