Information theory and alternate-hypothesis tests

A non-parametric sequential rank-sum probability ratio test method for binary hypothesis testing

Because computers (except for parallel computers) generate simulation outputs sequentially, we recommend sequential probability ratio tests (SPRTs) for the statistical analysis of these outputs. However, until now simulation analysts have ignored SPRTs. To change this situation, we review SPRTs for the simplest case; namely, the case of choosing between two hypothesized values for the mean simulation output. For this case, the classic SPRT of Wald (Wald A. Sequential tests of statistical hypotheses. Ann Math Stat 1945; 16: 117–186) allows general types of distribution, including normal distributions with known variances. A modification permits unknown variances that are estimated. Hall (Hall WJ. Some sequential analogs of Stein’s two-stage test. Biometrika 1962; 49: 367–378) developed a SPRT that assumes normal distributions with unknown variances estimated from a pilot sample. A modification uses a fully sequential variance estimator. In this paper, we quantify the performance of the various SPRTs, using several Monte Carlo experiments. In experiment #1, simulation outputs are normal. Whereas Wald’s SPRT with estimated variance gives too high error rates, Hall’s original and modified SPRTs are “conservative”; that is, the actual error rates are smaller than those prespecified (nominal). Furthermore, our experiment shows that the most efficient SPRT is Hall’s modified SPRT. In experiment #2, we estimate the robustness of these SPRTs for non-normal output. For these two experiments, we provide details on their design and analysis; these details may also be useful for simulation experiments in general.

In medical, health, and sports sciences, researchers desire a device with high reliability and validity. This article focuses on reliability and validity studies with n subjects and m ≥2 repeated measurements per subject. High statistical power can be achieved by increasing n or m, and increasing m is often easier than increasing n in practice unless m is too high to result in systematic bias. The sequential probability ratio test (SPRT) is a useful statistical method which can conclude a null hypothesis H0 or an alternative hypothesis H1 with 50% of the required sample size of a non-sequential test on average. The traditional SPRT requires the likelihood function for each observed random variable, and it can be a practical burden for evaluating the likelihood ratio after each observation of a subject. Instead, m observed random variables per subject can be transformed into a test statistic which has a known sampling distribution under H0 and under H1. This allows us to formulate a SPRT based on a sequence of test statistics. In this article, three types of study are considered: reliabilityof a device, reliability of a device relative to a criterion device, and validity of a device relative to a  criterion device. Using SPRT for testing the reliability of a device, for small m, results in an average sample size of about 50% of the fixed sample size for a non-sequential test. For comparing a device to criterion, the average sample size approaches to 60% approximately as m increases. The SPRT tolerates violation of normality assumption for validity study, but it does not for reliability study.

A method to solve multi-target classification problems with unknown clutter parameters is proposed in this paper. The unknown parameter is estimated and synthesized at each observation, and probability of each hypothesis is updated. Subsequently, the optimal waveform for the next illumination is designed based on NP criteria, and the final decision is made based on the sequential probability ratio testing. Simulated results are presented based on our method and show that the optimal waveform-based sequential testing can be decided through reduction of the average illumination number. Furthermore, results indicate a significant improvement over the non-optimal waveforms.

The semiconductor manufacturing is one of the most sophisticated processes in modern industries. It contains hundreds of complex operations, and this makes semiconductor reliability a challenging task especially in recent years that integrated circuits (ICs) have been broadly used at much wider areas including the health cares, automotive, and aerospace applications, where safety is the major concern. To make things worse, due to the shorter product life cycles and severe competitions, time-consuming reliability tests become unacceptable. The industries are in urgent need of a fast and effective way to quickly assess reliability risks. Under all these stringent limitations and the high reliability targets, we propose a consolidated approach that consists of an efficient and effective reliability baseline (BL) management system and a test scheme developed based on sequential probability ratio test (SPRT). To ensure a trustworthy setting for the SPRT test scheme, we apply the Bayesian approach to continuously update the BL database. An integrated flow chart is introduced to elucidate our proposed methodology. It is the first time SPRT is applied on reliability risk assessments with two unknown parameters. Several actual cases from semiconductor arena are reported to demonstrate the merits and powers of our proposed methodology. From these real cases, as a side benefit, we show the wide applicability of our method, which does not limit to reliability but on yield and functionality checks as well. To facilitate the use of our method by field practitioners, we do the best to explain derivations in a plain manner and provide analysis codes in R (in Supplementary Material) to make our approach as portable as possible.

In this paper, using the notion of a binary Channel Matrix as well known in Information Theory, we present an equivalent relation between the SPRT (Sequential Probability Ratio Test) and Bayes’ Updating. Moreover, we show the relationship between the SPRT and NSPR (New Sequential Probability Ratio) where a Hidden Markov Model with Poisson process is introduced as structural change model. And we also provide the change point detection performance of SPRT and NSPR by experimental results.

We consider a single-hop, random-access, wireless sensor network (WSN) performing sequential distributed detection under a bandwidth constraint. At each time slot, the sensor nodes will make their decisions on whether the observed event is happening. The sensor censoring strategy is applied such that only the local decisions equal to one (i.e., the event happens) will be sent to the fusion center (FC). Because only one transmission channel between the FC and the sensor nodes is assumed, if two or more local decisions are sent at a time slot, a packet collision happens. We design a sequential probability ratio test (SPRT) at the FC that is aware of the collisions. The performance measures of the collision-aware SPRT are analyzed based on a Markov chain. In addition, we derive the asymptotic relative efficiency of the collision-aware SPRT relative to another SPRT. Numerical results show some interesting characteristics of the collision-aware SPRT. For example, the collision-aware SPRT prefers the scenario that many local decisions are likely to be sent per time slot.

In this article the problem of a sequential test for the model of independent non-identically distributed observations is considered. Based on recursive calculation a new numerical approach to approximate test characteristics for a sequential probability ratio test (SPRT) and a truncated SPRT (TSPRT) is constructed. The problem of robustness evaluation is also studied when the contamination is presented by the distortion of the distributions of all increments of the log-likelihood ratio statistics. The two-side truncated functions are proposed to be used for constructing the robustified SPRT. An algorithm to choose the thresholds of these truncated functions is indicated. The results are applied for a sequential test on parameters of time series with trend. Some kinds of the contaminated models of time series with trend are used to study the robustness of the truncated SPRT. Numerical examples confirming the theoretical results mentioned above are given.

The sequential probability ratio test (SPRT) is a fundamental tool for sequential analysis. It forms the basis of numerous sequential techniques for different applications; for example, the truncated SPRT and Page's cumulative sum test (CUSUM). The performance of SPRT is characterized by two important functions—operating characteristic (OC) and average sample number (ASN), and CUSUM's performance is revealed by the average run length (ARL) function. These functions have been studied extensively under the assumption of independent and identically distributed log-likelihood ratios (LLRs) with constant bounds, which is too stringent for many applications. In this article, inductive integral equations governing these functions are developed under very general settings—the bounds can be time-varying and the LLRs are assumed independent but nonstationary. These inductive equations provide a theoretical foundation for performance analysis. Unfortunately, they have nonunique solutions in the general case except for the truncated SPRT. Numerical solutions for some frequently encountered special cases are developed and are compared with the results of Monte Carlo simulations.

Using one class SVM to counter intelligent attacks against an SPRT defense mechanism

We apply high-dimensional analysis of variance (HANOVA) and sequential probability ratio test (SPRT) to detect buried land mines from array ground penetrating radar (GPR) measurements. The GPR array surveys a region of interest in a progressive manner starting at a known position and moving step by step in a fixed direction. Our detection method consists of two stages. Because, at each stop of the array the path lengths are different from every transmitter/receiver pair to a mien target, there exists statistically significant difference among received signals when a mine target is presented. Thus, the first step in our processing consists of a HANOVA test to detect this statistical difference at each stop. HANOVA does not incorporate new data as the GPR array moves down-track. So secondly, we resort to a sequential probability ratio test to look for changes in the HANOVA statistics as the array proceeds down track. The SPRT allows for real-time processing as anew data are obtained by the GPR array. Finally, real sensor data are processed to verify the method.

Conditional frequentist tests of a precise hypothesis versus a composite alternative have recently been developed, and have been shown to be equivalent to conventional Bayes tests in the very strong sense that the reported frequentist error probabilities equal the posterior probabilities of the hypotheses. These results are herein extended to sequential testing, and yield fully frequentist sequential tests that are considerably easier to use than are conventional sequential tests. Among the interesting properties of these new tests is the lack of dependence of the reported error probabilities on the stopping rule, seeming to lend frequentist support to the stopping rule principle. Keywords:Bayes factor; Composite hypothesis; Conditional test; Error probability; Likelihood ratio; Sequential probability ratio test; Sequential test.

The SPRT control chart for the process mean with samples starting at fixed times

ABSTRACTWe develop truncated sequential probability ratio test (SPRT) procedures for multivariate normal data. The framework includes a general cost structure and arbitrary mean and covariance structures. The truncated SPRT solutions have a practical and easy-to-use decision boundary representation. In the homogeneous case, a very fast recursive algorithm is presented for calculating the decision boundaries. Misclassification rates and expected sample size are investigated and the results are compared with a nonsequential procedure. A real-life data set on kidney dysfunction following heart surgery is used to illustrate the truncated SPRT procedure.

The operating characteristic (OC) and average sample number (ASN) of the sequential probability ratio test (SPRT) and multi-hypothesis SPRT (MSPRT) are studied. We consider the case where the observation sequence is independent but not necessarily identically distributed. Also, the thresholds for the test can be time varying. Based on the governing equations for OC and ASN of the SPRT developed in our previous work, a solution for the general case is proposed. The governing equations for OC and ASN of the MSPRT are also obtained. Numerical solutions for MSPRT are developed. Basically, the solutions rely on approximating the original test by truncation, that is, truncating the test at some finite time $K$ . We show that under some mild conditions, the approximation error diminishes as $K$ increases, at the cost of increased computation. Numerical examples are provided to demonstrate our solutions by comparing with Monte Carlo simulations, Simon’s lower bound, and Dragalin’s method (if available) for ASN.