Sequential probability ratio test for Multiple-Objective Ranking and Selection

A non-parametric sequential rank-sum probability ratio test method for binary hypothesis testing

In medical, health, and sports sciences, researchers desire a device with high reliability and validity. This article focuses on reliability and validity studies with n subjects and m ≥2 repeated measurements per subject. High statistical power can be achieved by increasing n or m, and increasing m is often easier than increasing n in practice unless m is too high to result in systematic bias. The sequential probability ratio test (SPRT) is a useful statistical method which can conclude a null hypothesis H0 or an alternative hypothesis H1 with 50% of the required sample size of a non-sequential test on average. The traditional SPRT requires the likelihood function for each observed random variable, and it can be a practical burden for evaluating the likelihood ratio after each observation of a subject. Instead, m observed random variables per subject can be transformed into a test statistic which has a known sampling distribution under H0 and under H1. This allows us to formulate a SPRT based on a sequence of test statistics. In this article, three types of study are considered: reliabilityof a device, reliability of a device relative to a criterion device, and validity of a device relative to a  criterion device. Using SPRT for testing the reliability of a device, for small m, results in an average sample size of about 50% of the fixed sample size for a non-sequential test. For comparing a device to criterion, the average sample size approaches to 60% approximately as m increases. The SPRT tolerates violation of normality assumption for validity study, but it does not for reliability study.

Conditional frequentist tests of a precise hypothesis versus a composite alternative have recently been developed, and have been shown to be equivalent to conventional Bayes tests in the very strong sense that the reported frequentist error probabilities equal the posterior probabilities of the hypotheses. These results are herein extended to sequential testing, and yield fully frequentist sequential tests that are considerably easier to use than are conventional sequential tests. Among the interesting properties of these new tests is the lack of dependence of the reported error probabilities on the stopping rule, seeming to lend frequentist support to the stopping rule principle. Keywords:Bayes factor; Composite hypothesis; Conditional test; Error probability; Likelihood ratio; Sequential probability ratio test; Sequential test.

In this article the problem of a sequential test for the model of independent non-identically distributed observations is considered. Based on recursive calculation a new numerical approach to approximate test characteristics for a sequential probability ratio test (SPRT) and a truncated SPRT (TSPRT) is constructed. The problem of robustness evaluation is also studied when the contamination is presented by the distortion of the distributions of all increments of the log-likelihood ratio statistics. The two-side truncated functions are proposed to be used for constructing the robustified SPRT. An algorithm to choose the thresholds of these truncated functions is indicated. The results are applied for a sequential test on parameters of time series with trend. Some kinds of the contaminated models of time series with trend are used to study the robustness of the truncated SPRT. Numerical examples confirming the theoretical results mentioned above are given.

Because computers (except for parallel computers) generate simulation outputs sequentially, we recommend sequential probability ratio tests (SPRTs) for the statistical analysis of these outputs. However, until now simulation analysts have ignored SPRTs. To change this situation, we review SPRTs for the simplest case; namely, the case of choosing between two hypothesized values for the mean simulation output. For this case, the classic SPRT of Wald (Wald A. Sequential tests of statistical hypotheses. Ann Math Stat 1945; 16: 117–186) allows general types of distribution, including normal distributions with known variances. A modification permits unknown variances that are estimated. Hall (Hall WJ. Some sequential analogs of Stein’s two-stage test. Biometrika 1962; 49: 367–378) developed a SPRT that assumes normal distributions with unknown variances estimated from a pilot sample. A modification uses a fully sequential variance estimator. In this paper, we quantify the performance of the various SPRTs, using several Monte Carlo experiments. In experiment #1, simulation outputs are normal. Whereas Wald’s SPRT with estimated variance gives too high error rates, Hall’s original and modified SPRTs are “conservative”; that is, the actual error rates are smaller than those prespecified (nominal). Furthermore, our experiment shows that the most efficient SPRT is Hall’s modified SPRT. In experiment #2, we estimate the robustness of these SPRTs for non-normal output. For these two experiments, we provide details on their design and analysis; these details may also be useful for simulation experiments in general.

A sequential test for correlation coefficients is presented. The method consists of reducing the test of a correlation coefficient to that of a variance ratio and making use of Helmert transformations of sequential observations. The resulting test is identified as the sequential probability ratio test regarding the parameter of the Cauchy distribution. The average sample number of the test is given and it is shown that the proposed sequential test yields an appreciable saving in the required average number of observations compared to the usual fixed sample size test.

: The objective of this paper is to develop acceptance procedures based on a sequential probability ratio test when the system failures follow Weibull probability distribution. The acceptance procedures in both Military Handbook 108 and Mil-Std-781C are based on exponential distribution of failures. Exponential distribution implies a constant hazard rate. The Weibull distribution is much more comprehensive and, with suitable choice of parameters, can describe constant, increasing or decreasing hazard rates.

The bias of maximum likelihood estimates calculated at the end of a sequential procedure is investigated. For the two sequential designs considered in detail, the sequential probability ratio test and the triangular test, this bias is appreciable. A method of calculating an adjusted estimate with reduced bias is described, and an approximation to the standard error of the new estimate is provided. Examples of the implementation of the method are given, and its advantages and disadvantages relative to alternative approaches are discussed.

Early stopping of clinical trials either in the case of beneficial or deleterious effect of treatment on quality of life (QoL) is an important issue. QoL is usually evaluated using self-assessment questionnaires and responses to the items are combined into scores assumed to be normally distributed (which is rarely the case). An alternative is to use item response theory (IRT) models such as the partial credit model (PCM) for polytomous items which takes into account the categorical nature of the items.Sequential analysis and mixed partial credit models were combined in the context of phase II noncomparative trials. The statistical properties of the sequential probability ratio test (SPRT) and of the triangular test (TT) were compared using mixed PCM and traditional average scores methods (ASM) by means of simulations.The type I error of the sequential tests was correctly maintained for both methods, the mixed PCM being more conservative than the ASM. Although remaining a bit underpowered, the mixed PCM displayed higher power than the ASM for both sequential tests. Both methods allowed substantial reductions in average sample numbers as compared with fixed sample designs. Overlapping of item category particularly affected the ASM by inflating the type I error and power. The use of IRT models in sequential analysis of QoL endpoints is promising and should provide a more powerful method to detect therapeutic effects than the traditional ASM.

Sequential sampling is a fast efficient tool for many sampling problems. Sequential sampling may be used (1) to obtain precise estimate(s) of the parameters), or (2) to test hypotheses concerning the parameters. Sequential estimation is used when the purpose of sampling is to obtain precise parameter estimates. Several sequential estimation procedures are discussed in Chapter 4. The focus of this chapter is sequential hypothesis testing. This approach is appropriate when we are interested in determining whether the population density is above or below a stated threshold. As in sequential estimation, sequential hypothesis testing requires taking observations sequentially until some stopping criterion is satisfied. The observations are taken at random over the sampling area. Generally, the accumulated total of the observations relative to the number of observations taken determines when sampling is stopped. The sequential hypothesis testing we consider requires some prior knowledge of the population distribution. This permits most computations to be completed in advance of sampling and to be stored in handheld calculators, laptop computers, or printed on cards or sheets. Wald’s sequential probability ratio test was the earliest sequential test and is described first. Lorden’s 2-SPRT is a more recent development that has some exciting possibilities for tests of hypotheses concerning population density and is discussed in the latter parts of this chapter.

We apply high-dimensional analysis of variance (HANOVA) and sequential probability ratio test (SPRT) to detect buried land mines from array ground penetrating radar (GPR) measurements. The GPR array surveys a region of interest in a progressive manner starting at a known position and moving step by step in a fixed direction. Our detection method consists of two stages. Because, at each stop of the array the path lengths are different from every transmitter/receiver pair to a mien target, there exists statistically significant difference among received signals when a mine target is presented. Thus, the first step in our processing consists of a HANOVA test to detect this statistical difference at each stop. HANOVA does not incorporate new data as the GPR array moves down-track. So secondly, we resort to a sequential probability ratio test to look for changes in the HANOVA statistics as the array proceeds down track. The SPRT allows for real-time processing as anew data are obtained by the GPR array. Finally, real sensor data are processed to verify the method.

The sequential probability ratio test (SPRT) is a fundamental tool for sequential analysis. It forms the basis of numerous sequential techniques for different applications; for example, the truncated SPRT and Page's cumulative sum test (CUSUM). The performance of SPRT is characterized by two important functions—operating characteristic (OC) and average sample number (ASN), and CUSUM's performance is revealed by the average run length (ARL) function. These functions have been studied extensively under the assumption of independent and identically distributed log-likelihood ratios (LLRs) with constant bounds, which is too stringent for many applications. In this article, inductive integral equations governing these functions are developed under very general settings—the bounds can be time-varying and the LLRs are assumed independent but nonstationary. These inductive equations provide a theoretical foundation for performance analysis. Unfortunately, they have nonunique solutions in the general case except for the truncated SPRT. Numerical solutions for some frequently encountered special cases are developed and are compared with the results of Monte Carlo simulations.

In this paper, we will discuss the sequential probability ratio test to test the zero inflation in counting data with the sample size being a random variable with specified Type I and Type II errors. We will introduce the zero-inflated non-central negative binomial distribution, and explain how its operating characteristic and average sample number functions are derived. Simulations are studied to indicate the performances of the proposed tests with different scenarios. A real application is also discussed to explain the process of the sequential probability ratio test.

ABSTRACTWe develop truncated sequential probability ratio test (SPRT) procedures for multivariate normal data. The framework includes a general cost structure and arbitrary mean and covariance structures. The truncated SPRT solutions have a practical and easy-to-use decision boundary representation. In the homogeneous case, a very fast recursive algorithm is presented for calculating the decision boundaries. Misclassification rates and expected sample size are investigated and the results are compared with a nonsequential procedure. A real-life data set on kidney dysfunction following heart surgery is used to illustrate the truncated SPRT procedure.

Because of rare but serious adverse events, pharmaceutical drugs and vaccines are sometimes withdrawn from the market, either by a government agency such as the Food and Drug Administration (FDA) in the United States or by the manufacturing pharmaceutical company. In other cases, a drug may be generally safe but increase the risk for serious adverse events for certain subpopulations such as pregnant women or people with heart problems. Due to limited sample size and selected study populations, rare adverse events are often impossible to detect during phase 3 trials conducted before the drug is approved for general use. It is then important to conduct post-approval drug safety surveillance, using, for example, health insurance claims data. In such surveillance, the goal should be to detect serious adverse events as early as possible without too many false alarms, and it is then natural to use a continuous or near-continuous sequential test procedure that reevaluates the data on a daily or weekly basis. In this article, we first show that Wald's classical sequential probability ratio test (SPRT) for continuous surveillance is very sensitive to the choice of relative risk required in the specification of the alternative hypothesis, making it difficult to use for drug and vaccine safety surveillance. We instead propose the use of a maximized sequential probability ratio test (MaxSPRT) based on a composite alternative hypothesis, which works well across a range of relative risks. We illustrate the use of this method on vaccine safety surveillance and compare it with the classical SPRT. A table of critical values for the MaxSPRT is provided, covering most parameter choices relevant for vaccine and drug safety surveillance. The critical values are based on exact numerical calculations. We also calculate the statistical power, the expected time until the null hypothesis is rejected, and the average length of surveillance.