Selection using “play-the-winner” sampling for non-negative observations

The cycle time-throughput curve is one of the most important analytical tools used to assess operating policies in manufacturing systems. Unfortunately, the generation of this curve is complicated and time consuming when generation is based upon extensive simulation analysis. This paper presents a simulation-based, fixed sample size strategy for generating a cycle time-throughput curve with minimal mean square error that mitigates the typical problems associated with a simulation-based cycle time-throughput curve. The strategy comprises two components, the sampling method and sampling weights. A queuing network of five workstations in series is used for validation of the approach. Results indicate that the sampling method using antithetic variates is effective in reducing the variance as well as bias of a cycle time-throughput curve. Furthermore, this method is robust to the sample size. Given a sufficiently large sample, the combination of common random numbers and antithetic variates is preferred. A reduction in the sample size and complexity of the system increases the significance of the sampling weights.

We deal with the design problem of early phase dose-finding clinical trials with monotone biologic endpoints, such as biological measurements, laboratory values of serum level, and gene expression. A specific objective of this type of trial is to identify the minimum dose that exhibits adequate drug activity and shifts the mean of the endpoint from a zero dose to the so-called minimum effective dose. Stepwise test procedures for dose finding have been well studied in the context of nonhuman studies where the sampling plan is done in one stage. In this article, we extend the notion of stepwise testing to a two-stage enrollment plan in an attempt to reduce the potential sample size requirement by shutting down unpromising doses in a futility interim. In particular, we examine four two-stage designs and apply them to design a statin trial with four doses and a placebo in patients with Hodgkin's disease. We discuss the calibration of the design parameters and the implementation of these proposed methods. In the context of the statin trial, a calibrated two-stage design can reduce the average total sample size up to 38% (from 125 to 78) from a one-stage step-down test, while maintaining comparable error rates and probability of correct selection. The price for the reduction in the average sample size is the slight increase in the maximum total sample size from 125 to 130.

Ventilator-associated pneumonia (VAP) is an important healthcare acquired infection, which is associated with high morbidity and mortality. Conducting conventional randomised controlled trials (RCTs) on VAP prevention is often challenging, due to low numbers of eligible patients and events per site, especially for pathogen-specific interventions. We explored how group sequential designs (GSD) and sample size re-estimation (SSR) trial designs could improve RCT efficiency in simulated superiority trials to prevent VAP. Simulations were informed using data from the prospective observational Hospital Network Study - Preparation for a Randomised Evaluation of anti-Pneumonia Strategies (HONEST-PREPS). We tested the impact of different GSD and SSR designs on expected sample size (considering early stopping) and maximum sample size (no early stopping). We varied the type of stopping boundary, number and timepoint of interim analyses, and assumed and true prevention effect. We applied time-to-event analyses, with effect estimates expressed as hazard ratios, for the primary endpoint. The estimated 28-day cumulative incidence of VAP in HONEST-PREPS was 15.5%. For a 30% reduction in VAP (hazard ratio of 0.68), a standard RCT (power 80%) would require a sample size of 1291 patients. For GSD, Pocock boundariesresult in a smaller expected sample size (E[N] = 1128), but a larger maximum sample size (max(N) = 1578) than O'Brien Fleming boundaries (E[N] = 1170 and max(N) = 1389), when utilising the optimal placement of a single interim analysis, 48% and 64% of the maximum number of events for Pocock and O'Brien Fleming boundaries, respectively. SSR is more efficient compared to GSD when the incorrect prevention effect is initially used to plan the trial, as it maintains a power closer to the pre-specified desired power without substantial impact on the expected sample size. GSD and SSR are effective adaptive designs, preferable to fixed RCTs in a superiority trial comparing the effectiveness of an investigational intervention with a standard of care in preventing VAP among critically ill, ventilated patients. They can reduce the expected sample size between 9% and 12% and should be considered at the trial design stage.

The limitations of traditional indifference-zone (IZ) ranking and selection procedures stem from their reliance on an IZ parameter, which is often unknown in practice and can lead to inefficiencies. Additionally, since exact evaluation of the probability of correct selection (PCS) is generally unattainable, these approaches commonly rely on a Bonferroni-Inequality-type lower bound to ensure the worst-case PCS, which introduces further conservativeness. To overcome these challenges, we propose fully sequential IZ-free procedures that eliminate the need for an IZ parameter and reduce the conservativeness associated with Bonferroni Inequality approaches. We demonstrate that, in the context of IZ-free procedures, the screening statistic corresponds to the Euclidean distance of a multi-dimensional standard Brownian motion, which behaves as a Bessel process. This connection simplifies the derivation of statistical guarantees for the proposed procedures and facilitates the analytical expressions for key parameters. Our procedures accommodate both known equal variances and unknown unequal variances, with statistical guarantees established in the known equal variance case. Numerical experiments confirm that the proposed procedures outperform existing methods in reducing PCS conservativeness and improving sample size efficiency. These advancements enhance the robustness and practicality of IZ-free procedures for diverse applications.

In this paper, we introduce a sequential procedure for the Multi-Objective Ranking and Selection (MOR&S) problems that identifies the Pareto front with a guaranteed probability of correct selection (PCS). In particular, the proposed procedure is fully sequential using the test statistics built upon the generalized sequential probability ratio test (GSPRT). The main features of the new proposed procedure are: 1) a unified framework, the new procedure treats the multi-objective problems in the same way as the single-objective problems; 2) an indifference-zone-free formulation, the new procedure eliminates the necessity of indifference-zone parameter; 3) asymptotically optimality, the GSPRT achieves asymptotically the shortest expected sample size among all sequential tests; 4) general distribution, the procedure uses the empirical likelihood for generally distributed observation. A numerical evaluation demonstrates the efficiency of the new procedure.

Selection of planting locations is an important issue in plant breeding and may significantly impact the probability of correctly identifying the desired genotype based on noisy data. This is especially true for early‐stage experimentation where a set of experimental genotypes is observed in very few locations. This study aims to identify locations that are most helpful for discriminating the main effects of two or more genotypes. A regression‐based model is proposed that characterizes the sensitivity of location to the genotype main effect, and locations with significant positive sensitivity are termed discriminative. Comparing or ranking genotypes using these locations results in exaggerated expected differences between genotype main effects, allowing for more accurate comparisons in the presence of noise. Based on the proposed model, an expression for the probability of making correct genotype selection is derived given a set of locations with known location sensitivities. The probability of correct selection is then used to characterize an optimal set of location that maximizes the probability of selecting the genotype with better main effect. These locations are independent of the genotypes being compared and only depend on the number of locations being considered and the sensitivities of these locations to genotype effects. The theoretical implications of the model are validated using data obtained from a commercial soybean breeding program. The results show that by using discriminating locations correct selection is made 76% of the time, versus 67% for all locations and only 63% of the time for non‐discriminating locations.

In this paper, our aim is to compare the scale parameters of two independent Pareto distributions using the hypothesis testing approach. We develop testing procedures for the ratio of scale parameters of two independent Pareto populations with unknown but proportional shape parameters and establish that such a comparison is not possible with any fixed sample size procedure. Hence, we implement a practical accelerated sequential sampling design to obtain the required sample sizes while controlling both type I and type II error probabilities. The first and second-order asymptotic properties of the proposed procedure are studied in detail. A large-scale simulation analysis is carried out to validate the theoretical findings under different parameter configurations. In addition to the extensive simulations, we provide two real-data illustrations (incomes of two occupations and rice production of two regions) to complement the practical utility of the proposed study.

Confirmatory clinical trials comparing the efficacy of a new treatment with an active control typically aim at demonstrating either superiority or non-inferiority. In the latter case, the objective is to show that the experimental treatment is not worse than the active control by more than a pre-specified non-inferiority margin. We consider two classes of group-sequential designs that combine the superiority and non-inferiority objectives: non-adaptive designs with fixed group sizes and adaptive designs where future group sizes may be based on the observed treatment effect. For both classes, we derive group-sequential designs meeting error probability constraints that have the lowest possible expected sample size averaged over a set of values of the treatment effect. These optimized designs provide an efficient means of reducing expected sample size under a range of treatment effects, even when the separate objectives of proving superiority and non-inferiority would require quite different fixed sample sizes. We also present error spending versions of group-sequential designs that are easily implementable and can handle unpredictable group sizes or information levels. We find the adaptive choice of group sizes to yield some modest efficiency gains; alternatively, expected sample size may be reduced by adding another interim analysis to a non-adaptive group-sequential design.

Assuming that there is a linear relationship between the parameters of a two-parameter exponential distribution, the distribution reduces to the one with known coefficient of variation. The problem of testing the scale parameter is considered using fixed sample and sequential testing procedures. A comparison of the two procedures shows that the difference between the fixed sample sizes and the expected sample sizes in the null case is remarkable. Therefore, a truncated test is proposed and its expected sample sizes in the null case are compared with those of the sequential test.

Subset selection of best simulated systems

The problem of selecting the Bernoulli population which has the highest success probability is considered. It has been noted in several articles that the probability of a correct selection is the same, uniformly in the Bernoulli p-vector (P1,P2,….,Pk), for two or more different selection procedures. We give a general theorem which explains this phenomenon. An application of particular interest arises when curtailment of a single-stage procedure (as introduced by Bechhofer and Kulkarni (1982a) )is employed; the corresponding result for curtailment of a single-stage procedure needs no proof. The use of strong curtailment in place of weak curtailment requires no more (and usually many less) observations to achieve the same.

Convergence rate analysis for optimal computing budget allocation algorithms

In recent years, with the rapid development of computing technology, developing parallel procedures to solve large-scale ranking and selection (R&S) problems has attracted a lot of research attention. In this paper, we take fixed-budget R&S procedure as an example to investigate potential issues of developing parallel procedures. We argue that to measure the performance of a fixed-budget R&S procedure in solving large-scale problems, it is important to quantify the minimal growth rate of the total sampling budget such that as the number of alternatives increases, the probability of correct selection (PCS) would not decrease to zero. We call such a growth rate of the total sampling budget the rate for maintaining correct selection (RMCS). We show that a tight lower bound for the RMCS of a broad class of existing fixed-budget procedures is in the order of [Formula: see text], where k is the number of alternatives. Then, we propose a new type of fixed-budget procedure, namely the fixed-budget knockout-tournament ([Formula: see text]) procedure. We prove that, in terms of the RMCS, our procedure outperforms existing fixed-budget procedures and achieves the optimal order, that is, the order of k. Moreover, we demonstrate that our procedure can be easily implemented in parallel computing environments with almost no nonparallelizable calculations. Last, a comprehensive numerical study shows that our procedure is indeed suitable for solving large-scale problems in parallel computing environments. History: Accepted by Bruno Tuffin, Area Editor for Simulation. Funding: Y. Zhong was supported by the National Natural Science Foundation of China [Grant 72101047]. L. J. Hong was supported by the National Natural Science Foundation of China [Grants 72091211 and 72161160340]. G. Jiang was supported by the National Natural Science Foundation of China [Grants 72121001 and 72171060]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/ijoc.2022.1221 .

In this article, two-stage and fixed sample size procedures are developed for constructing confidence intervals for the common correlation ρ of an equi-correlated multivariate normal distribution. Two different approaches for estimation are considered, namely, fixed-width and bounded-width interval estimation. In the fixed-width confidence interval estimation problem, a two-stage procedure is developed and the exact distribution of the corresponding stopping variable and the exact coverage probability of the interval estimator are derived. Asymptotic optimality properties such as asymptotic first-order efficiency and asymptotic consistency properties of the two-stage procedure are established. Bounded-width confidence intervals are obtained by applying fixed-accuracy estimation methodologies of Mukhopadhyay and Banerjee (2014, 2015a,b) and Banerjee and Mukhopadhyay (2016) using different transformations on ρ. Both fixed sample size and two-stage sampling methodologies for bounded-width confidence interval estimation of are developed incorporating such transformations. Finally, performances of all the procedures are compared via extensive sets of simulation studies.

Let x 1, x 2, … x n, … be a sequence of independent random variables with a common density function P(x = 1) = p, P(x = 0) = 1 – p, 0 < p < 1. This paper considers the non-randomized sequential procedures δ's for estimating p and the following three problems on choice of δ. (i) Choose δ to minimize E v (Z δ – P)2 subject to E p N δ ≤ m where m ≥ 1 and Z δ is an unbiased estimate of p; (ii) Choose δ to minimize Ep N δ subject to E p (Z δ – p)2 ≤ α where α is a real positive number; (iii) choose δ to minimize C Ep N δ + E p (Z δ – p)2 where C is the cost of an observation. In each case the minimization is to be done uniformly in p if possible; otherwise the supremum over p of the risk in question is to be minimized. A procedure is constructed for problem (i) when m is not an integer. A fixed sample size procedure is shown to be admissible and minimax for problem (ii). A procedure is constructed which is asymptotically uniformly better than the fixed sample size for problem (ii). Furthermore, for problem (iii) some optimum procedures are constructed.