Hypothesis testing via Euclidean separation

We present a new non-parametric statistic, called the weighed <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> divergence, based on empirical distributions for sequential change detection. We start by constructing the weighed <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> divergence as a fundamental building block for two-sample tests and change detection. The proposed statistic is proved to attain the optimal sample complexity in the offline setting. We then study the sequential change detection using the weighed <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> divergence and characterize the fundamental performance metrics, including the average run length (ARL) and the expected detection delay (EDD). We also present practical algorithms to find the optimal projection to handle high-dimensional data and the optimal weights, which is critical to quick detection since, in such settings, there are not many post-change samples. Simulation results and real data examples are provided to validate the good performance of the proposed method.

The problem of sequential detection of spontaneous changes in equations for unobserved state variables and observations of linear and nonlinear multivariate state-space models is considered. We consider the case of new additive terms and the case of changing coefficients of these equations. The proposed nonparametric method uses the idea of the “moving window” of observations. In Theorem 4.1 the exponential rate of convergence to zero for type 1 error probability (a wrong decision about a change) is proved. In Theorem 4.2 we consider type 2 error probability and the normalized (by the volume N of the moving window) delay time in change-point detection. Asymptotic optimality of the proposed method is proved in Theorem 5.1. Experimental study includes Monte Carlo examples of sequentially detected changes in coefficients of linear and nonlinear state-space models.

The analysis of remote sensing data enables us to detect changes and monitor land surface temperature (LST). However, analysis of times series data poses some challenges, including weather conditions, seasonality and noise, that limit the effectiveness of change detection algorithms. While existing algorithms perform relatively well for detecting abrupt transitions, reliable detection of gradual changes is more difficult. In this paper, we formulate the problem of spatiotemporal LST detection as a statistical sequential change detection problem. LST images are modeled as stochastic processes, with temperature changes reflected as changes in the parameters (i.e., mean) of the process. A generalized likelihood ratio test is used to detect these changes and estimate the exact time/space where they occur. To minimize processing time and memory requirements, we represent LST images by their reduced dimensionality using direct cosine transformation followed by principal component analysis. Statistical sequential analysis is used to provide a unified mathematical framework for the detection of both abrupt and gradual changes in LST observations of Bridgeton Missouri landfill over 17 years.

The problem of joint sequential change detection and isolation in a multichannel system is considered. It is assumed that a disruption occurs at some unknown time, and changes the distributions of the observations in an unknown subset of channels. The problem is to quickly detect the change, and at the same time to reliably isolate the affected channels. A novel scheme is proposed for this task, which admits a recursive structure, is scalable with respect to the number of channels, and does not require any prior information about the change-point. Its performance is analyzed in the special case that the number of affected channels is known. Specifically, explicit critical values are obtained for the control of the false alarm rate and the conditional probability of wrong isolation below arbitrary levels to be prescribed by the practitioner. Finally, the asymptotic optimality of the average detection delay of the proposed scheme is established as the error probabilities go to 0 and the effect of the prior distribution for the change point vanishes in the limit.

This paper considers a statistical signal processing problem involving agent-based models of financial markets, which at a microlevel are driven by socially aware and risk-averse agents. These agents trade (buy or sell) stocks at each trading instant by using the decisions of all previous agents (social learning) in addition to a private (noisy) signal they receive on the value of the stock. We are interested in the following: 1) modelling the dynamics of these risk averse agents and 2) sequential detection of a market shock based on the behaviour of these agents. Structural results that characterize social learning under a risk measure, conditional value-at-risk (CVaR), are presented and formulation of the Bayesian change point detection problem is provided. The structural results exhibit two interesting properties: 1) risk averse agents herd more often than risk neutral agents and 2) the stopping set in the sequential detection problem is nonconvex. The framework is validated on data from the Yahoo! Tech Buzz game dataset and it is revealed that 1) the model identifies the value changes based on agent’s trading decisions. 2) Reasonable quickest detection performance is achieved when the agents are risk-averse.

Sequential change detection in high-dimensional dynamic networks has attracted growing attention in modern applications. A major challenge is that as network scale increases, limited sensing resources make it difficult to fully observe links at each time point, resulting in partial observability and complicating detection. To tackle this, we propose a Latent Bi-Transit Network (LBTN) that learns unobserved edge formation, latent node states, and the evolution mechanisms of dynamic networks. Based on LBTN, we design a variational Bayesian method to infer sparse node-level changes and employ a likelihood ratio test as the detection statistic. By further formulating the statistic as the reward function of a combinatorial multi-armed bandit (CMAB) problem, we develop a Thompson sampling strategy to adaptively select edges for observation, balancing exploration and exploitation. Comprehensive simulations and real-world experiments show that our method consistently outperforms existing baselines and its variants across diverse scenarios.

Sequential change detection is a fundamental problem in statistics and signal processing, with the CUSUM procedure widely used to achieve minimax detection delay under a prescribed false alarm rate when pre- and post-change distributions are fully known. However, in many practical settings, raw observations cannot be shared with a trusted central curator, and privacy must be enforced at the data source, which prevents the computation of exact CUSUM statistics. We therefore introduce a local differentially private (DP) variant called LDP-CUSUM, which first applies a local DP mechanism to transform the raw data into privatized observations and then applies a CUSUM procedure to detect the change. We derive closed-form bounds on the average run length to false alarm and on the worst-case average detection delay, explicitly characterizing the tradeoff among privacy level, false alarm rate, and detection efficiency. Numerical simulations and a real-data case study were conducted to demonstrate the detection efficiency of our proposed LDP-CUSUM across various scenarios.

Sequential Detection of Abrupt Changes in ARMA Models

SummaryA review of methods suggested in the literature for sequential detection of changes in public health surveillance data is presented. Many researchers have noted the need for prospective methods. In recent years there has been an increased interest in both the statistical and the epidemiological literature concerning this type of problem. However, most of the vast literature in public health monitoring deals with retrospective methods, especially spatial methods. Evaluations with respect to the statistical properties of interest for prospective surveillance are rare. The special aspects of prospective statistical surveillance and different ways of evaluating such methods are described. Attention is given to methods that include only the time domain as well as methods for detection where observations have a spatial structure. In the case of surveillance of a change in a Poisson process the likelihood ratio method and the Shiryaev–Roberts method are derived.

Detecting change points sequentially in a streaming setting, especially when both the mean and the variance of the signal can change, is often a challenging task. A key difficulty in this context often involves setting an appropriate detection threshold, which for many standard change statistics may need to be tuned depending on the prechange and postchange distributions. This presents a challenge in a sequential change detection setting when a signal switches between multiple distributions. Unfortunately, change point detection schemes that use the log-likelihood ratio, such as cumulative sum (CUSUM) and the generalized log-likelihood ratio (GLR), are quick to react to changes but are not symmetric when both the mean and the variance of the signal change. This makes it difficult to set a single threshold to detect multiple change points sequentially in a streaming setting. We propose a modified version of CUSUM that we call data-adaptive symmetric CUSUM (DAS-CUSUM). The DAS-CUSUM procedure is symmetric for changes between distributions, making it suitable to set a single threshold to detect multiple change points sequentially in a streaming setting. We provide results that relate the expected detection delay and average run length for our proposed procedure when both prechange and postchange distributions are normally distributed. Experiments on simulated and real-world data show the utility of DAS-CUSUM.

In content-based video analysis, substantial research efforts have been focused on developing techniques to detect the boundaries between two successive shots, each comprising consecutive frames filmed with a single camera act. However, there is still room for further improvement in the detection performance. With the use of sequential change detection and the help of nonparametric density estimation principles, we propose A new shot boundary detection method that can maintain not only satisfactory detection accuracy, but also consistent detection performance based on the results of various test videos.

This paper formulates and solves a sequential detection problem that involves the mutual information (stochastic observability) of a Gaussian process observed in noise with missing measurements. The main result is that the optimal decision is characterized by a monotone policy on the partially ordered set of positive definite covariance matrices. This monotone structure implies that numerically efficient algorithms can be designed to estimate and implement monotone parametrized decision policies.The sequential detection problem is motivated by applications in radar scheduling where the aim is to maintain the mutual information of all targets within a specified bound. We illustrate the problem formulation and performance of monotone parametrized policies via numerical examples in fly-by and persistent-surveillance applications involving a GMTI (Ground Moving Target Indicator) radar.

Detecting emergence of a low-rank signal from high-dimensional data is an important problem arising from many applications such as camera surveillance and swarm monitoring using sensors. We consider a procedure based on the largest eigenvalue of the sample covariance matrix over a sliding window to detect the change. To achieve dimensionality reduction, we present a sketching-based approach for rank change detection using the low-dimensional linear sketches of the original high-dimensional observations. The premise is that when the sketching matrix is a random Gaussian matrix, and the dimension of the sketching vector is sufficiently large, the rank of sample covariance matrix for these sketches equals the rank of the original sample covariance matrix with high probability. Hence, we may be able to detect the low-rank change using sample covariance matrices of the sketches without having to recover the original covariance matrix. We character the performance of the largest eigenvalue statistic in terms of the false-alarm-rate and the expected detection delay, and present an efficient online implementation via subspace tracking.

Research on U-statistics within the general framework of sequential and change-point literature is surveyed. Some recent developments are discussed and extended. New sequential testing strategies based on Wiener process approximations are proposed, and empirical studies explore the finite sample performance of these tests. It allows users to choose one that is appropriate for their application.

A change detection method for sequential patterns