Change Detection Method of Time Series as an Optimal Stopping Problem: Constructive Proof of Optimal Solution Theorem

This paper addresses the change point detection problem in time series. A methodology based on the Metropolis-Hastings algorithm applied to time series modeled as a process with Beta distribution is discussed. In order to make this methodology useful in practice, a fuzzy cluster technique is applied to the initial time series at first, generating a new data set with Beta distribution. Bayesian procedures are considered for inference and the Metropolis-Hastings algorithm is used to sample from the posteriors. In the clustering process, a Kohonen neural network is used having as objective to find the best centers of the time series to be used in the fuzzyfication process. Finally, it will be presented a simulation results in the series of the electric energy consumption in Brazil, between January of 1976 and December of 2000, five months before the blackout occurred in 2001. Such result illustrates the efficiency of the proposed methodology for change point detection in time series.

We study the sequential hypothesis testing and quickest change-point (or disorder) detection problems with linear delay penalty costs for observable Wiener processes under (constantly) delayed detection times. The method of proof consists of the reduction of the associated delayed optimal stopping problems for one-dimensional diffusion processes to the equivalent free-boundary problems and solution of the latter problems by means of the smooth-fit conditions. We derive closed-form expressions for the Bayesian risk functions and optimal stopping boundaries for the associated weighted likelihood ratio processes in the original problems of sequential analysis.

As video content is responsible for more than 70% of the global IP traffic, related resource allocation approaches, e.g., using content caching, become increasingly important. In this context, to avoid under-provisioning, it is important to rapidly detect and respond to changes in content popularity dynamics, including volatility, i.e., changes in the second order moment of the underlying process. In this paper, we focus on the early identification of changes in the variance of video content popularity, which we address as a statistical change point (CP) detection problem. Unlike changes in the mean that can be well captured by non-parametric statistical approaches, to address this more demanding problem, we construct a hypothesis test that uses in the test statistic both parametric and non-parametric approaches. In the context of parametric models, we consider linear, in the form of autoregressive moving average (ARMA), and, nonlinear, in the form of generalized autoregressive conditional heteroskedasticity (GARCH) processes. We propose an integrated algorithm that combines off-line and on-line CP schemes, with the off-line scheme used as a training (learning) phase. The algorithm is first assessed over synthetic data; our analysis demonstrates that non parametric and GARCH model based approaches can better generalize and are better suited for content views time series with unknown statistics. Finally, the non-parametric and the GARCH based variations of our proposed integrated algorithm are applied on real YouTube video content views time series, to illustrate the performance of the proposed approach of volatility change detection.

Early detection of fuel leakage at service stations with underground petroleum storage systems is a crucial task to prevent catastrophic hazards. Current data-driven fuel leakage detection methods employ offline statistical inventory reconciliation, leading to significant detection delays. Consequently, this can result in substantial financial loss and environmental impact on the surrounding community. In this paper, we propose a novel framework called memory-based online change point detection (MOCPD) which operates in near real-time, enabling early detection of fuel leakage. MOCPD maintains a collection of representative historical data within a size-constrained memory, along with an adaptively computed threshold. Leaks are detected when the dissimilarity between the latest data and historical memory exceeds the current threshold. An update phase is incorporated in MOCPD to ensure diversity among historical samples in the memory. With this design, MOCPD is more robust and achieves a better recall rate while maintaining a reasonable precision score. We have conducted a variety of experiments comparing MOCPD to commonly used online change point detection (CPD) baselines on real-world fuel variance data with induced leakages, actual fuel leakage data and benchmark CPD datasets. Overall, MOCPD consistently outperforms the baseline methods in terms of detection accuracy, demonstrating its applicability to fuel leakage detection and CPD problems.

Wikipedia has been turned into an immensely popular crowd-sourced encyclopedia for information dissemination on numerous versatile topics in the form of subscription free content. It allows anyone to contribute so that the articles remain comprehensive and updated. For enrichment of content without compromising standards, the Wikipedia community enumerates a detailed set of guidelines, which should be followed. Based on these, articles are categorized into several quality classes by the Wikipedia editors with increasing adherence to guidelines. This quality assessment task by editors is laborious as well as demands platform expertise. As a first objective, in this paper, we study evolution of a Wikipedia article with respect to such quality scales. Our results show novel non-intuitive patterns emerging from this exploration. As a second objective we attempt to develop an automated data driven approach for the detection of the early signals influencing the quality change of articles. We posit this as a change point detection problem whereby we represent an article as a time series of consecutive revisions and encode every revision by a set of intuitive features. Finally, various change point detection algorithms are used to efficiently and accurately detect the future change points. We also perform various ablation studies to understand which group of features are most effective in identifying the change points. To the best of our knowledge, this is the first work that rigorously explores English Wikipedia article quality life cycle from the perspective of quality indicators and provides a novel unsupervised page level approach to detect quality switch, which can help in automatic content monitoring in Wikipedia thus contributing significantly to the CSCW community.

By detecting the changing trend, Change point detection (CPD) can describe the underlying behavior of the system with time series (e.g., equipment failure detection, auxiliary medical diagnosis, and climate change detection, etc.). However, in the current big data environment, how quickly obtaining time-series data dependencies and accurately detecting change points is still a challenge. We propose Sailfish, an unsupervised change point detection framework based on the Gaussian process for time series data. In comparison with existing CPD designs, Sailfish has two novel features: 1) using the deep Gaussian process as a hidden variable transformer and 2) integrating the Gaussian process into a bi-LSTM cell for capturing past and future embedded feature trends. Our extensive experiments show that the Sailfish significantly outperforms five state-of-the-art CPD methods with a faster speed and higher accuracy on three public real-time-series datasets. Especially on the large volume of datasets, Sailfish can achieve up to a 60% reduction in training time and 13% F1-score improvement with little overhead compared to the state-of-the-arts.KeywordsChange point detectionGaussian processBi-Long short term memoryBayesian neural network

Change points are abrupt variations in time series data. Such abrupt changes may represent transitions that occur between states. Detection of change points is useful in modelling and prediction of time series and is found in application areas such as medical condition monitoring, climate change detection, speech and image analysis, and human activity analysis. This survey article enumerates, categorizes, and compares many of the methods that have been proposed to detect change points in time series. The methods examined include both supervised and unsupervised algorithms that have been introduced and evaluated. We introduce several criteria to compare the algorithms. Finally, we present some grand challenges for the community to consider.

It is very often the case that at some moment a time series process abruptly changes its underlying structure and, therefore, it is very important to accurately detect such change-points. In this problem, which is called a change-point (or break-point) detection problem, we need to find a method that divides the original nonstationary time series into a piecewise stationary segments. In this paper, we develop a flexible method to estimate the unknown number and the locations of change-points in autoregressive time series. In order to find the optimal value of a performance function, which is based on the Minimum Description Length principle, we develop a Cross-Entropy algorithm for the combinatorial optimization problem. Our numerical experiments show that the proposed approach is very efficient in detecting multiple change-points when the underlying process has moderate to substantial variations in the mean and the autocorrelation coefficient. We also apply the proposed method to real data of daily AUD/CNY exchange rate series from 2 January 2018 to 24 March 2020.

Change-point of multiple biomarkers in women with ovarian cancer

High-dimensional changepoint analysis is a growing area of research and has applications in a wide range of fields. The aim is to accurately and efficiently detect changepoints in time series data when both the number of time points and dimensions grow large. Existing methods typically aggregate or project the data to a smaller number of dimensions, usually one. We present a high-dimensional changepoint detection method that takes inspiration from geometry to map a high-dimensional time series to two dimensions. We show theoretically and through simulation that if the input series is Gaussian, then the mappings preserve the Gaussianity of the data. Applying univariate changepoint detection methods to both mapped series allows the detection of changepoints that correspond to changes in the mean and variance of the original time series. We demonstrate that this approach outperforms the current state-of-the-art multivariate changepoint methods in terms of accuracy of detected changepoints and computational efficiency. We conclude with applications from genetics and finance.

In this paper, the problem of bridge health monitoring is formulated as an unsupervised semantic segmentation where the aim is to identify the onset of structural damage by detecting unexpected irregularities in the time series data. In particular, the main objective of this study is to approach this problem through an indirect sensing framework based on the concept of vehicle-mounted scanning. Thus, the sole source of information is the acceleration responses of the moving vehicle passing over the bridge. This is achieved through automatic feature learning by adopting the concept of matrix profile to conduct an all-pair-similarity search for a given time series, where the location of the contextual change is determined by the corresponding Corrected Arc Crossings (CAC) profile. The performance of the method is first evaluated through numerical investigations for different damage cases where the effect of operational uncertainties such as road roughness, and variability in vehicle properties are considered. Further, the method is experimentally evaluated using a lab-scale bridge model with different damage cases. Through these investigations, it is consistently observed that the location of the change point always coincides with the minimum point of the CAC profile. The proposed framework has three major advantages making it more appealing compared to the competing techniques. First, it has been fully formulated in the context of unsupervised learning where no access to labeled data is required. Second, it is capable of working with a limited amount of data yet providing promising results. Finally, it can be easily applied to online settings where data is streaming in real-time. These successful results demonstrate the potential of the proposed method as a cost-effective and rigorous bridge inspection tool and open up opportunities in the field of indirect bridge health monitoring.

We consider a optimal dividend and stopping problem for two-dimensional compound Poisson risk model. The two companies allow to help each other by transffering payments. The goal of the companies is to maximize the expected cumulative discounted dividends payments up to the time of ruin. The problem is formulated as a singular control and optimal stopping problem. We explicitly construct the value function and the optimal strategy when claims are exponentially distributed and present two numerical examples.

Capturing correlation changes by applying kernel change point detection on the running correlations

Evaluation and comparison of efficiencies of widely used objective homogenisation methods (OHOMs) are presented relying on some test-datasets and efficiency measures. Problems related to the choice of efficiency measure, creation of appropriate test-datasets and use of OHOM parameterisation are discussed. The detection parts of the OHOMs are examined only. Power of detection, false alarm rate, detection skill and skill of linear trend estimation are calculated and compared for eight OHOMs and six test-datasets. Each test-dataset comprises 10,000 100 year-long artificially simulated time series. In the simplest test dataset, each time series contains one inhomogeneity (IH), while a structure of inhomogeneities that is similar to that of real central European temperature time series is included in the most complex simulated dataset. Distinct attention is given to OHOMs that contain (1) cutting algorithm, (2) semihierarchic algorithm, (3) direct detection of multiple IHs, (4) detection of change-point and trend-line shaped IHs. Results show that Caussinus–Mestre method and Multiple Analysis of Series for Homogenization are the most powerful tools in detecting and correcting IHs in climatic time series.

MotivationConventional identification methods for gene regulatory networks (GRNs) have overwhelmingly adopted static topology models, which remains unchanged over time to represent the underlying molecular interactions of a biological system. However, GRNs are dynamic in response to physiological and environmental changes. Although there is a rich literature in modeling static or temporally invariant networks, how to systematically recover these temporally changing networks remains a major and significant pressing challenge. The purpose of this study is to suggest a two-step strategy that recovers time-varying GRNs.ResultsIt is suggested in this paper to utilize a switching auto-regressive model to describe the dynamics of time-varying GRNs, and a two-step strategy is proposed to recover the structure of time-varying GRNs. In the first step, the change points are detected by a Kalman-filter based method. The observed time series are divided into several segments using these detection results; and each time series segment belonging to two successive demarcating change points is associated with an individual static regulatory network. In the second step, conditional network structure identification methods are used to reconstruct the topology for each time interval. This two-step strategy efficiently decouples the change point detection problem and the topology inference problem. Simulation results show that the proposed strategy can detect the change points precisely and recover each individual topology structure effectively. Moreover, computation results with the developmental data of Drosophila Melanogaster show that the proposed change point detection procedure is also able to work effectively in real world applications and the change point estimation accuracy exceeds other existing approaches, which means the suggested strategy may also be helpful in solving actual GRN reconstruction problem.