Robust probabilistic predictable feature analysis and its application for dynamic process monitoring

Fault detection and fault identification are consecutive steps of multivariate statistical process monitoring. In recent years, increasing attention has been paid to process dynamics. For dynamic process modeling, canonical variate analysis (CVA) extracts process dynamics effectively. However, process noises are not well analyzed in traditional CVA and corresponding fault identification methods are less studied. To solve these issues, a variational Bayesian CVA (VBCVA) model is proposed for dynamic process monitoring. Through a probabilistic perspective, the inevitable noises in realistic industrial processes can be captured in the new model. Moreover, the proposed model is further extended in the variational Bayesian framework to overcome the common problems in probabilistic methods. Besides, an improved fault identification approach based on fault relevance is introduced, which avoids the smearing effect caused by data reconstruction. Finally, the feasibility of the proposed process monitoring scheme is verified on the TE benchmark and a real wastewater treatment process.

Collision Probability with Gaussian Mixture Orbit Uncertainty

Physics-informed probabilistic slow feature analysis

The origin of Gaussian distributions of synaptic potentials

Robust probabilistic principal component analysis for process modeling subject to scaled mixture Gaussian noise

A Botnet is a malicious network that can be leveraged in a wide range of cyber-attacks with possibly catastrophic results. Consequently, botnet detection is a high priority. In the past few years, many studies have been conducted regarding botnet detection of which the proposed approaches focus heavily on machine learning techniques. In this paper, a clustering approach based on Gaussian mixture distributions is proposed that allows accommodating ellipsoidal shaped clusters. Each cluster is associated with a Gaussian distribution by which an overall mixture of Gaussians is obtained. Hereby, an optimization problem is formalized ending up with a cost function based on the entropy of the overall distributions. The optimal solution is determined by using an optimization method. The proposed approach is then applied to bot-generated network traffic to extract models on bots behavior which can thus be leveraged to detect bot-infected hosts. The quality evaluation is performed by using some commonly used criteria that measure the accuracy and detection capability of models generated by the algorithm. The results show that high quality can be achieved by our method.

본 논문은 신호와 잡음이 혼합된 관측치로부터 신호 관측치를 분류하는 문제를 다룬다. 잡음은 가우시안 분포를 따르고 신호는 감마 분포를 따른다고 가정할 때 관측치의 분포는 가우시안과 감마의 혼합 분포를 따르게 된다. EM 알고리즘을 통해 혼합 모델의 모수를 추정하고 신호 및 잡음을 분류하는 것을 다중 가설 검정으로 간주하여 베이즈 오류를 바탕으로 분류를 위한 경계치를 설정한다. 제안하는 방법을 분광 데이터에 근거하여 철강 제품에서 개재물 유무를 검출하는 문제에 적용하였고 별도의 시뮬레이션 데이터를 통해 성능의 우수성을 보였다. A problem of separating signals from noises is considered, when they are randomly mixed in the observation. It is assumed that the noise follows a Gaussian distribution and the signal follows a Gamma distribution, thus the underlying distribution of an observation will be a mixture of Gaussian and Gamma distributions. The parameters of the mixture model will be estimated from the EM algorithm. Then the signals and noises will be classified by a fixed threshold approach based on multiple testing using positive false discovery rate and Bayes error. The proposed method is applied to a real optical emission spectroscopy data for the quantitative analysis of inclusions. A simulation is carried out to compare the performance with the existing method using 3 sigma rule.

Gaussian mixtures (i.e. linear combinations of multivariate Gaussian probability densities) appear in numerous applications due to their universal ability to approximate multimodal probability distributions. Finding the modes (maxima) of a Gaussian mixture is a fundamental problem arising in many practical applications such as machine learning and digital image processing. In this paper, we propose a computationally efficient method for finding a significant mode of the Gaussian mixture. Such a mode represents an area of large probability, and it often coincides with the global mode of the mixture. The proposed method uses a Gaussian convolution in order to remove undesired local maxima of the Gaussian mixture and preserve its underlying structure. The transformation between the maximizers of the smoothed Gaussian mixture and the original one is formulated as a differential equation. A robust trust region method for tracing the solution curve of this differential equation is described. Our formulation also allows mixtures with negative weights or even negative values, which occur in some applications related to machine learning or quantum mechanics. The applicability of the method to mode-finding of Gaussian kernel density estimates obtained from experimental data is illustrated. Finally, some numerical results are given to demonstrate the ability of the method to find significant modes of Gaussian mixtures and kernel density estimates.

Improving the efficiency of photovoltaic (PV) systems has gained priority in current research due to the large volumes of PV panels installed. Moreover, the remarkable efforts made to investigate different methods of diagnosing PV failures have multiplied, giving additional impetus to research on the efficiency of PV systems. However, most of these methods are limited in the number of faults that can be identified; some are expensive and complex, and others require huge amounts of data to train. In this paper, a simple and robust multivariate statistical analysis method is proposed for the diagnosis and identification of faults in a PV system. From the multitude of data that can be collected on a PV system in operation or available on most new inverters on the market, the method used here is based on kernel principal component analysis, which looks at and analyses the variance between these data. Combined with the Hotelling statistic ( T 2 ) and Squared Prediction Error (SPE) index associated with Rolle’s theorem, this analysis identifies six operating states of the PV system: normal operation, short-circuited panels, open-circuit panels, partially shaded panels, serial resistance degradation, and MPPT error. 200 samples of different faults at different irradiance have been generated between an irradiance from 50 to 500 w/m2. The accuracy rate was 64% for the degradation of series resistance, 91.11% for partial shading, 100% for open circuit, 100% for short circuit, and 100% for MPPT error. The various results obtained first from a Matlab-Simulink model, and then from a real system of 18 modules, demonstrate the efficiency and performance of the proposed algorithms.

The accurate modeling of uncertain variables, such as wind power and load uncertainty, is an essential foundation for the planning operation and scheduling decisions in power systems. Currently, Gaussian mixture model (GMM) and Dirichlet process mixture model (DPMM) are the primary techniques used to characterize uncertainties in power systems, which are commonly solved by expectation-maximization (EM) algorithm. However, for the massive data of uncertain variables, the algorithm encounters challenges in accurately obtaining GMM and DPMM with a lower time consumption. To address this issue, we propose a method for GMM uncertainty modeling in power systems considering the mutual assistance of latent variables. Specifically, the GMM of uncertain variables is first constructed, and the conditional probability is employed to characterize the mutual assistance of latent variables. Then, an improved EM algorithm is developed to obtain the optimal GMM parameters, in which the expectation step (E-step) and maximization step (M-step) of the algorithm are revised using the conditional probability. Importantly, the closed-form solutions for GMM parameters are rederived in the revised E-step and M-step. Finally, the proposed uncertainty modeling method is compared with the traditional GMM and DPMM on actual wind power and load data from Australia. The proposed method performs the efficiency and accuracy in characterizing uncertainty.

Dynamic is another critical behaviour of the process data, which has also caught much attention in process monitoring. Based on the previous works on dynamic process monitoring in the past years, this chapter introduces two different methods. These approaches rely on (i) recent work on independent component analysis and support vector data description that is applied to a dynamic data structure and (ii) the incorporation of the statistical local approach into a dynamic data representation. The analysis of experimental data from a gearbox system confirms (i) significant auto- and cross-correlation within and among these signals and (ii) that they cannot be assumed to follow Gaussian distributions. The application of both approaches showed that they are more sensitive to incipient faults than conventional multivariate statistical methods.

Underwater acoustic (UWA) communication in shallow water areas is often affected by the impulsive components of ambient noise. The noise statistics in such cases does not follow the traditional Gaussian behavior. On the other hand, most communication receivers are designed with Gaussian noise assumption. Performance of such systems may not be optimum in the presence of non-Gaussian noise in a UWA channel, and, so, the scenario demands an investigation. Among the several non-Gaussian noise models proposed in various literature, the Gaussian Mixture (GM) distribution is popular due to its "universal approximation" properties. However, despite its popularity in modeling UWA noise, none has yet reported any analytical analysis of error probability assuming GM noise statistics in the UWA channel, to the best of our knowledge. In this paper, we present a study on the error performance of a UWA communication system considering channel noise to be a mixture of Gaussian components. We derive the analytical expressions of probability of error for BPSK, QPSK and M-PAM constellations. We also simulate a UWA system where the performance of a traditional Gaussian receiver is studied in the presence of GM noise. It has been observed that in the presence of GM noise, the system performance degrades significantly from the usual one expected in a Gaussian noise environment and becomes a function of the mixing coefficients of the GM distribution.

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

The low-rank tensor factorization (LRTF) technique has received increasing attention in many computer vision applications. Compared with the traditional matrix factorization technique, it can better preserve the intrinsic structure information and thus has a better low-dimensional subspace recovery performance. Basically, the desired low-rank tensor is recovered by minimizing the least square loss between the input data and its factorized representation. Since the least square loss is most optimal when the noise follows a Gaussian distribution, -norm-based methods are designed to deal with outliers. Unfortunately, they may lose their effectiveness when dealing with real data, which are often contaminated by complex noise. In this paper, we consider integrating the noise modeling technique into a generalized weighted LRTF (GWLRTF) procedure. This procedure treats the original issue as an LRTF problem and models the noise using a mixture of Gaussians (MoG), a procedure called MoG GWLRTF. To extend the applicability of the model, two typical tensor factorization operations, i.e., CANDECOMP/PARAFAC factorization and Tucker factorization, are incorporated into the LRTF procedure. Its parameters are updated under the expectation-maximization framework. Extensive experiments indicate the respective advantages of these two versions of MoG GWLRTF in various applications and also demonstrate their effectiveness compared with other competing methods.

Researchers have traditionally employed Gaussian distributions to model quantitative biological traits. Recently, mixtures of Gaussian distributions have begun to be used as well. However, there are many alternatives to the Gaussian distribution. From a theoretical perspective, the lognormal distribution is as applicable as the Gaussian (both are justified on the basis of the Central Limit Theorem). Here, the utility of mixtures of Gaussians and lognormals for describing birthweight and gestational age distributions are compared. This is carried out within the context of the hybrid-lognormal distribution, in which the Gaussian and lognormal are special cases. The data consists of African American births (1985-1988) and European American births (1988) in the state of New York. The results suggest that of the conventional distributions, a mixture of two Gaussians generally provides the best fit to birthweight and gestational age. However, in the case of birthweight a two-component hybrid-lognormal fits better than any of the simpler models. This may be due to a feature of the hybrid-lognormal distribution that can be interpreted as maternal constraints on fetal development.