Robust Generative Restricted Kernel Machines Using Weighted Conjugate Feature Duality

The minimum covariance determinant (MCD) estimator is a highly robust estimator of multivariate location and scatter. It can be computed efficiently with the FAST‐MCD algorithm of Rousseeuw and Van Driessen. Since estimating the covariance matrix is the cornerstone of many multivariate statistical methods, the MCD has also been used to develop robust and computationally efficient multivariate techniques.In this paper, we review the MCD estimator, along with its main properties such as affine equivariance, breakdown value, and influence function. We discuss its computation, and list applications and extensions of the MCD in theoretical and applied multivariate statistics. Copyright © 2009 John Wiley & Sons, Inc.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods

Aims: The aim of this study is to investigate the impact of thresholds on the detection of outliers by comparing the performance of two estimators, namely the minimum covariance determinant (MCD) and minimum regularized covariance determinant (MRCD), at different sample sizes. The study uses simulated data generated from the standard normal distribution to assess how varying thresholds affect the ability of these estimators to detect outliers.
 Study Design: This study employs a quantitative research design. It involves the generation of simulated data, the application of the MCD and MRCD estimators for outlier detection, and the systematic manipulation of thresholds and sample size as independent variables.
 Place and Duration: The study is conducted using computational tools and did not require a physical location.
 Methodology: Simulated data is generated from the standard normal distribution to create a controlled environment for outlier detection experiments. The MCD and MRCD estimators are applied to the simulated data to detect outliers. These estimators are sensitive to deviations from the norm in the data. Different thresholds are systematically applied to the data, and the performance of the estimators is assessed at each threshold level. Thresholds may vary in their extremeness. The study investigates the impact of different sample sizes on outlier detection. This involves using datasets with varying numbers of observations. The r programming language and associated packages are used as the statistical tool for data generation, analysis, and visualization.
 Results: The study's findings indicate that the choice of thresholds in data analysis significantly affects the performance of the MCD and MRCD estimators in outlier detection. If the thresholds used for both estimators are the same, their performance is similar. However, differences emerge when thresholds differ from each other. Higher thresholds are shown to identify less extreme outliers, while lower thresholds are effective at identifying more extreme outliers. These results provide insights into the behavior of these estimators in outlier detection scenarios, shedding light on their sensitivity to threshold choices and sample size.Conclusion: Our study has shed light on the critical interdependencies among threshold choices, sample sizes, and the performance of the minimum covariance determinant (MCD) and minimum regularized covariance determinant (MRCD) estimators in the context of outlier detection. By conducting a systematic exploration in a controlled environment with simulated data, we have gleaned valuable insights that can inform both researchers and practitioners in the field of organizational science research.

The Minimum Covariance Determinant (MCD) estimator proposed by Rousseeuw (1985) is a highly robust way to estimate location and scatter of multivariate and possibly contaminated data sets. Recently Rousseeuw and Van Driessen (1999) have proposed a fast algorithm which allows practical application of MCD even in very large and contaminated data sets. As an initial estimate of location and scatter, MCD could be a better alternative than the so-called minimum volume ellipsoid (MVE) estimator proposed by Rousseeuw (1985), because MCD is asymptotically normal and MVE has a lower convergence rate. Robust distances using MVE have been used for labeling multiple outliers, as it was suggested by Rousseeuw and van Zomeren (1990). For small - sample cases Rousseeuw and van Zomeren (1991) computed by simulation a correction factor in terms to be able to compare MVE robust distances against distribution quantiles. In accordance with its superior theoretical performance, MCD estimate might be used for the same purpose; however it also requires a correction factor for small samples. We present in this paper the results from a simulation study for small - sample cases, with two objectives: first, when both types of estimators are used to build tolerance regions, we compare them in relation to their actual coverage probabilities; and second, to compute required correction factors. We followed the methodology from Rousseeuw and van Zomeren (1991).

The feasible solution algorithm for the minimum covariance determinant estimator in multivariate data

This contribution gives a brief summary of robust estimators of multivariate location and scatter. We assume that the original (uncontaminated) data follow an elliptical distribution with location vector μ and positive definite scatter matrix Σ. Robust methods aim to estimate μ and Σ even though the data has been contaminated by outliers. The well-known multivariate M-estimators can break down when the outlier fraction exceeds 1/(p+1) where p is the number of variables. We describe several robust estimators that can withstand a high fraction (up to 50 %) of outliers, such as the minimum covariance determinant estimator (MCD), the Stahel–Donoho estimator, S-estimators and MM-estimators. We also discuss faster methods that are only approximately equivariant under linear transformations, such as the orthogonalized Gnanadesikan–Kettenring estimator and the deterministic MCD algorithm.

Problem statement: High leverage points are extreme outliers in the X-direction. In regression analysis, the detection of these leverage points becomes important due to their arbitrary large effects on the estimations as well as multicollinearity problems. Mahalanobis Distance (MD) has been used as a diagnostic tool for identification of outliers in multivariate analysis where it finds the distance between normal and abnormal groups of the data. Since the computation of MD relies on non-robust classical estimations, the classical MD can hardly detect outliers accurately. As an alternative, Robust MD (RMD) methods such as Minimum Covariance Determinant (MCD) and Minimum Volume Ellipsoid (MVE) estimators had been used to identify the existence of high leverage points in the data set. However, these methods tended to swamp some low leverage points even though they can identify high leverage points correctly. Since, the detection of leverage points is one of the most important issues in regression analysis, it is imperative to introduce a novel detection method of high leverage points. Approach: In this study, we proposed a relatively new two-step method for detection of high leverage points by utilizing the RMD (MVE) and RMD (MCD) in the first step to identify the suspected outlier points. Then, in the second step the MD was used based on the mean and covariance of the clean data set. We called this method two-step Robust Diagnostic Mahalanobis Distance (RDMDTS) which could identify high leverage points correctly and also swamps less low leverage points. Results: The merit of the newly proposed method was investigated extensively by real data sets and Monte Carlo Simulations study. The results of this study indicated that, for small sample sizes, the best detection method is (RDMDTS) (MVE)-mad while there was not much difference between (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad for large sample sizes. Conclusion/Recommendations: In order to swamp less low leverage as high leverage point, the proposed robust diagnostic methods, (RDMDTS) (MVE)-mad and (RDMDTS) (MCD)-mad were recommended.

Mechanical metamaterials - engineered materials whose behavior is defined by their internal structure rather than their base material - have received great at- tention for the unusual properties they can exhibit. However, while the forward problem of predicting the mechanical behavior of a given metamaterial can be reliably done through simulation or experiments, the inverse problem - designing metamaterials with specific properties - remains difficult, especially for heteroge- neous metamaterials composed of multiple unit cells. This dissertation explores the use of generative machine learning to address this inverse design problem. The first part of the work investigates how inverse and generative models can be conditioned to produce unit cells with prescribed mechanical properties and how they can be used to generate multiple valid solutions for the same target. This is examined using a curved-beam metamaterial, which allows a compact geometric representation without complex design restrictions. The effect these restrictions have on the generative models is investigated in the second part. For this, a number of generative models—Variational Autoencoders, Generative Adversarial Networks, and Diffusion Models—are evaluated on a kirigami metamaterial where dependencies between the cuts cause intricate design constraints. The final part covers the challenges that arise when generative models are applied to heteroge- neous metamaterials, in which multiple unit cells are combined to create spatially varying mechanical properties. The results show that the applicability of generative models for the design of metamaterials strongly depends on the choice of geometric parameterization. Generative models make assumptions about their input data that generally hold for images, but can be invalid for metamaterials when complex dependencies be- tween the input parameters exist. B-splines can provide a representation without those dependencies and are therefore well suited to be paired up with generative models. Furthermore, it is also important how these geometrical parameters are sampled to create the training data for the generative models, as this sampling also affects the distribution of the properties in the training data. Overall, while generative machine learning models prove effective for the inverse design of mechanical metamaterials their successful application requires careful parameterization and selection of the training data.

In this paper, a robust statistical method called projection statistics (PS) is developed into an TV-dimensional complex form required to detect strong nonhomogeneities, or TV-dimensional outlier vectors (i.e., outliers), in space-time adaptive processing (STAP) radar training data, where N is the number of degrees of freedom (DOF) associated with the STAP scenario. The PS technique does not require the estimation of a covariance matrix, as is the case with the generalized inner product (GIP) test. Rather, it uses robust estimators of location (the median) and scale (the median absolute deviation from the median (MAD)) to identify and then excise strong nonhomogeneities in STAP training data that are capable of rendering traditional nonhomogeneity detection (NHD) techniques, such as the GIP, ineffective. Shown in the results section of this paper is sample matrix inversion (SMI) STAP performance using three separate NHD techniques: 1) a power test for high power outlier snapshots, 2) a GIP test, and 3) a PS-based test. The performance when not using any form of NHD is shown for comparison. It is clear from the results that the PS NHD method provides an SMI-STAP processor the best training data selection in the multi channel airborne radar measurements (MCARM) [B.N.S Babu et al., 1996] based radar data scenario studied here, and does so across all range cells of the scenario.

Robust principal component regression is development of principal component regression that applies robust method at principal component analysis and principal component regression analysis. Robust principal component regression does not only overcome multicollinearity problems, but also overcomes outlier problems. The robust methods used in this research are Minimum Covariance Determinant (MCD) that is applied when doing principal component analysis and Least Trimmed Squares (LTS) that is applied when doing principal component regression analysis. The case study in this research is Human Development Index (HDI) in Central Java in 2017 which is influenced by labor force participation rates, school enrollment rates, percentage of poor population, population aged 15 years and over who are employed, health facilities, gross enrollment rates, and net enrollment rates. The model of HDI in Central Java in 2017 using robust principal component regression MCD-LTS provides the most effective result for handling multicollinearity and outliers with Adjusted R2 value of 0.6913 and RSE value of 0.469. Keywords: Robust Principal Component Regression, Multicollinearity, Outliers, Minimum Covariance Determinant (MCD), Least Trimmed Squares (LTS), Human Development Index (HDI).

Influence Function and Efficiency of the Minimum Covariance Determinant Scatter Matrix Estimator

Abstract. This paper investigates the problem of local surface reconstruction and best fitting for planar surfaces from unorganized 3D point cloud data. Least Squares (LS), its equivalent Principal Component Analysis (PCA) and RANSAC are the three most popular techniques for fitting planar surfaces to 3D data. LS and PCA are sensitive to outliers and do not give reliable and robust parameter estimation. The RANSAC algorithm is robust but it is not completely free from the effect of outliers and is slow for large datasets. In this paper, we propose a diagnostic-robust statistical algorithm that uses both diagnostics and robust approaches in combination for fitting planar surfaces in the presence of outliers. Recently introduced high breakdown and fast Minimum Covariance Determinant (MCD) based location and scatter estimates are used for robust distance to identify outliers and a MCD based robust PCA approach is used as an outlier resistant technique for plane fitting. The benefits of the new diagnostic-robust algorithm are demonstrated with artificial and real laser scanning point cloud datasets. Results show that the proposed method is significantly better and more efficient than the other three methods for planar surface fitting. This method also has great potential for robust local normal estimation and for other surface shape fitting applications.

The minimum volume ellipsoid (MVE) estimator is based on the smallest volume ellipsoid that covershof thenobservations. It is an affine equivariant, high‐breakdown robust estimator of multivariate location and scatter. The MVE can be computed by a resampling algorithm. Its low bias makes the MVE very useful for outlier detection in multivariate data, often through the use of MVE‐based robust distances.We review the basic MVE definition as well as some useful extensions such as the one‐step reweighted MVE. We discuss the main properties of the MVE including its breakdown value, affine equivariance, and efficiency. We discuss the basic resampling algorithm to calculate the MVE and illustrate its use on two examples. An overview of applications is given, as well as some related classes of robust estimators of multivariate location and scatter. Copyright © 2009 John Wiley & Sons, Inc.This article is categorized under:Statistical and Graphical Methods of Data Analysis > Robust Methods

A method for pattern recognition analysis of near‐infrared spectra has been developed using robust distances determined by minimum volume ellipsoid (MVE) estimators of multivariate location and scatter. Classical methods such as the Mahalanobis distance method often fail in the presence of a moderate number of outliers in a training data set, while robust distance methods can tolerate a considerably larger proportion of outliers in a training data set Outliers can be detected by their relatively large robust distances and can be excluded from a training set without a priori knowledge of the nature of the data set. In this paper the properties of a robust distance method are examined using near‐infrared spectra of sulfamethoxazole and mixtures with its major degradation products, sulfanilic acid and sulfanilamide. The robust distance method successfully detected unacceptable samples (71.4%–89.3% (α = 0.05) or 78.6%–92.9% (α = 0.10)) even when they were inadvertently included in the training data set.

Dynamic spectrum access (DSA) benefits from detection and classification of interference sources including in-network users, out-network users, and jammers that may all coexist in a wireless network. We present a deep learning based signal (modulation) classification solution in a realistic wireless network setting, where 1) signal types may change over time; 2) some signal types may be unknown for which there is no training data; 3) signals may be spoofed such as the smart jammers replaying other signal types; and 4) different signal types may be superimposed due to the interference from concurrent transmissions. For case 1, we apply continual learning and train a Convolutional Neural Network (CNN) using an Elastic Weight Consolidation (EWC) based loss. For case 2, we detect unknown signals via outlier detection applied to the outputs of convolutional layers using Minimum Covariance Determinant (MCD) and k-means clustering methods. For case 3, we extend the CNN structure to capture phase shifts due to radio hardware effects to identify the spoofing signal sources. For case 4, we apply blind source separation using Independent Component Analysis (ICA) to separate interfering signals. We utilize the signal classification results in a distributed scheduling protocol, where in-network (secondary) users employ signal classification scores to make channel access decisions and share the spectrum with each other while avoiding interference with out-network (primary) users and jammers. Compared with benchmark TDMA-based schemes, we show that distributed scheduling constructed upon signal classification results provides major improvements to in-network user throughput and out-network user success ratio.

In bioinformatics, regularized linear discriminant analysis is commonly used as a tool for supervised classification problems tailor-made for high-dimensional data with the number of variables exceeding the number of observations. However, its various available versions are too vulnerable to the presence of outlying measurements in the data. In this paper, we exploit principles of robust statistics to propose new versions of regularized linear discriminant analysis suitable for high-dimensional data contaminated by (more or less) severe outliers. The work exploits a regularized version of the minimum weighted covariance determinant estimator, which is one of highly robust estimators of multivariate location and scatter. The performance of the novel classification methods is illustrated on real data sets with a detailed analysis of data from brain activity research.