Advances in high-dimensional covariance matrix estimation

We introduce nonparametric regularization of the eigenvalues of a sample covariance matrix through splitting of the data (NERCOME), and prove that NERCOME enjoys asymptotic optimal nonlinear shrinkage of eigenvalues with respect to the Frobenius norm. One advantage of NERCOME is its computational speed when the dimension is not too large. We prove that NERCOME is positive definite almost surely, as long as the true covariance matrix is so, even when the dimension is larger than the sample size. With respect to the Stein’s loss function, the inverse of our estimator is asymptotically the optimal precision matrix estimator. Asymptotic efficiency loss is defined through comparison with an ideal estimator, which assumed the knowledge of the true covariance matrix. We show that the asymptotic efficiency loss of NERCOME is almost surely 0 with a suitable split location of the data. We also show that all the aforementioned optimality holds for data with a factor structure. Our method avoids the need to first estimate any unknowns from a factor model, and directly gives the covariance or precision matrix estimator, which can be useful when factor analysis is not the ultimate goal. We compare the performance of our estimators with other methods through extensive simulations and real data analysis.

This paper develops a parallel implementation of the ensemble Kalman filter (EnKF) based on shrinkage covariancematrix estimation. The EnKF is a sequential Monte Carlo method for parameter and state estimation of highly nonlinearmodels. In the context of ensemble based methods, background error correlations are estimated via anensemble of model realizations. In practice, the model dimension (number of model components) is several timeslarger than the ensemble size owing to the computational effort involved in one single model propagation. Thisconstraint yields to spurious correlations (correlation between distant model components in space) in the backgrounderror estimations. Commonly, localization methods are utilized in order to reduce the impact of spuriouscorrelations. For instance, correlations of model components are reduced based on some function of their physicaldistances. Some EnKF implementations are formulated in such manner that, localization methods are implicitlyinvolved in the assimilation of observations. In this context, one of the best ensemble Kalman filter implementationsis the local ensemble transform Kalman filter (LETKF). In the LETKF, each model component is surrounded by alocal box of size r and then, the information contained within the local domain (i.e., observed components) is used toperform the assimilation. However, for sparse observational networks, r can become sufficiently large with respectto the ensemble size and therefore, the local analysis corrections can be impacted by spurious correlations. Wepropose an ensemble Kalman filter implementation in which the background error covariance matrix is estimated viathe Rao-Blackwell Ledoit and Wolf (RBLW) estimator. This estimator has proven to be better conditioned than thetrue (background) error covariance matrix. In addition, the explicit computation of the RBLW estimator is notrequired in our formulation. This implies enormous savings in terms of memory considering the typical order ofbackground error covariance matrices O(10e8) Furthermore, the proposed EnKF implementation can be efficientlyperformed in parallel: each model component is surrounded by a local box, a local background error covariancematrix is estimated based on the RBLW estimator and then, the ensemble Kalman filter equations are locally used inorder to compute the local analysis corrections. After the local assimilation, each model component is mapped backonto the full model domain from where the global analysis solution is obtained. In operational data assimilation, EnKF implementations are attractive owing to their potential for parallelization. Experiments are performed making use of the Atmospheric General Circulation Model SPEEDY. The proposed method is coded in FORTRAN and MPI is used for the local assimilation of observations. The predicted variables are the zonal and meridional wind components, the temperature and the specific humidity. For each model variable, 8 layers with the T-63 resolution (192 x 96 grid points) are considered which provide a total number of 589,824 model components. The ensemble size is 94. The number of processors utilized are 96, 256, 512, 768, 1024 and 1536. The results reveal that, the use of the RBLW estimator can mitigate the impact of spurious correlations under sparse observational networks and even more, for different number of processors, the computational effort of the proposed EnKF implementation is comparable to that of the LETKF where no covariance estimation is performed. In terms of accuracy, the proposed implementation performs better than the LETKF formulation in the Root Mean Square Error sense.

Robust covariance estimates based on resampling

Arrays can be designed as a full array of sensors or a sparse array of sensors with an equivalent aperture. Reductions in cost, size, weight and power consumption are important factors to consider when designing an array of sensors. Sparse arrays can provide a reduction in these factors. It is important to observe the performance of sparse arrays in relation to full arrays to see if they are able to achieve similar performance. The ability for the full and sparse arrays of sensors to accurately estimate the direction of arrival (DOA) of plane waves impinging them is observed in this thesis. The Mean-squared error (MSE) of the DOA estimation is measured against the Creamer-Rao bound (CRB) as a benchmark for optimal performance for an unbiased estimator. We derive an optimal way to remove sensors for one signal impinging on an array by finding the minimum CRB. This method considers all the unique combinations a sparse array could be assembled for a given number of sensors and aperture. Parameters such as number of sensors, sensor locations, signal-to-noise ratio (SNR), number of snapshots, and signal correlation affect the MSE and CRB. Through observing trends in the MSE as a result of changing these parameters we can come to conclusions to how full and sparse arrays perform as a result of adjusting these parameters. The MSE of the DOA of the full and sparse arrays is obtained through a covariance estimation from subspace-based DOA algorithms. The sample covariance and the diagonal-averaged sample covariance matrix performances are compared. The sample covariance matrix for a full array provides the best estimations at high SNR while the diagonal-averaging method provides a more accurate estimation at low SNR. The sparse array diagonal-averaged covariance matrix estimation performs similarly to the diagonal-averaged covariance matrix of the full array, but is still slightly worse due to missing information. How sparsity affects performance was also observed. As sparsity increases for an array, the performance change is non-linear for the MSE and CRB. Increases in signal correlation cause an increase in the overall MSE and a decrease in the overall CRB for both sparse and full arrays.

When shrinking a covariance matrix towards (a multiple) of the identity matrix, the trace of the covariance matrix arises naturally as the optimal scaling factor for the identity target. The trace also appears in other context, for example when measuring the size of a matrix or the amount of uncertainty. Of particular interest is the case when the dimension of the covariance matrix is large. Then the problem arises that the sample covariance matrix is singular if the dimension is larger than the sample size. Another issue is that usually the estimation has to based on correlated time series data. We study the estimation of the trace functional allowing for a high-dimensional time series model, where the dimension is allowed to grow with the sample size—without any constraint. Based on a recent result, we investigate a confidence interval for the trace, which also allows us to propose lower and upper bounds for the shrinkage covariance estimator as well as bounds for the variance of projections. In addition, we provide a novel result dealing with shrinkage towards a diagonal target. We investigate the accuracy of the confidence interval by a simulation study, which indicates good performance, and analyze three stock market data sets to illustrate the proposed bounds, where the dimension (number of stocks) ranges between 32 and 475. Especially, we apply the results to portfolio optimization and determine bounds for the risk associated to the variance-minimizing portfolio.

A Machine Learning Approach to Estimating Large Positive Definite Covariance Matrix of High Frequency Data

Adaptive beamformers often rely on estimates of the interference covariance structure. When minimal data is used in estimating the unknown covariance matrix, performance may be improved by modifying the eigenvalues of the sample covariance matrix. The orthogonally invariant covariance estimators proposed by Dey and Srinivasan [1985] and a constant risk, minimax estimator are applied to adaptive beamforming by considering the signal-to-interference ratio (SIR) loss factor of Reed, Mallett, and Brennan [1974] where the average SIR loss is seen to decrease, and, to adaptive detection, by considering the adaptive detectors of Kelly [1986] and Robey et. al. [1992] where detection performance is seen to improve. >

Sparse linear arrays, such as nested and co-prime arrays, are capable of resolving $O(M^2)$ sources using only $O(M)$ sensors by exploiting their so-called difference coarray model. One popular approach to exploit the difference coarray model is to construct an augmented covariance matrix from the sample covariance matrix. By applying common direction-of-arrival (DOA) estimation algorithms to this augmented covariance matrix, more sources than the number of sensors can be identified. In this letter, inspired by the optimal transport theory, we develop a new approach to construct this augmented covariance matrix. We formulate a structured covariance estimation problem that minimizes the Bures–Wasserstein distance between the sample covariance matrix and the subsampled augmented covariance matrix, which can be either casted to a semi-definite programming problem, or directly solved using gradient-based methods. Our approach contributes to a new grid-less DOA estimation algorithm for sparse linear arrays. Numerical examples show that our approach achieves state-of-art estimation performance.

This study intends to analyze the role of entrepreneurship as a moderating variable on the effects of capital structure decisions on financial performance. Population, all the small and medium industries (SMEs) in the town of Probolinggo, with a sample set of 100 units of SMEs and data collection using purposive random sampling technique. Data were analyzed using factor analysis in the SEM is used to confirm the most dominant factors in a group of variables and regression weight on the SEM used for confirmatory examine how much the relationship between variables. Furthermore, SEM modeling needs to step (a) the development of a theoretical model, (b) the development path diagram (c) selecting an input matrix and estimation of covariance or correlation models, (d) the possibility of the emergence of problem identification, (e) evaluation criteria of goodness-of-fit and (f) the interpretation and modification of the model. The results showed, capital structure decisions have a significant effect on financial performance. Capital structure decisions have a significant effect on the financial performance through entrepreneurship moderating variable. Entrepreneurship significant effect on the financial performance.

This paper addresses improvements on the design of the adaptive normalized matched filter (ANMF) for radar detection. It is well acknowledged that the estimation of the noise-clutter covariance matrix is a fundamental step in adaptive radar detection. In this paper, we consider regularized estimation methods that force by construction the eigenvalues of the covariance estimates to be greater than a positive regularization parameter $\rho$ . This makes them more suitable for high-dimensional problems with a limited number of secondary data samples than traditional sample covariance estimates. The motivation behind this paper is to understand the effect and properly set the value of $\rho$ that improves estimate conditioning while maintaining a low-estimation bias. More specifically, we consider the design of the ANMF detector for two kinds of regularized estimators, namely the regularized sample covariance matrix, the regularized Tyler estimator (RTE). The rationale behind this choice is that the RTE is efficient in mitigating the degradation caused by the presence of impulsive noises while inducing little loss when the noise is Gaussian. Based on asymptotic results brought by recent tools from random matrix theory, we propose a design for the regularization parameter that maximizes the asymptotic detection probability under constant asymptotic false alarm rates. Provided simulations support the efficiency of the proposed method, illustrating its gain over conventional settings of the regularization parameter.

Agriculture extension and advisory services is a multidisciplinary discipline based on human interaction seeking to improve the livelihoods of farming communities and individuals by providing information and technologies. The training of extension staff is important as it has a bearing on their effectiveness in the office and in the field. This study sought to determine the HRD activities in agricultural extension and advisory services in the public and private sector. The study was done in 5 counties in Kenya and a total of 440 agricultural extension agents were sampled from the public and private extension service. HRD activities focused on formal and in-service training. 68 % of the respondents had attended formal education to improve their education with the majority 63.5 percent, having trained at the diploma level from certificate level while 21.1 % had undergone training at the degree level from diploma level. The main areas of specialization were Agricultural education (34.1 %), General agriculture (28.1 %) and Horticulture (11.7 %). The inclusion of non-agricultural areas of specialization such as Sustainable development and Strategic planning and management show the multidisciplinary nature of agriculture. Inservice courses attended were in the form of short courses, seminars, or workshops. These were clustered in five general areas; Crop Management, Management, Agricultural Economics, Agricultural Extension, and Animal Science. Most respondents (40.9 percent) had undergone training in Crop management which covered various crop enterprises from breeding to postharvest management. The shift of agricultural policy toward business orientation is reflected in 13.0 % the respondents specializing in Agricultural Economics. These HRD activities show commitment of agricultural extension providers to improve the competencies of their staff to deliver effective services to farmers. The wide range of formal and in-service courses attended also reflects the need to meet the management and technical requirements of a pluralistic and demand driven extension service.

Detecting emergence of a low-rank signal from high-dimensional data is an important problem arising from many applications such as camera surveillance and swarm monitoring using sensors. We consider a procedure based on the largest eigenvalue of the sample covariance matrix over a sliding window to detect the change. To achieve dimensionality reduction, we present a sketching-based approach for rank change detection using the low-dimensional linear sketches of the original high-dimensional observations. The premise is that when the sketching matrix is a random Gaussian matrix, and the dimension of the sketching vector is sufficiently large, the rank of sample covariance matrix for these sketches equals the rank of the original sample covariance matrix with high probability. Hence, we may be able to detect the low-rank change using sample covariance matrices of the sketches without having to recover the original covariance matrix. We character the performance of the largest eigenvalue statistic in terms of the false-alarm-rate and the expected detection delay, and present an efficient online implementation via subspace tracking.

Factor analysis is a statistical method used to describe a set of variables based on common dimensions. Factor analysis that is often used is the classical factor analysis with principal components method. Classical factor analysis can not work properly if the data contained many outliers. In order factor analysis remains optimal in explaining a set of variables even in conditions of data containing many outliers, we need a robust estimator. Through factor analysis is expected to obtain robust high accuracy analysis results for data containing many outliers. Estimator fast-MCD is one of the robust estimator that aims to get the smallest determinant of the covariance matrix. Robust factor analysis with fast-MCD method in this thesis is applied to explain the many subgroups of food at food inflation rate in Central Java into a more modest dimensions. The total proportion of the data variance can be explained by factors that are formed through a robust method of factor analysis in foodstuffs inflation data in Central Java that is equal to 72.9 percent larger than the classical factor analysis method which generates at 53.5 percent. This suggests that a more robust factor analysis method is able to cope with food inflation data in Central Java group containing outliers of the classical factor analysis method.

Using conditional bias in principal component analysis for the evaluation of joint influence on the eigenvalues of the covariance matrix

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.