Efficient estimation of approximate factor models via penalized maximum likelihood

The major problem in adaptively processing data from a large array is inversion of the covariance matrix. The matrix must be well-conditioned to be inverted. The number of independent samples that is needed to form a well-conditioned matrix is at least twice the number of array elements. The integration time to form a well-conditioned large array covariance matrix may then be unreasonably long relative to the coherent time of signal in a changing environment. A sub-optimal subarray processing has been developed. The adaptive weight vectors are calculated for each subarray. These weight vectors are linearly combined to form a full-array adaptive weight vector and are applied to the full-array covariance matrix. The required integration time is reduced to that which is needed to form the well-conditioned subarray covariance matrices. In a recent shallow water experiment, a three-dimensional array had been deployed. The array consisted of 5 VLAs each with 30-element spaced evenly on a circular baseline of a radius of 200 m. The subarray processing approach had been used to localize a moving source that had a fast range/bearing change relative to the baseline.

A Machine Learning Approach to Estimating Large Positive Definite Covariance Matrix of High Frequency Data

(ProQuest: ... denotes formulae omitted.)I.INTRODUCTIONKorea and Japan have been closely linked not only economically but also in many aspects. For example, Japan has been one of the major trading partners of Korea and the volume of trade between the two countries has significantly increased since the development of the Korean economy. Further, there has been a structural transformation in the Korean economy: Korea's capital markets have been extensively integrated to the international financial markets and Korea significantly opened its product markets to the world since the Asian financial crisis in 1997-98 so that the ratios of Korean export and import to its GDP are around 50% as of 2015, respectively. Although all these events point to the direction in favor of a closer integration of product markets, this paper empirically investigates the degree of product market integration between Korea and Japan using a measure of sectoral relative consumer price persistence over the period of 1985-2016. In particular, we examine if sectoral relative prices between the two countries converted with the common currency contain a stochastic trend or converge to their equilibrium value in the long run.We apply a new econometric method for testing the unit root hypothesis of real exchange rates: we use the panel unit root test in the presence of cross-section dependence developed by Pesaran (2007). His method is to augment the crosssectional averages of lagged levels and first-differences of the individual series in the standard augmented Dickey-Fuller (ADF) regressions to control for crosssection dependence. The use of Pesaran's method has two advantages compared to previous studies.One advantage is that the method obtains an additional cross-sectional power using panel data and mitigates the well-known shortcoming of univariate unit root tests in terms of their power properties.1 The other advantage is that the method takes into account of cross-section dependence frequently presented in panel data sets. In particular, panels with sectoral real exchange rates are likely exhibit high crosssection dependence since they are constructed using a common nominal exchange rate. The well-known panel unit root tests such as the LLC test developed by Levin et al. (2002), the IPS test by Im et al. (2003), and the Maddala-Wu test by Maddala and Wu (1999) assume that individual time series in the panel are independently distributed across cross-sections. However, O'Connell (1998) shows that the typical panel unit root tests developed under the cross-sectional independence assumption reject too often the unit root hypothesis in the presence of significant cross-section dependence. Further, recent econometric studies show that the method of demeaning the individual series cross-sectionally does not effectively mitigate the problem of cross-section dependence in a general setting. Pesaran's method overcomes this deficiency of the previous tests. Pesaran (2007) also confirms O'Connell (1998)'s finding applying the new econometric method: he examines if the CPI-based real exchange rates constructed using quarterly consumer price indices of 17 OECD countries contain a stochastic trend for the periods of 1974Q1-1998Q4 and 1988Q1-1998Q4, and finds that the IPS test rejects the unit root hypothesis for both cases, while his test developed under the assumption of cross-section dependence does not reject it. He also shows that these opposite results are mainly due to the significant cross-section dependence in those panels. As formally and informally discussed in Section III and IV, the error term of the regression for our panel of sectoral real exchange rates exhibits significant cross-section dependence. Therefore, the use of Pesaran (2007)'s method is appropriate for our study.We find that sectoral relative prices between Korea and Japan do not revert to their equilibrium value in the long run applying Pesaran (2007)'s panel unit root test. …

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

The eigenvector Empirical Spectral Distribution (VESD) is adopted to investigate the limiting behavior of eigenvectors and eigenvalues of covariance matrices. In this paper, we shall show that the Kolmogorov distance between the expected VESD of sample covariance matrix and the Mar\v{c}enko-Pastur distribution function is of order $O(N^{-1/2})$. Given that data dimension $n$ to sample size $N$ ratio is bounded between 0 and 1, this convergence rate is established under finite 10th moment condition of the underlying distribution. It is also shown that, for any fixed $\eta>0$, the convergence rates of VESD are $O(N^{-1/4})$ in probability and $O(N^{-1/4+\eta})$ almost surely, requiring finite 8th moment of the underlying distribution.

We consider sample covariance matrices $${S_N=\frac{1}{p}\Sigma_N^{1/2}X_NX_N^* \Sigma_N^{1/2}}$$ where X N is a N × p real or complex matrix with i.i.d. entries with finite 12th moment and Σ N is a N × N positive definite matrix. In addition we assume that the spectral measure of Σ N almost surely converges to some limiting probability distribution as N → ∞ and p/N → γ > 0. We quantify the relationship between sample and population eigenvectors by studying the asymptotics of functionals of the type $${\frac{1}{N}\text{Tr} ( g(\Sigma_N) (S_N-zI)^{-1}),}$$ where I is the identity matrix, g is a bounded function and z is a complex number. This is then used to compute the asymptotically optimal bias correction for sample eigenvalues, paving the way for a new generation of improved estimators of the covariance matrix and its inverse.

Bias reduction in nonlinear and dynamic panels in the presence of cross-section dependence

The smoothness of subdiagonals of the Cholesky factor of large covariance matrices is closely related to the degree of nonstationarity of autoregressive models for time series data. Heuristically, one expects for nearly stationary covariance matrix entries in each subdiagonal of the Cholesky factor of its inverse to be approximately the same in the sense that the sum of the absolute values of successive differences is small. Statistically, such smoothness is achieved by regularizing each subdiagonal using fused-type lasso penalties. We rely on the standard Cholesky factor as the new parameter within a regularized normal likelihood setup which guarantees: (a) joint convexity of the likelihood function, (b) strict convexity of the likelihood function restricted to each subdiagonal even when n < p, and (c) positive-definiteness of the estimated covariance matrix. A block coordinate descent algorithm, where each block is a subdiagonal, is proposed, and its convergence is established under mild conditions. Lack of decoupling of the penalized likelihood function into a sum of functions involving individual subdiagonals gives rise to some computational challenges and advantages relative to two recent algorithms for sparse estimation of the Cholesky factor, which decouple row-wise. Simulation results and real data analysis show the scope and good performance of the proposed methodology. Software for our method is freely available in R language. Supplementary materials for this article are available online.

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&amp;P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

Inferences in panel data with interactive effects using large covariance matrices

Obtaining accurate estimates of the genetic covariance matrix Sigma(G) for multivariate data is a fundamental task in quantitative genetics and important for both evolutionary biologists and plant or animal breeders. Classical methods for estimating Sigma(G) are well known to suffer from substantial sampling errors; importantly, its leading eigenvalues are systematically overestimated. This article proposes a framework that exploits information in the phenotypic covariance matrix Sigma(P) in a new way to obtain more accurate estimates of Sigma(G). The approach focuses on the "canonical heritabilities" (the eigenvalues of Sigma(P)(-1)Sigma(G)), which may be estimated with more precision than those of Sigma(G) because Sigma(P) is estimated more accurately. Our method uses penalized maximum likelihood and shrinkage to reduce bias in estimates of the canonical heritabilities. This in turn can be exploited to get substantial reductions in bias for estimates of the eigenvalues of Sigma(G) and a reduction in sampling errors for estimates of Sigma(G). Simulations show that improvements are greatest when sample sizes are small and the canonical heritabilities are closely spaced. An application to data from beef cattle demonstrates the efficacy this approach and the effect on estimates of heritabilities and correlations. Penalized estimation is recommended for multivariate analyses involving more than a few traits or problems with limited data.

In this work, we describe advanced numerical tools for working with multivariate functions and for the analysis of large data sets. These tools will drastically reduce the required computing time and the storage cost, and, therefore, will allow us to consider much larger data sets or finer meshes. Covariance matrices are crucial in spatio-temporal statistical tasks, but are often very expensive to compute and store, especially in three dimensions. Therefore, we approximate covariance functions by cheap surrogates in a low-rank tensor format. We apply the Tucker and canonical tensor decompositions to a family of Matérn- and Slater-type functions with varying parameters and demonstrate numerically that their approximations exhibit exponentially fast convergence. We prove the exponential convergence of the Tucker and canonical approximations in tensor rank parameters. Several statistical operations are performed in this low-rank tensor format, including evaluating the conditional covariance matrix, spatially averaged estimation variance, computing a quadratic form, determinant, trace, loglikelihood, inverse, and Cholesky decomposition of a large covariance matrix. Low-rank tensor approximations reduce the computing and storage costs essentially. For example, the storage cost is reduced from an exponential 𝒪 ⁢ ( n d ) {\mathcal{O}(n^{d})} to a linear scaling 𝒪 ⁢ ( d ⁢ r ⁢ n ) {\mathcal{O}(drn)} , where d is the spatial dimension, n is the number of mesh points in one direction, and r is the tensor rank. Prerequisites for applicability of the proposed techniques are the assumptions that the data, locations, and measurements lie on a tensor (axes-parallel) grid and that the covariance function depends on a distance, ∥ x - y ∥ {\|x-y\|} .

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.

Journal Article Nonparametric estimation of large covariance matrices of longitudinal data Get access Wei Biao Wu, Wei Biao Wu Search for other works by this author on: Oxford Academic Google Scholar Mohsen Pourahmadi Mohsen Pourahmadi Search for other works by this author on: Oxford Academic Google Scholar Biometrika, Volume 90, Issue 4, December 2003, Pages 831–844, https://doi.org/10.1093/biomet/90.4.831 Published: 01 December 2003

Estimating large covariance matrix with network topology for high-dimensional biomedical data