An Efficient Dictionary Learning Algorithm for Sparse Representation

Slip/antiblocking agents

Medical image fusion based on sparse representation of classified image patches

Corporate Bond Spreads and Real Activity in the Euro Area - Least Angle Regression Forecasting and the Probability of the Recession

In this paper a model selection algorithm for a nonlinear system identification method is proposed to study functional magnetic resonance imaging (fMRI) effective connectivity. Unlike most other methods, this method does not need a pre-defined structure/model for effective connectivity analysis. Instead, it relies on selecting significant nonlinear or linear covariates for the differential equations to describe the mapping relationship between brain output (fMRI response) and input (experiment design). These covariates, as well as their coefficients, are estimated based on a least angle regression (LARS) method. In the implementation of the LARS method, Akaike's information criterion corrected (AICc) algorithm and the leave-one-out (LOO) cross-validation method were employed and compared for model selection. Simulation comparison between the dynamic causal model (DCM), nonlinear identification method, and model selection method for modelling the single-input-single-output (SISO) and multiple-input multiple-output (MIMO) systems were conducted. Results show that the LARS model selection method is faster than DCM and achieves a compact and economic nonlinear model simultaneously. To verify the efficacy of the proposed approach, an analysis of the dorsal and ventral visual pathway networks was carried out based on three real datasets. The results show that LARS can be used for model selection in an fMRI effective connectivity study with phase-encoded, standard block, and random block designs. It is also shown that the LOO cross-validation method for nonlinear model selection has less residual sum squares than the AICc algorithm for the study.

It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. \cite{LARS}), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: 1) the local geometry of samples is well preserved for low dimensional data representation, 2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, 3) the projection matrix of MEN improves the parsimony in computation, 4) the elastic net penalty reduces the over-fitting problem, and 5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.

This work presents the use of Dictionary Learning methods for a sparse representation of 4D seismic data in history matching. We consider a trade-off between the number of coefficients retained in the sparse data representation, the computational cost, and how well we can capture the main features of the 4D seismic signal. K-SVD is an iterative algorithm used in Dictionary Learning problems that alternates between the calculation of the sparse representation vector and dictionary update. For the definition of the sparse representation vector, one can constrain the problem into two distinct approaches: (1) sparsity-constrained; and (2) error-constrained. We evaluated the two methods and the influence of critical parameters of the algorithm (dictionary size, number of iterations, patch size, and training dataset size) using a synthetic reservoir model. Results showed that regardless of which of the constrained approaches we used, the dictionary learning method can capture the main features of the 4D seismic signal with a sparse representation. However, the number of nonzero coefficients is highly dependent on the approach. Besides, dictionary size, number of iterations, patch size, and training dataset size, also have a significant impact on the number of coefficients and the computational cost. Consequently, the selection of these variables is crucial as it might affect the history-matching process. In a permanent-reservoir-monitoring (big data) scenario, the use of sparse representation allows us to retain the main features of the reservoir in a reasonably sized data representation that can be directly used in a history matching method, leading to improved reservoir characterization and a better understanding of the reservoir properties. This paper gives a practical implementation of a technique already used in other areas (image denoising, processing of raw seismic data, and facies representation) applied to 4D seismic data.

Quadratic approximation on SCAD penalized estimation

Feature-guided regularization parameter selection in sparse de-noising for fault diagnosis

In multivariate statistical process control (MSPC) applications, process mean shifts sometimes occur in only a few components. To solve this MSPC problem, many control charts were proposed in the literature. Most of these charts assumed that the multivariate quality characteristics are normally distributed. Among them, the control chart proposed by Zou and Qiu (2009), incorporating the least absolute shrinkage and selection operator (LASSO) method into the EWMA scheme, has the best overall performance. In this paper, we extend the classical multivariate LASSO control chart to a robust version that has an affine-invariance property and is distribution free under the family of elliptical direction distributions, indicating that the in-control run-length distribution is the same for any continuous distribution in this family and the control limit can be acquired from the multivariate standard normal distribution. Our simulation results show that the proposed method is very efficient in detecting various sparse shifts under heavy-tailed and skewed multivariate distributions. In addition, it is easy to implement with an iterative algorithm and the least angle regression (LARS) algorithm. White-wine data illustrates that the proposed control chart performs quite well in applications.

The Lasso and Ridge estimators are effective variable selection methods for a simple linear model. While the multiple linear model is often used in practice, only few studies have examined its problem of variable selection. To address this gap, we study such problem by taking advantage of the Lasso and Ridge estimate. We propose two variable selection methods in the multiple linear model and introduce the multiple random simulation (MRS) algorithm, whose efficiency is similar to that of the least angle regression (LARS) algorithm in a simple linear model. We verify the performance of these two algorithms by applying them to real diabetes data and find that the LARS algorithm cannot be easily applied in a multiple linear model. We also illustrate the excellent performance of MRS by applying this algorithm on a simulated dataset.

Variable selection plays a significant role in statistics. There are many variable selection methods. Forward stagewise regression takes a different approach among those. In this thesis Least Angle Regression (LAR) is discussed in detail. This approach has similar principles as forward stagewise regression but does not suffer from its computational difficulties. By using a small artificial data set and the well-known Longley data set, the LAR algorithm is illustrated in detail and the coefficient profiles are obtained. Furthermore a penalized approach to variable reduction called the LASSO is discussed, and it is shown how to compute its coefficient profiles efficiently using the LAR algorithm with a small modification. Finally, a method called K-fold cross validation used to select the constraint parameter for the LASSO is presented and illustrated with the Longley data.

Uncertainties in gas turbine aero-engines are unavoidable among which manufacturing tolerance is a typical manifestation. Uncertainties from manufacturing tolerance directly influence the thrust regulation performance, which may lead to the technical risks of newly produced aero-engines. Moreover, classic sample-based uncertainty quantification approaches are usually computationally intensive. In this paper, to consider the uncertainties in the control design phase in advance, a polynomial chaos expansion-based uncertainty model (PCEUM) using a sparse regression method is proposed to get the accurate probability distribution of thrust regulation performance and other concerned engine variables at a decreased computational burden. In PCEUM, engine variables are initially expressed as linear combinations of several orthogonal polynomials, whose weighting coefficients are solved by a sparse-regression-based method, i.e. orthogonal matching pursuit (OMP). A nominal aero-thermal model for a large turbofan engine, whose maximum errors for key engine parameters are within 2.25% against NPSS data, serves as the basis of PCEUM. Meanwhile, two classic sample-based uncertainty quantification approaches, (i.e. Monte-Carlo simulations (MCS), Latin hypercube sampling (LHS)) and a least angle regression (LARS) based PCE are set as benchmarks. Numerical simulations using publicly available manufacturing tolerance statistics are conducted at take-off states for the tested engine on a desktop computer. Results show that the proposed PCEUM costs only 47.06s at the expense of 200 samples to obtain stable probability distributions for interested engine parameters, e.g. thrust, whose errors of mean and standard deviation compared with MCS at 100,000 samples are within 0.01% and 1%, respectively. While the computational burden of MCS, LHS, and LARS are 854.63s at 9,000 samples, 250.13s at 3,000 samples, and 54.16s at 200 samples at the same precision level, respectively. It means that compared to the latter three methods, PCEUM can save 94.5%, 81.2%, and 13.1% of the simulation time, respectively. Hence, the proposed model is verified regarding both the accuracy and speed for uncertainty assessment, which provides a promising solution for both conventional gas turbine engines and future aero-propulsion systems.

New insights into the spatial distribution of particle number concentrations by applying non-parametric land use regression modelling

DNA microarray SNP associations with clinical efficacy and side effects of domperidone treatment for gastroparesis

ABSTRACTA new speech enhancement method using sparse reconstruction of the log-spectra is presented. Similar to the traditional sparse coding methods, the proposed algorithm makes use of the least angle regression (LARS) with a coherence criterion (LARC) algorithm to reconstruct the log power spectrum of clean speech. However, a new stopping criterion is introduced to allow the LARC algorithm to adapt to various background noise environments. In addition, a modified two-step noise reduction with a log-MMSE filter is applied which solves the bias of estimated a-priori signal-to-noise ratio (SNR). A notable improvement in the proposed algorithm over traditional speech enhancement methods is its adaptability to the changes in the SNR of noisy speech. The performance of the proposed algorithm is evaluated using standard measures based on a large set of speech and noise signals. The results show that a significant improvement is achieved compared to traditional approaches, especially in non-stationary noise en...