Elastic Net (Model Selection)

In this paper a model selection algorithm for a nonlinear system identification method is proposed to study functional magnetic resonance imaging (fMRI) effective connectivity. Unlike most other methods, this method does not need a pre-defined structure/model for effective connectivity analysis. Instead, it relies on selecting significant nonlinear or linear covariates for the differential equations to describe the mapping relationship between brain output (fMRI response) and input (experiment design). These covariates, as well as their coefficients, are estimated based on a least angle regression (LARS) method. In the implementation of the LARS method, Akaike's information criterion corrected (AICc) algorithm and the leave-one-out (LOO) cross-validation method were employed and compared for model selection. Simulation comparison between the dynamic causal model (DCM), nonlinear identification method, and model selection method for modelling the single-input-single-output (SISO) and multiple-input multiple-output (MIMO) systems were conducted. Results show that the LARS model selection method is faster than DCM and achieves a compact and economic nonlinear model simultaneously. To verify the efficacy of the proposed approach, an analysis of the dorsal and ventral visual pathway networks was carried out based on three real datasets. The results show that LARS can be used for model selection in an fMRI effective connectivity study with phase-encoded, standard block, and random block designs. It is also shown that the LOO cross-validation method for nonlinear model selection has less residual sum squares than the AICc algorithm for the study.

Corporate Bond Spreads and Real Activity in the Euro Area - Least Angle Regression Forecasting and the Probability of the Recession

Sparse and redundant representation of data assumes an ability to describe signals as linear combinations of a few atoms from a dictionary. If the model of the signal is unknown, the dictionary can be learned from a set of training signals. Like the K-SVD, many of the practical dictionary learning algorithms are composed of two main parts: sparse-coding and dictionary-update. This paper first proposes a Stagewise least angle regression (St-LARS) method for performing the sparse-coding operation. The St-LARS applies a hard-thresholding strategy into the original least angle regression (LARS) algorithm, which enables it to select many atoms at each iteration and thus results in fast solutions while still provides good results. Then, a dictionary update method named approximated singular value decomposition (ASVD) is used on the dictionary update stage. It is a quick approximation of the exact SVD computation and can reduce the complexity of it. Experiments on both synthetic data and 3-D image denoising demonstrate the advantages of the proposed algorithm over other dictionary learning methods not only in terms of better trained dictionary but also in terms of computation time.

The purpose of Multivariate Curve Resolution (MCR) is to recover the concentration profile and the source spectra without any prior knowledge. We hypothesis that each source is characterized by a linear superposition of Gaussian peaks of fixed spread. Multivariate curve resolution–alternating least squares (MCR-ALS) is a Conventional MCR method. MCR-ALS has some disadvantages. We proposed a solver with L1 regularizer and L2 regularizer to obtain a sparse solution within MCR-ALS. L1-norm involves a sparse but non-smooth solution, L2-norm will keep all the information and bring the smoothness, but it will lead non-sparse solutions. So we combined the L1-norm and the squared L2-norm to seek the optimal solutions. This is accomplished via Elastic Net Regularization algorithem which is LARS (least-angle regression). We named this method MCR-LARS. This paper applies MCR-LARS to resolve the hard overlapped spectroscopic signals belonging to the three aromatic amino acids (phenylalanine, tyrosine and tryptophan) in their mixtures. MCR–LARS was compared with MCR-ALS. The results show the effectiveness and efficiency of MCR–LARS and the results show that MCR-LARS provides more nicely resolved concentration profiles and spectra than pure MCR-ALS solution.

It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. \cite{LARS}), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: 1) the local geometry of samples is well preserved for low dimensional data representation, 2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, 3) the projection matrix of MEN improves the parsimony in computation, 4) the elastic net penalty reduces the over-fitting problem, and 5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.

In public health research, it has been well established that geographic location plays an important role in influencing health outcomes. In recent years, there has been an increased emphasis on the impact of neighborhood or contextual factors as potential risk factors for childhood obesity. Some neighborhood factors relevant to childhood obesity include access to food sources, access to recreational facilities, neighborhood safety, and socioeconomic status (SES) variables. It is common for neighborhood or area-level variables to be available at multiple spatial scales (SS) or geographic units, such as the census block group and census tract, and selection of the spatial scale for area-level variables can be considered as a model selection problem. In this paper, we model the variation in body mass index (BMI) in a study of pediatric patients of the Virginia Commonwealth University (VCU) Medical Center, while considering the selection of spatial scale for a set of neighborhood-level variables available at multiple spatial scales using four recently proposed spatial scale selection algorithms: SS forward stepwise regression, SS incremental forward stagewise regression, SS least angle regression (LARS), and SS lasso. For pediatric BMI, we found evidence of significant positive associations with visit age and black race at the individual level, percent Hispanic white at the census block group level, percent Hispanic black at the census tract level, and percent vacant housing at the census tract level. We also found significant negative associations with population density at the census tract level, median household income at the census tract level, percent renter at the census tract level, and exercise equipment expenditures at the census block group level. The SS algorithms selected covariates at different spatial scales, producing better goodness-of-fit in comparison to traditional models, where all area-level covariates were modeled at the same scale. These findings underscore the importance of considering spatial scale when performing model selection.

PurposeWith the prosperity of grey extension models, the form and structure of grey forecasting models tend to be complicated. How to select the appropriate model structure according to the data characteristics has become an important topic. The purpose of this paper is to design a structure selection method for the grey multivariate model.Design/methodology/approachThe linear correction term is introduced into the grey model, then the nonhomogeneous grey multivariable model with convolution integral [NGMC(1,N)] is proposed. Then, by incorporating the least absolute shrinkage and selection operator (LASSO), the model parameters are compressed and estimated based on the least angle regression (LARS) algorithm.FindingsBy adjusting the values of the parameters, the NGMC(1,N) model can derive various structures of grey models, which shows the structural adaptability of the NGMC(1,N) model. Based on the geometric interpretation of the LASSO method, the structure selection of the grey model can be transformed into sparse parameter estimation, and the structure selection can be realized by LASSO estimation.Practical implicationsThis paper not only provides an effective method to identify the key factors of the agricultural drought vulnerability, but also presents a practical model to predict the agricultural drought vulnerability.Originality/valueBased on the LASSO method, a structure selection algorithm for the NGMC(1,N) model is designed, and the structure selection method is applied to the vulnerability prediction of agricultural drought in Puyang City, Henan Province.

Variable selection plays a significant role in statistics. There are many variable selection methods. Forward stagewise regression takes a different approach among those. In this thesis Least Angle Regression (LAR) is discussed in detail. This approach has similar principles as forward stagewise regression but does not suffer from its computational difficulties. By using a small artificial data set and the well-known Longley data set, the LAR algorithm is illustrated in detail and the coefficient profiles are obtained. Furthermore a penalized approach to variable reduction called the LASSO is discussed, and it is shown how to compute its coefficient profiles efficiently using the LAR algorithm with a small modification. Finally, a method called K-fold cross validation used to select the constraint parameter for the LASSO is presented and illustrated with the Longley data.

Uncertainties in gas turbine aero-engines are unavoidable among which manufacturing tolerance is a typical manifestation. Uncertainties from manufacturing tolerance directly influence the thrust regulation performance, which may lead to the technical risks of newly produced aero-engines. Moreover, classic sample-based uncertainty quantification approaches are usually computationally intensive. In this paper, to consider the uncertainties in the control design phase in advance, a polynomial chaos expansion-based uncertainty model (PCEUM) using a sparse regression method is proposed to get the accurate probability distribution of thrust regulation performance and other concerned engine variables at a decreased computational burden. In PCEUM, engine variables are initially expressed as linear combinations of several orthogonal polynomials, whose weighting coefficients are solved by a sparse-regression-based method, i.e. orthogonal matching pursuit (OMP). A nominal aero-thermal model for a large turbofan engine, whose maximum errors for key engine parameters are within 2.25% against NPSS data, serves as the basis of PCEUM. Meanwhile, two classic sample-based uncertainty quantification approaches, (i.e. Monte-Carlo simulations (MCS), Latin hypercube sampling (LHS)) and a least angle regression (LARS) based PCE are set as benchmarks. Numerical simulations using publicly available manufacturing tolerance statistics are conducted at take-off states for the tested engine on a desktop computer. Results show that the proposed PCEUM costs only 47.06s at the expense of 200 samples to obtain stable probability distributions for interested engine parameters, e.g. thrust, whose errors of mean and standard deviation compared with MCS at 100,000 samples are within 0.01% and 1%, respectively. While the computational burden of MCS, LHS, and LARS are 854.63s at 9,000 samples, 250.13s at 3,000 samples, and 54.16s at 200 samples at the same precision level, respectively. It means that compared to the latter three methods, PCEUM can save 94.5%, 81.2%, and 13.1% of the simulation time, respectively. Hence, the proposed model is verified regarding both the accuracy and speed for uncertainty assessment, which provides a promising solution for both conventional gas turbine engines and future aero-propulsion systems.

New insights into the spatial distribution of particle number concentrations by applying non-parametric land use regression modelling

DNA microarray SNP associations with clinical efficacy and side effects of domperidone treatment for gastroparesis

Adaptive lasso penalized generalized linear models (GLMs) are a powerful tool for analyzing the high-dimensional sparse data where the classical linear or normal assumption is not met. In non-distributed environments, the estimation problem of adaptive lasso penalized GLMs is often solved by the coordinate descent based algorithm developed in Friedman, Hastie, and Tibshirani (2010), which has been well implemented in the R package glmnet. However, when applied to distributed big data, this algorithm is usually inflexible or even infeasible due to its non-parallel implementation, especially when the communication costs between the central and local machines are expensive, or the storage and computing capabilities of the central machine are insufficient. In this paper, we propose a new method, QAGLM-alasso, for the adaptive lasso penalized GLMs problem in distributed big data by applying the quadratic approximation representation of GLMs, and further develop a path-following algorithm for its estimation based on the Least Angle Regression (LARS). Theoretical analyses show that, under mild regularity conditions, the QAGLM-alasso enjoys the oracle property, and the obtained estimator is asymptotically equivalent to the original adaptive lasso. Simulation studies demonstrate that the new algorithm has similar estimation accuracy with glmnet, but is significantly faster than glmnet in distributed environments. We further illustrate the practical performance of the proposed method by analyzing a supersymmetric (SUSY) benchmark data set.

Medical image fusion based on sparse representation of classified image patches

It is important to reduce intra prediction error for efficient image coding. However, as the existing methods are based on static structure model, there is no guarantee of their prediction efficiency for images whose structure model are unknown. To remedy the problem of existing methods, we formulate a linear prediction design problem with the goal of minimizing prediction error by placing sparsity constraints on the prediction coefficients. To solve the predictor design problem, we propose a novel method that extends Least Angle Regression(LARS). Coding gain of 1.01% to 2.24% is achieved over HM16.7.

ABSTRACTA new speech enhancement method using sparse reconstruction of the log-spectra is presented. Similar to the traditional sparse coding methods, the proposed algorithm makes use of the least angle regression (LARS) with a coherence criterion (LARC) algorithm to reconstruct the log power spectrum of clean speech. However, a new stopping criterion is introduced to allow the LARC algorithm to adapt to various background noise environments. In addition, a modified two-step noise reduction with a log-MMSE filter is applied which solves the bias of estimated a-priori signal-to-noise ratio (SNR). A notable improvement in the proposed algorithm over traditional speech enhancement methods is its adaptability to the changes in the SNR of noisy speech. The performance of the proposed algorithm is evaluated using standard measures based on a large set of speech and noise signals. The results show that a significant improvement is achieved compared to traditional approaches, especially in non-stationary noise en...