High-Dimensional LASSO-Based Computational Regression Models: Regularization, Shrinkage, and Selection

The traditional group sparse optimization method can simultaneously achieve the channel selection and classification for the motor imagery electroencephalogram (EEG) signals, but it doesn’t consider the spatial structure information between the electrode channels. Combining the group sparsity and spatial smoothness of EEG signals, a new EEG classification model is proposed, which is an improvement of group least absolute shrinkage and selection operator (LASSO). We call it fused group LASSO. First, group LASSO is used to model the group sparsity of EEG signals, the features of the same channel are assigned the same weights. Then, based on group LASSO, channel weights are regularized by total variation norm (TV-norm), which constrains the weights of adjacent channels to the same or similar, thereby the spatial smoothness modeling of EEG signals can be achieved. Using the primal-dual theory, an optimization algorithm for the new model is given. In order to verify the effectiveness of the new model, experiments were performed on two public brain-computer interface (BCI) competition data sets and one self-collected data set. Compared with the existing sparse optimization methods, the proposed method has achieved the highest average classification accuracy of 79.24%, 86.64% and 81.09%, respectively, and with better physiological interpretability. Compared with spatial filtering methods with smooth constraints, the proposed method realized global spatial smooth in a data-driver manner, and achieved the highest average classification accuracy of 84.96% in two competition data sets. All the experimental results showed that the proposed method can significantly improve the performance of BCI systems.

We consider the problem of predicting survival times of cancer patients from the gene expression profiles of their tumor samples via linear regression modeling of log-transformed failure times. The partial least squares (PLS) and least absolute shrinkage and selection operator (LASSO) methodologies are used for this purpose where we first modify the data to account for censoring. Three approaches of handling right censored data-reweighting, mean imputation, and multiple imputation-are considered. Their performances are examined in a detailed simulation study and compared with that of full data PLS and LASSO had there been no censoring. A major objective of this article is to investigate the performances of PLS and LASSO in the context of microarray data where the number of covariates is very large and there are extremely few samples. We demonstrate that LASSO outperforms PLS in terms of prediction error when the list of covariates includes a moderate to large percentage of useless or noise variables; otherwise, PLS may outperform LASSO. For a moderate sample size (100 with 10,000 covariates), LASSO performed better than a no covariate model (or noise-based prediction). The mean imputation method appears to best track the performance of the full data PLS or LASSO. The mean imputation scheme is used on an existing data set on lung cancer. This reanalysis using the mean imputed PLS and LASSO identifies a number of genes that were known to be related to cancer or tumor activities from previous studies.

Abstrak
 Pertumbuhan ekonomi akhir-akhir ini menjadi lebih penting karena implementasinya secara luas, konsep pertumbuhan ekonomi merupakan ukuran penilaian ekonomi suatu negara atau regional. Data pertumbuhan ekonomi dalam penelitian ini yang diukur dengan Produk Domestik Regional Bruto (PDRB) rentan terhadap masalah asumsi klasik. Salah satunya adalah tidak adanya multikolinearitas yaitu korelasi atau hubungan yang tinggi diantara variabel independen. Multikolinearitas menjadi masalah dalam analisis regresi, terutama pada Ordinary Least Square (OLS) karena menyebabkan estimasi koefisien regresi menjadi tidak efisien. Salah satu metode untuk mengatasi multikolinearitas menggunakan Least Absolute Shrinkage and Selection Operator (LASSO). LASSO adalah metode regresi berganda yang digunakan untuk shrinkage yaitu menyusutkan koefisien taksiran mendekati angka nol dan selection operator yaitu menyeleksi variabel-variabel independen sehingga menghasilkan model dengan variabel terbaik karena kendala itu, LASSO dapat mengkecilkan koefisien menuju nol atau mengaturnya dengan tepat nol sehingga dapat melakukan pemilihan variabel juga. Berdasarkan Variance Inflation Factor (VIF), terdapat korelasi yang tinggi antar variabel independen, sehingga terdapat multikolinearitas pada data pertumbuhan ekonomi Provinsi Maluku 1999-2021 jika kita menggunakan OLS. Dalam penelitian ini, LASSO mengkecilkan dua koefisien penduga variabel independen menjadi tepat nol, sehingga variabel-variabel tersebut dianggap tidak memiliki pengaruh yang signifikan terhadap model.

Features and interaction between features of liver disease is of great significance for the classification of liver disease. Based on least absolute shrinkage and selection operator (LASSO) and interaction LASSO, the generalized interaction LASSO model is proposed in this paper for liver disease classification and compared with other methods. Firstly, the generalized interaction logistic classification model was constructed and the LASSO penalty constraints were added to the interactive model parameters. Then the model parameters were solved by an efficient alternating directions method of multipliers (ADMM) algorithm. The solutions of model parameters were sparse. Finally, the test samples were fed to the model and the classification results were obtained by the largest statistical probability. The experimental results of liver disorder dataset and India liver dataset obtained by the proposed methods showed that the coefficients of interaction features of the model were not zero, indicating that interaction features were contributive to classification. The accuracy of the generalized interaction LASSO method is better than that of the interaction LASSO method, and it is also better than that of traditional pattern recognition methods. The generalized interaction LASSO method can also be popularized to other disease classification areas.

BackgroundFactors associated with development of medication-overuse headache (MOH) in migraine patients are not fully understood, but with respect to prevention, the ability to predict the onset of MOH is clinically important. The aims were to examine if personality characteristics, disability and physical activity level are associated with the onset of MOH in a group of migraine patients and explore to which extend these factors combined can predict the onset of MOH.MethodsThe study was a single-center prospective observational study of migraine patients. At inclusion, all patients completed questionnaires evaluating 1) personality (NEO Five-Factor Inventory), 2) disability (Migraine Disability Assessment), and 3) physical activity level (Physical Activity Scale 2.1). Diagnostic codes from patients’ electronic health records confirmed if they had developed MOH during the study period of 20 months. Analyses of associations were performed and to identify which of the variables predict onset MOH, a multivariable least absolute shrinkage and selection operator (LASSO) logistic regression model was fitted to predict presence or absence of MOH.ResultsOut of 131 participants, 12 % (n=16) developed MOH. Migraine disability score (OR=1.02, 95 % CI: 1.00 to 1.04), intensity of headache (OR=1.49, 95 % CI: 1.03 to 2.15) and headache frequency (OR=1.02, 95 % CI: 1.00 to 1.04) were associated with the onset of MOH adjusting for age and gender. To identify which of the variables predict onset MOH, we used a LASSO regression model, and evaluating the predictive performance of the LASSO-mode (containing the predictors MIDAS score, MIDAS-intensity and –frequency, neuroticism score, time with moderate physical activity, educational level, hours of sleep daily and number of contacts to the headache clinic) in terms of area under the curve (AUC) was weak (apparent AUC=0.62, 95% CI: 0.41-0.82).ConclusionDisability, headache intensity and frequency were associated with the onset of MOH whereas personality and the level of physical activity were not. The multivariable LASSO model based on personality, disability and physical activity is applicable despite moderate study size, however it can be considered as a weak classifier for discriminating between absence and presence of MOH.

Undernutrition among children under the age of five is a major public health concern, especially in developing countries. This study aimed to use machine learning (ML) algorithms to predict undernutrition and identify its associated factors. Secondary data analysis of the 2017 Multiple Indicator Cluster Survey (MICS) was performed using R and Python. The main outcomes of interest were undernutrition (stunting: height-for-age (HAZ) < -2 SD; wasting: weight-for-height (WHZ) < -2 SD; and underweight: weight-for-age (WAZ) < -2 SD). Seven ML algorithms were trained and tested: linear discriminant analysis (LDA), logistic model, support vector machine (SVM), random forest (RF), least absolute shrinkage and selection operator (LASSO), ridge regression, and extreme gradient boosting (XGBoost). The ML models were evaluated using the accuracy, confusion matrix, and area under the curve (AUC) receiver operating characteristics (ROC). In total, 8564 children were included in the final analysis. The average age of the children was 926 days, and the majority were females. The weighted prevalence rates of stunting, wasting, and underweight were 17%, 7%, and 12%, respectively. The accuracies of all the ML models for wasting were (LDA: 84%; Logistic: 95%; SVM: 92%; RF: 94%; LASSO: 96%; Ridge: 84%, XGBoost: 98%), stunting (LDA: 86%; Logistic: 86%; SVM: 98%; RF: 88%; LASSO: 86%; Ridge: 86%, XGBoost: 98%), and for underweight were (LDA: 90%; Logistic: 92%; SVM: 98%; RF: 89%; LASSO: 92%; Ridge: 88%, XGBoost: 98%). The AUC values of the wasting models were (LDA: 99%; Logistic: 100%; SVM: 72%; RF: 94%; LASSO: 99%; Ridge: 59%, XGBoost: 100%), for stunting were (LDA: 89%; Logistic: 90%; SVM: 100%; RF: 92%; LASSO: 90%; Ridge: 89%, XGBoost: 100%), and for underweight were (LDA: 95%; Logistic: 96%; SVM: 100%; RF: 94%; LASSO: 96%; Ridge: 82%, XGBoost: 82%). Age, weight, length/height, sex, region of residence and ethnicity were important predictors of wasting, stunting and underweight. The XGBoost model was the best model for predicting wasting, stunting, and underweight. The findings showed that different ML algorithms could be useful for predicting undernutrition and identifying important predictors for targeted interventions among children under five years in Ghana.

An estimation method known as least absolute shrinkage and selection operator (LASSO) or ℓ1-regularized LS estimation has been found to perform well in a number of applications. In this paper, we use the majorize-minimize method to develop an algorithm for minimizing the LASSO objective function, which is the sum of a linear LS objective function plus an ℓ1 penalty term. The proposed algorithm, which we call the LASSO estimation via majorization-minimization (LMM) algorithm, is straightforward to implement, parallelizable, and guaranteed to produce LASSO objective function values that monotonically decrease. In addition, we formulate an extension of the LMM algorithm for reconstructing ground penetrating radar (GPR) images, that is much faster than the standard LMM algorithm and utilizes significantly less memory. Thus, the GPR specific LMM (GPR-LMM) algorithm is able to accommodate the big data associated with GPR imaging. We compare our proposed algorithms to the state-of-the-art ℓ1-regularized LS algorithms using a time and space complexity analysis. The GPR-LMM greatly outperforms the competing algorithms in terms of the performance metrics we considered. In addition, the reconstruction results of the standard LMM and GPR-LMM algorithms are evaluated using both simulated and real GPR data.

This paper proposes appearance-based localization using online vision images collected from an omnidirectional camera attached on a mobile robot or a vehicle. Our approach builds on a combination of the group Least Absolute Shrinkage and Selection Operator (LASSO) and the extended Kalman filter (EKF). Fast Fourier transform (FFT) and Histogram are extracted from omni-directional images, the features of which are selected via the group LASSO regression. The EKF takes the output of the group LASSO regression based first-stage localization as the observation. The indoor experimental results demonstrate the effectiveness of our approach.

Variable selection in genome-wide association studies can be a daunting task and statistically challenging because there are more variables than subjects. We propose an approach that uses principal-component analysis (PCA) and least absolute shrinkage and selection operator (LASSO) to identify gene-gene interaction in genome-wide association studies. A PCA was used to first reduce the dimension of the single-nucleotide polymorphisms (SNPs) within each gene. The interaction of the gene PCA scores were placed into LASSO to determine whether any gene-gene signals exist. We have extended the PCA-LASSO approach using the bootstrap to estimate the standard errors and confidence intervals of the LASSO coefficient estimates. This method was compared to placing the raw SNP values into the LASSO and the logistic model with individual gene-gene interaction. We demonstrated these methods with the Genetic Analysis Workshop 16 rheumatoid arthritis genome-wide association study data and our results identified a few gene-gene signals. Based on our results, the PCA-LASSO method shows promise in identifying gene-gene interactions, and, at this time we suggest using it with other conventional approaches, such as generalized linear models, to narrow down genetic signals.

Non‐preference‐based patient‐reported outcome measures (PROMs) are popular in health outcomes research. These measures, however, cannot be used to estimate health state utilities, limiting their usefulness for economic evaluations. Mapping PROMs to a multi‐attribute utility instrument is one solution. While mapping is commonly conducted using econometric techniques, failing to specify the complex interactions between variables may lead to inaccurate prediction of utilities, resulting in inaccurate estimates of cost‐effectiveness and suboptimal funding decisions. These issues can be addressed using machine learning. This paper evaluates the use of machine learning as a mapping tool. We adopt a comprehensive approach to compare six machine learning techniques with eight econometric techniques to map the Patient‐Reported Outcomes Measurement Information System Global Health 10 (PROMIS‐GH10) to the EuroQol five dimensions (EQ‐5D‐5L). Using data collected from 2015 Australians, we find the least absolute shrinkage and selection operator (LASSO) model out‐performed all machine learning techniques and the adjusted limited dependent variable mixture model (ALDVMM) out‐performed all econometric techniques, with the LASSO performing better than ALDVMM. The variable selection feature of LASSO was then used to enhance the performance of the ALDVMM in a hybrid model. Our analysis identifies the potential benefits and challenges of using machine learning techniques for mapping and offers important insights for future research.

Alfalfa (Medicago sativa L.) crop vigor and yield characterization using high-resolution aerial multispectral and thermal infrared imaging technique

BackgroundEULAR guideline in rheumatoid arthritis (RA) recommended the primary failure of the first-line conventional synthetic modifying anti-rheumatic drugs (csDMARDs) patients switching to tumor necrosis factor alpha inhibitors (TNFi) [1]. Nevertheless,...

Background This study aims to evaluate the risk factors associated with untreated hypertension and develop and internally validate untreated risk nomograms in patients with hypertension among primary health care of less developed Northwest China. Methods A total of 895 eligible patients with hypertension in primary health care of less developed Northwest China were divided into a training set (n = 626) and a validation set (n = 269). Untreated hypertension was defined as not taking antihypertensive medication during the past two weeks. Using least absolute shrinkage and selection operator (LASSO) regression model, we identified the optimized risk factors of nontreatment, followed by establishment of a prediction nomogram. The discriminative ability, calibration, and clinical usefulness were determined using the area under the receiver operating characteristic curve (AUC), calibration curve, and decision analysis. The results were assessed by internal validation in the validation set. Results Five independent risk factors were derived from LASSO regression model and entered into the nomogram: age, herdsman, family income per member, altitude of habitation, and comorbidity. The nomogram displayed a robust discrimination with an AUC of 0.859 (95% confidence interval: 0.812–0.906) and good calibration. The nomogram was clinically useful when the intervention was decided at the untreated possibility threshold of 7% to 91% in the decision curve analysis. Results were confirmed by internal validation. Conclusions Our nomogram showed favorable predictive accuracy for untreated hypertension in primary health care of less developed Northwest China and might help primary health care assess the risk of nontreatment in patients with hypertension.

Forest above-ground biomass estimation based on strongly collinear variables derived from airborne laser scanning data

We present a framework for hyperspectral image (HSI) analysis validation, specifically abundance fraction estimation based on HSI measurements of water soluble dye mixtures printed on microarray chips. In our work we focus on the performance of two algorithms, the Least Absolute Shrinkage and Selection Operator (LASSO) and the Spatial LASSO (SPLASSO). The LASSO is a well known statistical method for simultaneously performing model estimation and variable selection. In the context of estimating abundance fractions in a HSI scene, the “sparse” representations provided by the LASSO are appropriate as not every pixel will be expected to contain every endmember. The SPLASSO is a novel approach we introduce here for HSI analysis which takes the framework of the LASSO algorithm a step further and incorporates the rich spatial information which is available in HSI to further improve the estimates of abundance. In our work here we introduce the dye mixture platform as a new benchmark data set for hyperspectral biomedical image processing and show our algorithm’s improvement over the standard LASSO.