Sparse partial least-squares regression and its applications to high-throughput data analysis

Mixed-norm partial least squares

Correction for ‘Accuracy improvement of iron ore analysis using laser-induced breakdown spectroscopy with a hybrid sparse partial least squares and least-squares support vector machine model’ by Y. M. Guo et al., J. Anal. At. Spectrom., 2018, 33, 1330–1335, DOI: 10.1039/C8JA00119G.

Most nutritional epidemiology studies investigating trends between diet and heart disease use outcome-independent dimension reduction methods, like principal component analysis, to create dietary patterns. While these methods construct patterns that describe important aspects of food consumption, these patterns are not inherently related to heart disease. Incorporating disease data into the pattern construction offers the possibility of more concisely summarizing the most disease-related foods. Sparse partial least squares (SPLS), one such method, was found to have favorable interpretation and prediction properties in the continuous outcome setting; while selecting a subset of relevant foods, it constructed a few dietary patterns that were correlated with BMI while also capturing variation in diet composition. These results were validated with simulated data. We propose incorporating SPLS into the Cox proportional hazards model to analyze a right-censored survival outcome. We hypothesized that this method would inherit the beneficial parsimony properties seen in the continuous setting, and we assessed whether this proposed method could use the most relevant covariates to create a few patterns that were associated with a survival outcome. While the proposed method targets covariate-level sparsity (i.e. variable selection), one competitor method exists that integrates pattern-level parsimony and partial least squares (PLS) in the Cox model, but it imposes more model parameters than the proposed method. We compared the variable selection, pattern selection, and predictive performance of four survival methods (Lasso, PLS, competitor sparse PLS, and proposed SPLS) via a simulation study. Simulation settings were informed in part by the Multi-Ethnic Study of Atherosclerosis (MESA), which has detailed food frequency questionnaire data on a large multi-ethnic population-based sample (6814 participants aged 45-84), as well as subsequent cardiovascular disease follow-up for over 15 years. In most studied simulation settings, the proposed method selected all 9 relevant predictors and the fewest number of irrelevant predictors (of 15) while creating a similar number of patterns and maintaining predictive ability of the outcome. In the setting most comparable to MESA, PLS chose all 24 predictors (by default) and 3.4 patterns (C-statistic=0.90), the competitor SPLS selected 21.1 predictors and 4.4 patterns (C-statistic=0.91), Lasso chose 16.4 predictors (C-statistic=0.91), and the proposed SPLS selected 11.7 predictors and 4.3 patterns (C-statistic=0.91), on average. We will also present an analysis of a coronary event in MESA using these four survival methods. In conclusion, we propose that using methods like SPLS to summarize food intake can create more heart disease-tailored dietary patterns that can complement the current nutritional epidemiology literature.

In the supervised high dimensional settings with a large number of variables and a low number of individuals, variable selection allows a simpler interpretation and more reliable predictions. That subspace selection is often managed with supervised tools when the real question is motivated by variable prediction. We propose a partial least square (PLS) based method, called data‐driven sparse PLS (ddsPLS), allowing variable selection both in the covariate and the response parts using a single hyperparameter per component. The subspace estimation is also performed by tuning a number of underlying parameters. The ddsPLS method is compared with existing methods such as classical PLS and two well established sparse PLS methods through numerical simulations. The observed results are promising both in terms of variable selection and prediction performance. This methodology is based on new prediction quality descriptors associated with the classicaland, and uses bootstrap sampling to tune parameters and select an optimal regression model.

The well‐known nonlinear iterative partial least squares (NIPALS) algorithm is commonly used for computation of components in partial least squares regression (PLSR) with orthogonalized score vectors. Based on generalized inverse formalism, Pell et al.[1] have recently claimed that the NIPALS results are inconsistent with respect to model spaces for residual‐based outlier detection and prediction purposes. This theoretically important result is supported by the present paper, where it is also shown that a simple re‐interpretation of the results from the NIPALS algorithm solves the problem. This is valid for cases with both one and several response variables (PLS1 and PLS2). It is also shown, however, that the price to pay for this solution is that the latent variables in the model will no longer be completely independent of the residual noise. Since the original PLSR model with non‐orthogonal score vectors does not have the same inconsistency, a method for orthogonalization of this model is also included. Copyright © 2008 John Wiley & Sons, Ltd.

The statistical methods for variable selection and prediction could be challenging when missing covariates exist. Although multiple imputation (MI) is a universally accepted technique for solving missing data problem, how to combine the MI results for variable selection is not quite clear, because different imputations may result in different selections. The widely applied variable selection methods include the sparse partial least-squares (SPLS) method and the penalized least-squares method, e.g. the elastic net (ENet) method. In this paper, we propose an MI-based weighted elastic net (MI-WENet) method that is based on stacked MI data and a weighting scheme for each observation in the stacked data set. In the MI-WENet method, MI accounts for sampling and imputation uncertainty for missing values, and the weight accounts for the observed information. Extensive numerical simulations are carried out to compare the proposed MI-WENet method with the other competing alternatives, such as the SPLS and ENet. In addition, we applied the MI-WENet method to examine the predictor variables for the endothelial function that can be characterized by median effective dose (ED50) and maximum effect (Emax) in an ex-vivo phenylephrine-induced extension and acetylcholine-induced relaxation experiment.

As a laboratory proximal sensing technique, the capability of visible and near-infrared (Vis-NIR) diffused reflectance spectroscopy with partial least squares (PLS) regression to determine soil properties has previously been demonstrated. However, the evaluation of the soil phosphorus (P) content—a major nutrient constraint for crop production in the tropics—is still a challenging task. PLS regression with waveband selection can improve the predictive ability of a calibration model, and a genetic algorithm (GA) has been widely applied as a suitable method for selecting wavebands in laboratory calibrations. To develop a laboratory-based proximal sensing method, this study investigated the potential to use GA-PLS regression analyses to estimate oxalate-extractable P in upland and lowland soils from laboratory Vis-NIR reflectance data. In terms of predictive ability, GA-PLS regression was compared with iterative stepwise elimination PLS (ISE-PLS) regression and standard full-spectrum PLS (FS-PLS) regression using soil samples collected in 2015 and 2016 from the surface of upland and lowland rice fields in Madagascar (n = 103). Overall, the GA-PLS model using first derivative reflectance (FDR) had the best predictive accuracy (R2 = 0.796) with a good prediction ability (residual predictive deviation (RPD) = 2.211). Selected wavebands in the GA-PLS model did not perfectly match wavelengths of previously known absorption features of soil nutrients, but in most cases, the selected wavebands were within 20 nm of previously known wavelength regions. Bootstrap procedures (N = 10,000 times) using selected wavebands also confirmed the improvements in accuracy and robustness of the GA-PLS model compared to those of the ISE-PLS and FS-PLS models. These results suggest that soil oxalate-extractable P can be predicted from Vis-NIR spectroscopy and that GA-PLS regression has the advantage of tuning optimum bands for PLS regression, contributing to a better predictive ability.

Thermal risk modeling and safety optimization of an arylamine diazo reaction based on PLS algorithm

Genomic selection in the French Lacaune dairy sheep breed

BackgroundEven though R is one of the most commonly used statistical computing environments, it lacks a graphical user interface (GUI) that appeals to students, researchers, lecturers, and practitioners in medicine and pharmacy for conducting standard data analytics. Current GUIs built on top of R, such as EZR and R-Commander, aim to facilitate R coding and visualization, but most of the functionalities are still accessed through a command-line interface (CLI). To assist practitioners of medicine and pharmacy and researchers to run most routines in fundamental statistical analysis, we developed an interactive GUI; i.e., MEPHAS, to support various web-based systems that are accessible from laptops, workstations, or tablets, under Windows, macOS (and IOS), or Linux. In addition to fundamental statistical analysis, advanced statistics such as the extended Cox regression and dimensional analyses including partial least squares regression (PLS-R) and sparse partial least squares regression (SPLS-R), are also available in MEPHAS.ResultsMEPHAS is a web-based GUI (https://alain003.phs.osaka-u.ac.jp/mephas/) that is based on a shiny framework. We also created the corresponding R package mephas (https://mephas.github.io/). Thus far, MEPHAS has supported four categories of statistics, including probability, hypothesis testing, regression models, and dimensional analyses. Instructions and help menus were accessible during the entire analytical process via the web-based GUI, particularly advanced dimensional data analysis that required much explanation. The GUI was designed to be intuitive for non-technical users to perform various statistical functions, e.g., managing data, customizing plots, setting parameters, and monitoring real-time results, without any R coding from users. All generated graphs can be saved to local machines, and tables can be downloaded as CSV files.ConclusionMEPHAS is a free and open-source web-interactive GUI that was designed to support statistical data analyses and prediction for medical and pharmaceutical practitioners and researchers. It enables various medical and pharmaceutical statistical analyses through interactive parameter settings and dynamic visualization of the results.

Role of Excipients on Solid-State Properties of Piroxicam During Processing

Canonical correlation analysis (CCA) and partial least squares (PLS) are powerful multivariate methods for capturing associations across 2 modalities of data (e.g., brain and behavior). However, when the sample size is similar to or smaller than the number of variables in the data, standard CCA and PLS models may overfit, i.e., find spurious associations that generalize poorly to new data. Dimensionality reduction and regularized extensions of CCA and PLS have been proposed to address this problem, yet most studies using these approaches have some limitations. This work gives a theoretical and practical introduction into the most common CCA/PLS models and their regularized variants. We examine the limitations of standard CCA and PLS when the sample size is similar to or smaller than the number of variables. We discuss how dimensionality reduction and regularization techniques address this problem and explain their main advantages and disadvantages. We highlight crucial aspects of the CCA/PLS analysis framework, including optimizing the hyperparameters of the model and testing the identified associations for statistical significance. We apply the described CCA/PLS models to simulated data and real data from the Human Connectome Project and Alzheimer’s Disease Neuroimaging Initiative (both of n > 500). We use both low- and high-dimensionality versions of these data (i.e., ratios between sample size and variables in the range of ∼1–10 and ∼0.1–0.01, respectively) to demonstrate the impact of data dimensionality on the models. Finally, we summarize the key lessons of the tutorial.

Algorithms for partial least squares (PLS) modelling are placed into a sound theoretical context focusing on numerical precision and computational efficiency. NIPALS and other PLS algorithms that perform deflation steps of the predictors (X) may be slow or even computationally infeasible for sparse and/or large‐scale data sets. As alternatives, we develop new versions of the Bidiag1 and Bidiag2 algorithms. These include full reorthogonalization of both score and loading vectors, which we consider to be both necessary and sufficient for numerical precision. Using a collection of benchmark data sets, these 2 new algorithms are compared to the NIPALS PLS and 4 other PLS algorithms acknowledged in the chemometrics literature. The provably stable Householder algorithm for PLS regression is taken as the reference method for numerical precision. Our conclusion is that our new Bidiag1 and Bidiag2 algorithms are the methods of choice for problems where both efficiency and numerical precision are important. When efficiency is not urgent, the NIPALS PLS and the Householder PLS are also good choices. The benchmark study shows that SIMPLS gives poor numerical precision even for a small number of factors. Further, the nonorthogonal scores PLS, direct scores PLS, and the improved kernel PLS are demonstrated to be numerically less stable than the best algorithms. Prototype MATLAB codes are included for the 5 PLS algorithms concluded to be numerically stable on our benchmark data sets. Other aspects of PLS modelling, such as the evaluation of the regression coefficients, are also analyzed using techniques from numerical linear algebra.

Objective. Partial Least Squares (PLS) regression is a suitable linear decoder model for correlated and high dimensional neural data. This algorithm has been widely used in the application of brain–computer interface (BCI) for the decoding of motor parameters. PLS does not consider nonlinear relations between brain signal features. The nonlinear version of PLS that considers a nonlinear relationship between the latent variables has not been proposed for the decoding of intracranial data. This nonlinear model may cause overfitting in some cases due to a larger number of free parameters. In this paper, we develop a new version of nonlinear PLS, namely nonlinear sparse PLS (NLS PLS) and test it in BCI applications. Approach. In motor related BCI systems, improving the decoding accuracy of both kinetic and kinematic parameters of movement is crucial. To do this, two BCI datasets were chosen to decode the force amplitude and position of hand trajectory using the nonlinear and sparse versions of PLS algorithm. In our new NLS PLS method, we considered a polynomial relationship between the latent variables and used the lasso penalization in the latent space to avoid overfitting and to improve the decoding accuracy. Main results. Some linear and nonlinear based PLS models and our new proposed method, NLS PLS, were applied to the two datasets. According to our results, significant improvement from the NLS PLS method is confirmed over other methods. Our results show that nonlinear PLS outperforms generic PLS in the force decoding but it has lower accuracy in the hand trajectory decoding because of high dimensional feature space. By using lasso penalization, we presented a sparse nonlinear PLS-based model that outperforms generic PLS in both datasets and improves the coefficient of determination, 34% in the force decoding and 10% in the hand trajectory decoding. Significance. We constructed a simple PLS-based model that considers a nonlinear relationship between features and it is also robust to overfitting because of using the lasso penalty in the latent space. This model is suitable for a high dimensional and correlated datasets, like intracranial data and can improve the accuracy of estimation.

Distributed partial least squares based residual generation for statistical process monitoring