Re‐interpretation of NIPALS results solves PLSR inconsistency problem

Sparse partial least-squares regression and its applications to high-throughput data analysis

Partial least square regression (PLSR) is a reference statistical model in chemometrics. In agronomy, it is used to predict components (response variables y) of chemical composition of vegetal materials from spectral near infrared (NIR) data X collected from spectrometers. PLSR reduces the dimension of the spectral data X by defining vectors that are then used as latent variables (LVs) in a multiple linear model. One difficulty is to determine the relevant dimensionality (number of LVs) for the given data. This step can be very time consuming when many datasets have to be processed and/or the datasets are frequently updated. The paper focuses on an alternative, bypassing the determination of the PLSR dimensionality and allowing for automatizing the predictions. The strategy uses ensemble learning methods, such as averaging or stacking the predictions of a set of PLSR models with different dimensionalities. The paper presents various methods of PLSR averaging and stacking and compares their performances to the usual PLSR on six real datasets on different types of forages. The main finding of the study was the overall superiority of the averaging methods compared to the usual PLSR. We therefore believe that such methods can be recommended to analyze NIR data on forages.

The resistance of barley (Hordeum vulgare L.) to Rhynchosporium secalis (scald) has been investigated in two crosses between the susceptible cv. 'Ingrid' and two resistant Ethiopian landraces, 'Steudelli' and 'Jet'. Doubled haploids were inoculated in replicated tests using two isolates of R. secalis, '4004' and 'WRS1872'. Expression of resistance differed widely between replicated tests. AFLP, SSR and RFLP markers were used to develop chromosome maps. Results have been analysed using partial least squares regression (PLSR) and interval mapping. In PLSR the major covariance structures or 'latent variables' between X (markers) and Y (isolates, tests) are modelled as principal components and their optimal number determined by cross-validation. In 'Steudelli' two QTL were detected, one on each of chromosomes 3H and 7H, in 4 out of 5 tests, while in 'Jet' only one (different) allele at the 3H locus was found. The validated R(2) varied between 11.0% and 64.9% in the replicated tests with '4004'.With isolate 'WRS1872' the 7H locus and another 3H locus were detected. By interval mapping the QTL detected were less stable and generally gave lower R(2) values than PLSR. PLSR does not depend on maps, but interval mapping based on values predicted by PLSR had R(2) around 90%. It is suggested that PLSR may be a useful tool in QTL analysis.

The partial least squares regression (PLSR) algorithms of Wold and Martens provide alternative, powerful tools for handling ill-conditioned multivariate regression data, see e.g. Martens and Nees (1989) and Hoskuldson (1996) for overviews of the fundamental multivariate calibration concept, in which are presented the above two slightly different versions of the PLSR method. Both algorithms assume collinear regressor variables generated by underlying latent variables, and although they are different with respect to the score and loading matrices involved, they give identical predictors. There is certainly no practical need for yet another algorithm that provide the same predictor as the well-known algorithms of Wold and Martens. However, these algorithms, as presented in the chemometrical literature, make no use of the dynamic systems theory available for readers from the control community, and the main aim of the present paper is therefore to provide a predictor derivation at first especially for this category of readers. This actually also results in a simplified version of the Martens prediction algorithm making use of the so-called loading weight vectors only, which would appear to be of general interest (see Esbensen (2000) for a discussion of loadings and loading weights). The score vectors, which are also essential elements of chemometrics, are easily computed once a preliminary predictor is found. The new algorithm has a lot in common with an algorithm developed by Helland (1988), and a similar algorithm by Di Ruscio (2000). Again, the reason behind the present modifications is mainly didactic. The key step in the simplification is the use of an explicit latent variables model, which facilitates an early introduction of the Helland predictor form using only loading weight vectors. The present exposition, we believe, constitute a novel, easy, and complete introduction to the prediction aspect of multivariate calibration.

Partial least square (PLS) methods (also sometimes called projection to latent structures) relate the information present in two data tables that collect measurements on the same set of observations. PLS methods proceed by deriving latent variables which are (optimal) linear combinations of the variables of a data table. When the goal is to find the shared information between two tables, the approach is equivalent to a correlation problem and the technique is then called partial least square correlation (PLSC) (also sometimes called PLS-SVD). In this case there are two sets of latent variables (one set per table), and these latent variables are required to have maximal covariance. When the goal is to predict one data table the other one, the technique is then called partial least square regression. In this case there is one set of latent variables (derived from the predictor table) and these latent variables are required to give the best possible prediction. In this paper we present and illustrate PLSC and PLSR and show how these descriptive multivariate analysis techniques can be extended to deal with inferential questions by using cross-validation techniques such as the bootstrap and permutation tests.

Role of Excipients on Solid-State Properties of Piroxicam During Processing

Simple multilinear methods, such as partial least squares regression (PLSR), are effective at interrelating dynamic, multivariate datasets of cell–molecular biology through high-dimensional arrays. However, data collected in vivo are more difficult, because animal-to-animal variability is often high, and each time-point measured is usually a terminal endpoint for that animal. Observations are further complicated by the nesting of cells within tissues or tissue sections, which themselves are nested within animals. Here, we introduce principled resampling strategies that preserve the tissue-animal hierarchy of individual replicates and compute the uncertainty of multidimensional decompositions applied to global averages. Using molecular–phenotypic data from the mouse aorta and colon, we find that interpretation of decomposed latent variables (LVs) changes when PLSR models are resampled. Lagging LVs, which statistically improve global-average models, are unstable in resampled iterations that preserve nesting relationships, arguing that these LVs should not be mined for biological insight. Interestingly, resampling is less discriminatory for multidimensional regressions of in vitro data, where replicate-to-replicate variance is sufficiently low. Our work illustrates the challenges and opportunities in translating systems-biology approaches from cultured cells to living organisms. Nested resampling adds a straightforward quality-control step for interpreting the robustness of in vivo regression models.

The objective of this study was to compare the predictions of the protein contents and hardness values by partial least squares regression (PLSR) and principal components regression (PCR) models for bulk samples of Canadian wheat, which were obtained from different locations and crop years. Wheat samples of Canada Western Red Spring (CWRS), Canada Western Hard White Spring (CWHWS), Canada Western Soft White Spring (CWSWS), and Canada Prairie Spring Red (CPSR) classes were obtained from nearby agricultural farms in the main wheat growing locations in the Provinces of Alberta, Saskatchewan, and Manitoba from 2007, 2008, and 2009 crop years. Wheat samples were conditioned to moisture levels of 13, 16, and 19 % (wet basis) and pooled together for developing the regression models. A database of the near-infrared (NIR) hyperspectral image cubes of bulk samples of wheat classes was created in the wavelength region of 960–1,700 nm with 10 nm intervals. Reference protein contents and hardness values were determined using the Dumatherm method and single kernel characterization system (SKCS), respectively. A tenfold cross-validation was used for the ten-factor partial least squares regression (PLSR) and principal components regression (PCR) models for prediction purposes. Prediction performances of regression models were assessed by calculating the estimated mean square errors of prediction (MSEP), standard error of cross-validation (SECV), and correlation coefficient (r). Using the full data set in the protein prediction study, the ten-component PLSR model gave 1.76, 1.33, and 0.68 for the estimated MSEP, SECV, and r, respectively, which were better than the results for the ten-component PCR model (2.02, 1.42, and 0.62, respectively). For the hardness prediction, the estimated MSEP, SECV, and r values were 147.7, 12.15, and 0.82, respectively, for the ten-component PLSR model using the full data set. The PLSR models prediction performances outperformed the PCR models for predicting protein contents and hardness of wheat.

Objective. Partial Least Squares (PLS) regression is a suitable linear decoder model for correlated and high dimensional neural data. This algorithm has been widely used in the application of brain–computer interface (BCI) for the decoding of motor parameters. PLS does not consider nonlinear relations between brain signal features. The nonlinear version of PLS that considers a nonlinear relationship between the latent variables has not been proposed for the decoding of intracranial data. This nonlinear model may cause overfitting in some cases due to a larger number of free parameters. In this paper, we develop a new version of nonlinear PLS, namely nonlinear sparse PLS (NLS PLS) and test it in BCI applications. Approach. In motor related BCI systems, improving the decoding accuracy of both kinetic and kinematic parameters of movement is crucial. To do this, two BCI datasets were chosen to decode the force amplitude and position of hand trajectory using the nonlinear and sparse versions of PLS algorithm. In our new NLS PLS method, we considered a polynomial relationship between the latent variables and used the lasso penalization in the latent space to avoid overfitting and to improve the decoding accuracy. Main results. Some linear and nonlinear based PLS models and our new proposed method, NLS PLS, were applied to the two datasets. According to our results, significant improvement from the NLS PLS method is confirmed over other methods. Our results show that nonlinear PLS outperforms generic PLS in the force decoding but it has lower accuracy in the hand trajectory decoding because of high dimensional feature space. By using lasso penalization, we presented a sparse nonlinear PLS-based model that outperforms generic PLS in both datasets and improves the coefficient of determination, 34% in the force decoding and 10% in the hand trajectory decoding. Significance. We constructed a simple PLS-based model that considers a nonlinear relationship between features and it is also robust to overfitting because of using the lasso penalty in the latent space. This model is suitable for a high dimensional and correlated datasets, like intracranial data and can improve the accuracy of estimation.

As a laboratory proximal sensing technique, the capability of visible and near-infrared (Vis-NIR) diffused reflectance spectroscopy with partial least squares (PLS) regression to determine soil properties has previously been demonstrated. However, the evaluation of the soil phosphorus (P) content—a major nutrient constraint for crop production in the tropics—is still a challenging task. PLS regression with waveband selection can improve the predictive ability of a calibration model, and a genetic algorithm (GA) has been widely applied as a suitable method for selecting wavebands in laboratory calibrations. To develop a laboratory-based proximal sensing method, this study investigated the potential to use GA-PLS regression analyses to estimate oxalate-extractable P in upland and lowland soils from laboratory Vis-NIR reflectance data. In terms of predictive ability, GA-PLS regression was compared with iterative stepwise elimination PLS (ISE-PLS) regression and standard full-spectrum PLS (FS-PLS) regression using soil samples collected in 2015 and 2016 from the surface of upland and lowland rice fields in Madagascar (n = 103). Overall, the GA-PLS model using first derivative reflectance (FDR) had the best predictive accuracy (R2 = 0.796) with a good prediction ability (residual predictive deviation (RPD) = 2.211). Selected wavebands in the GA-PLS model did not perfectly match wavelengths of previously known absorption features of soil nutrients, but in most cases, the selected wavebands were within 20 nm of previously known wavelength regions. Bootstrap procedures (N = 10,000 times) using selected wavebands also confirmed the improvements in accuracy and robustness of the GA-PLS model compared to those of the ISE-PLS and FS-PLS models. These results suggest that soil oxalate-extractable P can be predicted from Vis-NIR spectroscopy and that GA-PLS regression has the advantage of tuning optimum bands for PLS regression, contributing to a better predictive ability.

The particle size of polymer colloids is a key characteristic. It directly affects Raman measurements due to light scattering by particles. This publication exploits the extent to which these spectral changes can be correlated to the particle sizes by using a spectral hard model and data‐driven model.To this end, spectra of aqueous polystyrene nanoparticles are decomposed into pure component spectral models by means of indirect hard modeling (IHM). Each pure component model is characterized by multiple peak‐shaped Voigt profiles. By fitting the model to spectra of polymer colloids, spectral changes due to light scattering by particles are incorporated in the model parameters. The resulting parameter values are used as input for data‐driven partial least squares (PLS) regression to extract particle sizes. The method is applied to Raman spectra of polystyrene nanoparticles of 23–60 nm diameter measured repeatedly at concentrations 0.17–1 wt%. The hybrid model predicts the particle size with R2 = 0.99. In contrast, purely data‐driven PLS regression of the spectral intensities with 3 latent PLS variables results in R2 from 0.78 to 0.83. Resulting PLS scores portray a clear differentiability of Raman scattering and light scattering by particles within IHM model parameters. PLS regression coefficients further allow identification of individual peak parameters that correlate with particle size, which substantiates previous findings on correlation between Raman signal and scattering by polymer colloids. Combining IHM and data‐driven PLS regression demonstrates more accurate prediction of particle size for extended usage of Raman spectroscopy as comprehensive process analytical technology to determine sample concentrations and particle data.

<p>Soil properties could be assessed with reflectance spectroscopy (soil spectroscopy, SS) in vis-NIR region (400-2500 nm) through absorption features found in soil spectra. A high spectral resolution (up to 1 nm) drives to high dimensional and multicollinear data. This issue is usually addressed prior to modelling with feature extraction methods such as Principal Component Analysis, or embedded methods such as Partial Least Squares Regression (PLSR). Feature Selection (FS) wrapper methods are promising dimensionality reduction approaches barely used in SS. The objective of this study was two-fold: i) evaluate the performance of FS wrapper methods built from Random Forest (RF) algorithm to predict soil organic matter (SOM), clay and carbonates using laboratory spectroscopy, ii) test the performance of FS methods for dimensionality reduction in SS. The reflectance of 100 soil samples from Sierra de las Nieves National Park (Spain), was measured under laboratory conditions using an ASD FieldSpec Pro JR. A spectral preprocessing method, Continumm Removal (CR), was applied to raw spectra. The RF wrapper considered two different feature searching approaches: Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS). The performance of RF with FS (RF-FS) was compared to that of Partial Least Squares Regression (PLSR) and RF (without FS). Models were evaluated with R-squared, root mean squared error (RMSE) and ratio of prediction to deviation (RPD).</p><p>RF-FS models outperformed PLSR and RF models for the three SAP. RF-FS best models had a RPD of 2.19 for SOM, 1.64 for carbonates and 1.52 for clay, whereas PLSR models had RPD values of 1.59, 1.22 and 1.3, and RF 1.38, 1.23 and 1.23 for SOM, carbonates, and clay, respectively. Therefore, FS was useful in obtaining models with improved accuracy by reducing redundant features and avoiding multicollinearity (Hughes effect). The application of FS wrapper methods reduced the number of features in the RF-FS models to less than 1% of the starting features. Features were selected across all spectra from SOM and clay, and around 900, 1900 and 2350 nm for carbonates. This research, thus, shows an alternative to different feature extraction approaches for modelling soil properties based on FS methods and machine learning.</p>

Current economic development in tropical regions (especially in India, China, and Brazil) is putting tremendous pressure on tropical forest cover. Some of the dominant and economically important species are planted at large scale in these countries. Teak and bamboo are two important species of tropical regions because of their commercial and conservation values. Accurate estimates of foliar chemistry can help in evaluating the health status of vegetation in these regions. An attempt has been made to derive canopy level estimation of chlorophyll and leaf area index (LAI) for these species utilizing Hyperion data. Partial least square (PLS) regression analysis was carried out to identify the correlation between measured parameters (chlorophyll and LAI) and Hyperion reflectance spectra. PLS regression identified 600–750 nm as a sensitive spectral region for chlorophyll and 1000–1507 nm for LAI. The PLS regression model tested in this study worked well for the estimation of chlorophyll (R 2 = 0.90, root mean square error (RMSE) = 0.182 for teak and R 2 = 0.84, RMSE = 0.113 for bamboo) and for the estimation of LAI (R 2 = 0.87, RMSE = 0.425). The lower predictive error obtained indicates the robustness of the data set and also of the applicability of the PLS regression analysis. Wavelengths recognized by the PLS regression model were utilized for the development of vegetation indices for estimating chlorophyll and LAI. Predictive performances of the developed simple ratios (SRs) were evaluated using the cross-validation method. SR 743/692 gave the best results for the prediction of chlorophyll with the leave-one-out cross-validation (LOO-CV) method (R 2 = 0.73, RMSE = 0.28 for teak and R 2 = 0.71, RMSE = 0.15 for bamboo). The normalized difference ratio (ND 1457/1084) gave the best results for the prediction of LAI with LOO-CV (R 2 = 0.66, RMSE = 0.57). Ratios developed here can be tested for teak and bamboo cover spread in tropical regions with similar environmental conditions.

This study aimed to compare stepwise multiple linear regression (SMLR), partial least squares regression (PLSR) and support vector machine regression (SVMR) for estimating soil total nitrogen (TN) contents with laboratory visible/near-infrared reflectance (Vis/NIR) of selected coarse and heterogeneous soils. Moreover, the effects of the first (1st) vs. second (2nd) derivative of spectral reflectance and the importance wavelengths were explored. The TN contents and the Vis/NIR were measured in the laboratory. Several methods were employed for Vis/NIR data pre-processing. The SMLR, PLSR and SVMR models were calibrated and validated using independent datasets. Results showed that the SVMR and the PLSR models had similar performances, and better performances than the SMLR. The spectral bands near 1450, 1850, 2250, 2330 and 2430 nm in the PLSR model were important wavelengths. In addition, the 1st derivative was more appropriate than the 2nd derivative for spectral data pre-processing. PLSR was the most suitable method for estimating TN contents in this study. SVMR may be a promising technique, and its potential needs to be further explored. Moreover, the future studies using outdoor and airborne/satellite hyperspectral data for estimating TN content are necessary for testing the findings.

Carbohydrate levels are important regulators of the growth and yield of tree crops. Current methods for measuring foliar carbohydrate concentrations are time consuming and laborious, but rapid imaging technologies have emerged with the potential to improve the effectiveness of tree nutrient management. Carbohydrate concentrations were predicted using hyperspectral imaging (400–1000 nm) of leaves of the evergreen tree crops, avocado, and macadamia. Models were developed using partial least squares regression (PLSR) and artificial neural network (ANN) algorithms to predict carbohydrate concentrations. PLSR models had R2 values of 0.51, 0.82, 0.86, and 0.85, and ANN models had R2 values of 0.83, 0.83, 0.78, and 0.86, in predicting starch, sucrose, glucose, and fructose concentrations, respectively, in avocado leaves. PLSR models had R2 values of 0.60, 0.64, 0.91, and 0.95, and ANN models had R2 values of 0.67, 0.82, 0.98, and 0.98, in predicting the same concentrations, respectively, in macadamia leaves. ANN only outperformed PLSR when predicting starch concentrations in avocado leaves and sucrose concentrations in macadamia leaves. Performance differences were possibly associated with nonlinear relationships between carbohydrate concentrations and reflectance values. This study demonstrates that PLSR and ANN models perform well in predicting carbohydrate concentrations in evergreen tree-crop leaves.