Efficient cross-validatory algorithm for identifying dynamic nonlinear process models

A couple of regularized least squares regression models in a feature space are extended by the kernel partial least squares (KPLS) regression model in this paper. PLS is a method based on the projection of input (explanatory) variables to the latent variables (components), and has been developed and established as one of the multivariate statistical process control (MSPC) methods. With PLS, the regression matrix is determined on the basis of a subset of the predictor variables and thus, PLS is able to reduce the number of variables to be considered. In this paper, two kinds of KPLS algorithm for construction of nonlinear regression models in possibly high-dimensional feature spaces are provided. According to the idea of structural risk minimization (SRM), the work described here provides an index of generalization errors. We give the theoretical description of the KPLS algorithms and experimentally compare the algorithms with some existing PLS and KPLS regression models. We will demonstrate that on the data sets employed KPLS achieves better results than PLS and some other KPLS. At the same time, validity of index of generalization errors is proved affirmatively.

The terahertz (THz) spectra of rapeseed leaves with different water content (WC) were investigated. The transmission and absorption spectra in the range of 0.3–2 THz were measured by using THz time-domain spectroscopy. The mean transmittance and absorption coefficients were applied to analyze the change regulation of WC. In addition, the Savitzky-Golay method was performed to preprocess the spectra. Then, the partial least squares (PLS), kernel PLS (KPLS), and Boosting-PLS were conducted to establish models for predicting WC based on the processed transmission and absorption spectra. Reliable results were obtained by these three methods. KPLS generated the best prediction accuracy of WC. The prediction coefficient correlation (Rval) and root mean square error (RMSEP) of KPLS based on transmission were Rval = 0.8508, RMSEP = 0.1015, and that based on absorption were Rval = 0.8574, RMSEP = 0.1009. Results demonstrated that THz spectroscopy combined with modeling methods provided an efficient and feasible technique for detecting plant physiological information.

Kernel partial least squares (KPLS) has become popular techniques for chemical and biological modeling, which is a nonlinear extension of linear PLS. Training samples are transformed into a feature space via a nonlinear mapping, and then PLS algorithm can be carried out in the feature space. However, one of the main limitations of KPLS is that each feature is given the same importance in the kernel matrix, thus explaining the poor performance of KPLS for data with many irrelevant features. In this study, we provide a new strategy incorporated variable importance into KPLS, which is termed as the WKPLS approach. The WKPLS approach by modifying the kernel matrix provides a feasible way to differentiate between the true and noise variables. On the basis of the fact that the regression coefficients of the PLS model reflect the importance of variables, we firstly obtain the normalized regression coefficients by establishing the PLS model with all the variables. Then, Variable importance is incorporated into primary kernel. The performance of WKPLS is investigated with one simulated dataset and two structure–activity relationship (SAR) datasets. Compared with standard linear kernel PLS and Gaussian kernel PLS, The results show that WKPLS yields superior prediction performances to standard KPLS. WKPLS could be considered as a good mechanism by introducing extra information to improve the performance of KPLS for modeling SAR.

Determination of free amino acid content in Radix Pseudostellariae using near infrared (NIR) spectroscopy and different multivariate calibrations

Point pattern matching is an essential step in many image processing applications. This letter investigates the spectral approaches of point pattern matching, and presents a spectral feature matching algorithm based on kernel partial least squares (KPLS). Given the feature points of two images, we define position similarity matrices for the reference and sensed images, and extract the pattern vectors from the matrices using KPLS, which indicate the geometric distribution and the inner relationships of the feature points. Feature points matching are done using the bipartite graph matching method. Experiments conducted on both synthetic and real-world data demonstrate the robustness and invariance of the algorithm.

Current economic development in tropical regions (especially in India, China, and Brazil) is putting tremendous pressure on tropical forest cover. Some of the dominant and economically important species are planted at large scale in these countries. Teak and bamboo are two important species of tropical regions because of their commercial and conservation values. Accurate estimates of foliar chemistry can help in evaluating the health status of vegetation in these regions. An attempt has been made to derive canopy level estimation of chlorophyll and leaf area index (LAI) for these species utilizing Hyperion data. Partial least square (PLS) regression analysis was carried out to identify the correlation between measured parameters (chlorophyll and LAI) and Hyperion reflectance spectra. PLS regression identified 600–750 nm as a sensitive spectral region for chlorophyll and 1000–1507 nm for LAI. The PLS regression model tested in this study worked well for the estimation of chlorophyll (R 2 = 0.90, root mean square error (RMSE) = 0.182 for teak and R 2 = 0.84, RMSE = 0.113 for bamboo) and for the estimation of LAI (R 2 = 0.87, RMSE = 0.425). The lower predictive error obtained indicates the robustness of the data set and also of the applicability of the PLS regression analysis. Wavelengths recognized by the PLS regression model were utilized for the development of vegetation indices for estimating chlorophyll and LAI. Predictive performances of the developed simple ratios (SRs) were evaluated using the cross-validation method. SR 743/692 gave the best results for the prediction of chlorophyll with the leave-one-out cross-validation (LOO-CV) method (R 2 = 0.73, RMSE = 0.28 for teak and R 2 = 0.71, RMSE = 0.15 for bamboo). The normalized difference ratio (ND 1457/1084) gave the best results for the prediction of LAI with LOO-CV (R 2 = 0.66, RMSE = 0.57). Ratios developed here can be tested for teak and bamboo cover spread in tropical regions with similar environmental conditions.

Inspired by the concept of knowledge-based scoring functions, a new quantitative structure-activity relationship (QSAR) approach is introduced for scoring protein-ligand interactions. This approach considers that the strength of ligand binding is correlated with the nature of specific ligand/binding site atom pairs in a distance-dependent manner. In this technique, atom pair occurrence and distance-dependent atom pair features are used to generate an interaction score. Scoring and pattern recognition results obtained using Kernel PLS (partial least squares) modeling and a genetic algorithm-based feature selection method are discussed.

We suggest in this article a dynamic reduced algorithm in order to enhance the monitoring abilities of nonlinear processes. Dynamic fault detection using data-driven methods is among the key technologies, which shows its ability to improve the performance of dynamic systems. Among the data-driven techniques, we find the kernel partial least squares (KPLS) which is presented as an interesting method for fault detection and monitoring in industrial systems. The dynamic reduced KPLS method is proposed for the fault detection procedure in order to use the advantages of the reduced KPLS models in online mode. Furthermore, the suggested method is developed to monitor the time-varying dynamic system and also update the model of reduced reference. The reduced model is used to minimize the computational cost and time and also to choose a reduced set of kernel functions. Indeed, the dynamic reduced KPLS allows adaptation of the reduced model, observation by observation, without the risk of losing or deleting important information. For each observation, the update of the model is available if and only if a further normal observation that contains new pertinent information is present. The general principle is to take only the normal and the important new observation in the feature space. Then the reduced set is built for the fault detection in the online phase based on a quadratic prediction error chart. Thereafter, the Tennessee Eastman process and air quality are used to precise the performances of the suggested methods. The simulation results of the dynamic reduced KPLS method are compared with the standard one.

Algorithms for partial least squares (PLS) modelling are placed into a sound theoretical context focusing on numerical precision and computational efficiency. NIPALS and other PLS algorithms that perform deflation steps of the predictors (X) may be slow or even computationally infeasible for sparse and/or large‐scale data sets. As alternatives, we develop new versions of the Bidiag1 and Bidiag2 algorithms. These include full reorthogonalization of both score and loading vectors, which we consider to be both necessary and sufficient for numerical precision. Using a collection of benchmark data sets, these 2 new algorithms are compared to the NIPALS PLS and 4 other PLS algorithms acknowledged in the chemometrics literature. The provably stable Householder algorithm for PLS regression is taken as the reference method for numerical precision. Our conclusion is that our new Bidiag1 and Bidiag2 algorithms are the methods of choice for problems where both efficiency and numerical precision are important. When efficiency is not urgent, the NIPALS PLS and the Householder PLS are also good choices. The benchmark study shows that SIMPLS gives poor numerical precision even for a small number of factors. Further, the nonorthogonal scores PLS, direct scores PLS, and the improved kernel PLS are demonstrated to be numerically less stable than the best algorithms. Prototype MATLAB codes are included for the 5 PLS algorithms concluded to be numerically stable on our benchmark data sets. Other aspects of PLS modelling, such as the evaluation of the regression coefficients, are also analyzed using techniques from numerical linear algebra.

we propose a regression method using partial least square (PLS) technique and kernel method, which we refer to as kernel partial least square regression II to distinguish the conventional kernel PLS regression. The motivation of the conventional kernel PLS regression is to establish a coordinator system where the independent and dependent variables have a stronger correlation, and which models a regression model using the coordinator system. However, it projects independent variables into a reproducing kernel Hilbert space (RKHS) alone. The proposal extends the basic framework of conventional kernel PLS regression. The proposed method does not only project independent variables into the RKHS, but also dependent variables, and establishes a coordinator system where the independent and dependent variables have a stronger correlation. We use two function regression cases to evaluate the proposed method compared with the conventional kernel PLS regression. The regression performance of the proposed method has almost the same regression accuracy arising from the evaluation result, and this depends on the regression tasks. We explain the correlation calculations of our proposed method, conventional kernel PLS regression, and PLS regression. The meaning of correlation depends on the application in question. We also analyse and discuss the algorithm implementation, correlation meaning, and other issues for further development of the proposal.

This paper investigated the feasibility of Fourier transform infrared transmission (FT-IR) spectroscopy to detect talcum powder illegally added in tea based on chemometric methods. Firstly, 210 samples of tea powder with 13 dose levels of talcum powder were prepared for FT-IR spectra acquirement. In order to highlight the slight variations in FT-IR spectra, smoothing, normalize and standard normal variate (SNV) were employed to preprocess the raw spectra. Among them, SNV preprocessing had the best performance with high correlation of prediction (RP = 0.948) and low root mean square error of prediction (RMSEP = 0.108) of partial least squares (PLS) model. Then 18 characteristic wavenumbers were selected based on a hybrid of backward interval partial least squares (biPLS) regression, competitive adaptive reweighted sampling (CARS) algorithm and successive projections algorithm (SPA). These characteristic wavenumbers only accounted for 0.64% of the full wavenumbers. Following that, 18 characteristic wavenumbers were used to build linear and nonlinear determination models by PLS regression and extreme learning machine (ELM), respectively. The optimal model with RP = 0.963 and RMSEP = 0.137 was achieved by ELM algorithm. These results demonstrated that FT-IR spectroscopy with chemometrics could be used successfully to detect talcum powder in tea.

Highly correlated predictors and nonlinear relationships between response and predictors potentially affected the performance of predictive modeling, especially when using the ordinary least square (OLS) method. The simple technique to solve this problem is by using another method such as Partial Least Square Regression (PLSR), Support Vector Regression with kernel Radial Basis Function (SVR-RBF), and Random Forest Regression (RFR). The purpose of this study is to compare OLS, PLSR, SVR-RBF, and RFR using simulation data. The methods were evaluated by the root mean square error prediction (RMSEP). The result showed that in the linear model, SVR-RBF and RFR have large RMSEP; OLS and PLSR are better than SVR-RBF and RFR, and PLSR provides much more stable prediction than OLS in case of highly correlated predictors and small sample size. In nonlinear data, RFR produced the smallest RMSEP when data contains high correlated predictors.

Plant nitrogen concentration (PNC) is a traditional standard index to measure the nitrogen nutritional status of winter wheat. Rapid and accurate diagnosis of PNC performs an important role in mastering the growth status of winter wheat and guiding field precision fertilization. In this study, the in situ hyperspectral reflectance data were measured by handheld SVC HR−1024I (SVC) passive field spectroradiometer and PNC were determined by the modified Kjeldahl digestion method. Continuous wavelet transform (CWT), successive projection algorithm (SPA) and partial least square (PLS) regression were combined to construct an efficient method for estimating winter wheat PNC. The main objectives of this study were to (1) use CWT to extract various wavelet coefficients under different decomposition scales, (2) use SPA to screen sensitive wavelet coefficients as independent variables and combine with PLS regression to establish winter wheat PNC estimation models, respectively, and (3) compare the precision of PLS regression models to find a reliable model for estimating winter wheat PNC during the growing season. The results of this paper showed that properly increasing the decomposition scale of CWT could weaken the impact of high-frequency noise on the prediction model. The number of wavelet coefficients has been significantly reduced after screened by SPA. The PNC estimation model (CWT–Scale6–SPA–PLS) based on the wavelet coefficients of the sixth decomposition scale most accurately predicted the PNC (the determination coefficient of the calibration set (Rc2) was 0.85. Root mean square error of the calibration set (RMSEc) was 0.27. The determination coefficient of the validation set (Rv2) was 0.84. Root mean square error of the validation set (RMSEv) was 0.28 and relative prediction deviation (RPD) was 2.47). CWT-Scale6-SPA-PLS can be used to predict PNC. The optimal winter wheat PNC prediction model based on CWT proposed in this study is a reliable method for rapid and nondestructive monitoring of PNC and provides a new technical method for precision nitrogen management.

Objective. Partial Least Squares (PLS) regression is a suitable linear decoder model for correlated and high dimensional neural data. This algorithm has been widely used in the application of brain–computer interface (BCI) for the decoding of motor parameters. PLS does not consider nonlinear relations between brain signal features. The nonlinear version of PLS that considers a nonlinear relationship between the latent variables has not been proposed for the decoding of intracranial data. This nonlinear model may cause overfitting in some cases due to a larger number of free parameters. In this paper, we develop a new version of nonlinear PLS, namely nonlinear sparse PLS (NLS PLS) and test it in BCI applications. Approach. In motor related BCI systems, improving the decoding accuracy of both kinetic and kinematic parameters of movement is crucial. To do this, two BCI datasets were chosen to decode the force amplitude and position of hand trajectory using the nonlinear and sparse versions of PLS algorithm. In our new NLS PLS method, we considered a polynomial relationship between the latent variables and used the lasso penalization in the latent space to avoid overfitting and to improve the decoding accuracy. Main results. Some linear and nonlinear based PLS models and our new proposed method, NLS PLS, were applied to the two datasets. According to our results, significant improvement from the NLS PLS method is confirmed over other methods. Our results show that nonlinear PLS outperforms generic PLS in the force decoding but it has lower accuracy in the hand trajectory decoding because of high dimensional feature space. By using lasso penalization, we presented a sparse nonlinear PLS-based model that outperforms generic PLS in both datasets and improves the coefficient of determination, 34% in the force decoding and 10% in the hand trajectory decoding. Significance. We constructed a simple PLS-based model that considers a nonlinear relationship between features and it is also robust to overfitting because of using the lasso penalty in the latent space. This model is suitable for a high dimensional and correlated datasets, like intracranial data and can improve the accuracy of estimation.

In order to test the applicability of support vector regression (SVR) in near-infrared (NIR) spectral analysis of tobacco,NIR calibration model for starch in tobacco were developed by processing NIR spectra and chemically determined starch content data of 187 tobacco samples with SVR,partial least square regression (PLS),multiplicative linear regression (MLR) and error back propagation artificial neural network (BP-ANN). The obtained model was tested through internal validation and external validation with leave-one-out cross validation (LOOCV) and independent sample set. The results showed that the accuracy of SVR model was slightly higher than that of BP-ANN,PLS and MLR models,it implied that SVR algorithm could be applied to NIR analysis of starch in tobacco.