Dimensionality Reduction, Regularization, and Generalization in Overparameterized Regressions

Regional regression models can be employed to estimate hydrologic statistics at ungauged river sites. The increased availability of watershed characteristics and the use of GIS-based methods to process this data have created situations where a large number of highly correlated watershed characteristics are available as potential model explanatory variables. The use of ordinary least squares (OLS) regression procedures with highly correlated variables can produce multicollinearity, creating highly sensitive parameter estimators with inflated variances, and improper model selection. A Monte Carlo simulation is developed to compare four techniques for handling multicollinearity: OLS, OLS with Variance Inflation Factor screening (VIF), principal component regression (PCR), and partial least squares regression (PLS). Results show the impact of multicollinearity is magnified at smaller samples sizes, higher correlations, and larger model error variances. Although PCR and PLS yield parameter estimators with reduced variances, there is no improvement in model prediction compared to OLS and VIF. Using VIF for screening variables produced small improvements in model predictions. The use of OLS appears warranted as the complexity of using biased regression techniques to address multicollinearity does little to improve model predictions.

We propose a selection procedure of principal components in principal component regression. Our method selects principal components using variable selection procedures instead of a small subset of major principal components in principal component regression. Our procedure consists of two steps to improve estimation and prediction. First, we reduce the number of principal components using the conventional principal component regression to yield the set of candidate principal components and then select principal components among the candidate set using sparse regression techniques. The performance of our proposals is demonstrated numerically and compared with the typical dimension reduction approaches (including principal component regression and partial least square regression) using synthetic and real datasets.

This study aims to estimate simultaneously metformin hydrochloride (MET) and glyburide (GLY) in a multicomponent tablets dosage form by spectrophotometric method using chemometric approaches such as principal component regression (PCR) and partial least-squares regression (PLS). Because of highly overlapped in UV spectra and difference proportions of two active ingredients, the conventional univariate calibration methods was not allowed without previous separation. The linearity ranges used to construct the calibration matrix were selected in the ranges from 40.00 to 200.00 mg L-1for MET and from 1.00 to 10.00 mg L-1for GLY. The absorbances were measured in the wavelength range of 200-400 nm, using ethanol as solvent. The resulting UV spectra were subjected to PCR and PLS algorithms and the optimum numbers of principal components (PCs) were selected according to prediction residual error sum of squares (PRESS) values of leave-one out cross-validation. The number of PCs for MET and GLY were found to be 5, 3 by PCR and 5, 3 by PLS, respectively. A set of synthetic mixtures was employed to verify the models and the performance of the models were shown in the values of the root mean square error in prediction (RMSEP). RMSEP values of MET and GLY were 1.806, 0.256 for PCR and 1.802, 0.185 for PLS, respectively. The suitable calibration models were applied to the analysis of these compounds in pharmaceutical formulation.

The multivariate methods, principal component regression (PCR) and partial least squares (PLS) were tested as a calibration procedure for simultaneous ultraviolet spectrophotometric determination of acetaminophen (AC) and ascorbic acid (AA). Determination of these compounds is important because of their pharmacotherapeutic advantages. Due to spectral overlapping of AC and AA, PCR and PLS were used for construction of the calibration sets. The concentration linear range of AC and AA were 1.5-24.2 and 1.8-21.1 microg mL(-1) respectively. The absorption spectra were recorded from 215-310 nm. The minimum root mean square error of prediction (RMSEP) was 1.3507 and 0.4088 for AC and AA, by PLS, 0.7525 and 0.4015 by PCR in original data and 0.9454 and 0.2875, by PLS and 1.0386 and 0.4000 by PCR in derivative data. The procedure allows the simultaneous determination of AC and AA in synthetic mixtures and real sample solutions made up from pharmaceutical products, human serum and urine.

The objective of this study was to compare the predictions of the protein contents and hardness values by partial least squares regression (PLSR) and principal components regression (PCR) models for bulk samples of Canadian wheat, which were obtained from different locations and crop years. Wheat samples of Canada Western Red Spring (CWRS), Canada Western Hard White Spring (CWHWS), Canada Western Soft White Spring (CWSWS), and Canada Prairie Spring Red (CPSR) classes were obtained from nearby agricultural farms in the main wheat growing locations in the Provinces of Alberta, Saskatchewan, and Manitoba from 2007, 2008, and 2009 crop years. Wheat samples were conditioned to moisture levels of 13, 16, and 19 % (wet basis) and pooled together for developing the regression models. A database of the near-infrared (NIR) hyperspectral image cubes of bulk samples of wheat classes was created in the wavelength region of 960–1,700 nm with 10 nm intervals. Reference protein contents and hardness values were determined using the Dumatherm method and single kernel characterization system (SKCS), respectively. A tenfold cross-validation was used for the ten-factor partial least squares regression (PLSR) and principal components regression (PCR) models for prediction purposes. Prediction performances of regression models were assessed by calculating the estimated mean square errors of prediction (MSEP), standard error of cross-validation (SECV), and correlation coefficient (r). Using the full data set in the protein prediction study, the ten-component PLSR model gave 1.76, 1.33, and 0.68 for the estimated MSEP, SECV, and r, respectively, which were better than the results for the ten-component PCR model (2.02, 1.42, and 0.62, respectively). For the hardness prediction, the estimated MSEP, SECV, and r values were 147.7, 12.15, and 0.82, respectively, for the ten-component PLSR model using the full data set. The PLSR models prediction performances outperformed the PCR models for predicting protein contents and hardness of wheat.

We present here that the surface-enhanced Raman spectroscopy (SERS) technique in conjunction with the partial least squares analysis is as a potential tool for the differentiation of pleural effusion in the course of the cancerous disease and a tool for faster diagnosis of lung cancer. Pleural effusion occurs mainly in cancer patients due to the spread of the tumor, usually caused by lung cancer. Furthermore, it can also be initiated by non-neoplastic diseases, such as chronic inflammatory infection (the most common reason for histopathological examination of the exudate). The correlation between pleural effusion induced by tumor and non-cancerous diseases were found using surface-enhanced Raman spectroscopy combined with principal component regression (PCR) and partial least squares (PLS) multivariate analysis method. The PCR predicts 96% variance for the division of neoplastic and non-neoplastic samples in 13 principal components while PLS 95% in only 10 factors. Similarly, when analyzing the SERS data to differentiate the type of tumor (squamous cell vs. adenocarcinoma), PLS gives more satisfactory results. This is evidenced by the calculated values of the root mean square errors of calibration and prediction but also the coefficients of calibration determination and prediction (R2C = 0.9570 and R2C = 0.7968), which are more robust and rugged compared to those calculated for PCR. In addition, the relationship between cancerous and non-cancerous samples in the dependence on the gender of the studied patients is presented.

In this paper, the influence of economic variables on the annual electricity consumption in Beijing in 2000-2013 has been investigated. The purpose is to find out their relationship with single data sets and multivariate statistic methods. The multivariate methods used in this paper include ordinary least squares regression (OLSR), principal component regression (PCR), partial least squares regression (PLSR), and modified partial least squares regression (MPLSR). In comparison of these methods, the PLSR model is shown to perform better than the others. The effectiveness of the electricity consumption prediction models is illustrated and verified based on the practical data sets of electricity consumption in Beijing.

Prediction models based on multivariate statistical methods and their applications for predicting railway freight volume

Unification of neural and statistical methods as applied to materials structure-property mapping

The utilization of selection methods such as genetic algorithm (GA) aims to construct better partial least squares (PLS) and principal component regression (PCR) models than those established from the full-spectrum range. Determination of paracetamol (PAR), orphenadrine citrate (Or.cit), and caffeine (CAF) in the presence of PAR nephrotoxic impurity [p-aminophenol (PAP)]. GA was applied to select the optimum wavelengths used. A calibration set was prepared in which the three drugs, together with PAP, were modeled by multilevel multifactor design. This calibration set was used to build the PLS and PCR models, either with or without preprocessing the data using GA. Results were compared with and without preprocessing, and this revealed that GA can find an optimized combination of spectral wavelengths, yielding a lower root mean square error of prediction as well as a lower number of latent variables used. The results of the two models show that simultaneous determination of the aforementioned drugs can be performed in the concentration ranges of 20-60, 3-11, and 1-9 μg/mL for PAR, Or.cit, and CAF, respectively. The proposed models were applied for the determination of the three drugs in their pharmaceutical formulations, and the results were verified by the standard addition technique. GA can be useful as a wavelength selection tool before applying multivariate PLS and PCR methods. GA gives an improvement in the predictive ability of the models with lower RMSEP and less number of latent variables (LVs). The proposed PLS, PCR, GA-PLS, and GA-PCR spectrophotometric methods were able to determine paracetamol, orphenadrine citrate, and caffeine in the presence of p-aminophenol and severe spectral overlapping.

Recycled polypropylene (rPP) often contains a small amount of polyethylene (PE). Since polypropylene (PP) and PE are incompatible, the presence of PE compromises the performance of rPP materials and needs to be closely monitored. In our previous work, Raman and near-infrared (NIR) spectrometries were evaluated to monitor PE content in rPP with partial least square regression (PLSR) modeling. The NIR spectrometry exhibited a wider application range, but the accuracy of the prediction models might be further improved. In the current work, a different modeling method, principal component regression (PCR) was employed to analyze PE content in rPP with NIR spectrometry. Spectrum pretreatment methods, including multivariate scatter correction (MSC), standard normal variate transformation (SNV), smoothing, and first derivative, were investigated to improve the NIR spectrum quality. Forward and backward interval methods were used to optimize spectral range selection. The outcomes were compared with our previous PLSR modeling results. The highest accuracy in independent validation was achieved by a PCR model with an R 2 of 0.9991 and a root-mean-square error of prediction (RMSEP) of 0.1596 PE%. On the other hand, a PLSR model achieved the lowest RMSEP of 0.9712 PE% for a non-colored post-consumer rPP sample. The PCR models might be sensitive to interference and more suitable for post-industrial materials, which have a simpler chemical composition. The PLSR models might have better stability and be more suitable for complicated post-consumer samples. Both the PCR and PLSR models were successfully applied to a gray commercial rPP sample.

Development of novel univariate and multivariate validated chemometric methods for the analysis of dasatinib, sorafenib, and vandetanib in pure form, dosage forms and biological fluids

Authentication of Patin fish oil (MIP) is essential to prevent adulteration practice, to ensure quality, nutritional value, and product safety. The purpose of this study is to apply the FTIR spectroscopy combined with chemometrics for MIP authentication. The chemometrics method consists of principal component regression (PCR) and partial least square regression (PLSR). PCR and PLSR were used for multivariate calibration, while for grouping the samples using discriminant analysis (DA) method. In this study, corn oil (MJ) was used as an adulterate. Twenty-one mixed samples of MIP and MJ were prepared with the adulterate concentration range of 0-50%. The best authentication model was obtained using the PLSR technique using the first derivative of FTIR spectra at a wavelength of 650-3432 cm-1. The coefficient of determination (R2) for calibration and validation was obtained 0.9995 and 1.0000, respectively. The value of root mean square error of calibration (RMSEC) and root mean square error of prediction (RMSEP) were 0.397 and 0.189. This study found that the DA method can group the samples with an accuracy of 99.92%.

One of the assumptions in ordinary least squares (OLS) in estimating regression parameter is lack of multicollinearity. If the multicollinearity exists, Partial Least Square (PLS) and Principal Component Regression (PCR) can be used as alternative approaches to solve the problem. This research intends to compare those methods in modeling factors that influence the Human Development Index (HDI) of North Sumatra Province in 2019 obtained from the Central Bureau of Statistics. The result indicates that the PLS outperforms the PCR in term of the coefficient of determination and squared error

Robust principal component regression s-estimator is principal component regression that applies robust approach method at principal component analysis and s-estimator at principal component regression analysis. The aim of robust principal component regression s-estimator is to overcome multicollinearity problems in multiple linier regression Ordinary Least Square (OLS) and to overcome outlier problems in principal component regression so get the most effective model. Minimum Volume Ellipsoid (MVE) is one of the robust approach methods that applied when doing principal component analysis and S-Estimator is one of the estimation methods that applied when doing principal component regression analysis. The case in this study is the factors that influence the Number of Unemployment in Central Java in 2017. The model that provides the most effective result to handling multicolliniearity and ouliers in the case study Number of Unemployment in Central Java in 2017 is using robust principal component regression MVE-(S-Estimator) with Adjusted R2 value of 0.9615 and RSE value of 0.4073. Keywords: Robust Principal Component Regression S-Estimator, Multicollinearity, Outliers, Minimum Volume Ellipsoid (MVE), Number of Unemployment.