A Novel Nonlinear Combination Model Based on Support Vector Machine for Rainfall Prediction

Objective. Partial Least Squares (PLS) regression is a suitable linear decoder model for correlated and high dimensional neural data. This algorithm has been widely used in the application of brain–computer interface (BCI) for the decoding of motor parameters. PLS does not consider nonlinear relations between brain signal features. The nonlinear version of PLS that considers a nonlinear relationship between the latent variables has not been proposed for the decoding of intracranial data. This nonlinear model may cause overfitting in some cases due to a larger number of free parameters. In this paper, we develop a new version of nonlinear PLS, namely nonlinear sparse PLS (NLS PLS) and test it in BCI applications. Approach. In motor related BCI systems, improving the decoding accuracy of both kinetic and kinematic parameters of movement is crucial. To do this, two BCI datasets were chosen to decode the force amplitude and position of hand trajectory using the nonlinear and sparse versions of PLS algorithm. In our new NLS PLS method, we considered a polynomial relationship between the latent variables and used the lasso penalization in the latent space to avoid overfitting and to improve the decoding accuracy. Main results. Some linear and nonlinear based PLS models and our new proposed method, NLS PLS, were applied to the two datasets. According to our results, significant improvement from the NLS PLS method is confirmed over other methods. Our results show that nonlinear PLS outperforms generic PLS in the force decoding but it has lower accuracy in the hand trajectory decoding because of high dimensional feature space. By using lasso penalization, we presented a sparse nonlinear PLS-based model that outperforms generic PLS in both datasets and improves the coefficient of determination, 34% in the force decoding and 10% in the hand trajectory decoding. Significance. We constructed a simple PLS-based model that considers a nonlinear relationship between features and it is also robust to overfitting because of using the lasso penalty in the latent space. This model is suitable for a high dimensional and correlated datasets, like intracranial data and can improve the accuracy of estimation.

This paper tries to explore the optimum variable selection strategy for Partial Least Squares (PLS) regression using a model dataset of cytoprotection data. The compounds of the dataset were classified using K‐means clustering technique applied on standardized descriptor matrix and ten combinations of training and test sets were generated based on the obtained clusters. For a particular training set, PLS models were developed with a number of components optimized by leave‐one‐out Q2 and then the developed models were validated (externally) using the test set compounds. For each set, PLS model was initially constructed using all descriptors (variables). The variables having least standardized values of regression coefficients were deleted and the next model was developed with a reduced set of variables. These steps were performed several times until further reduction in number of variables did not improve Q2 value. In each case, statistical parameters like predictive R2 (R2pred), squared correlation coefficient between observed and predicted values with (r2) and without ($\rm{ r_0^{\rm{2}} }$) intercept and Root Mean Square Error of Prediction (RMSEP) were calculated from the test set compounds. In case of all ten sets, Q2 values steadily increase on deletion of variables while R2pred values do not show any specific trend. In no case, the highest Q2 and highest R2pred appear in the same trial, i.e., with the same combinations of variables. This suggests that from the viewpoint of external predictability, choice of variables for PLS based on Q2 value may not be optimum. Moreover, a clear separation of r2 and r02 curves in some sets suggests that such models may not be truly predictive in spite of acceptable R2pred values. Another observation is that coefficient of determination R2 for the training set is more immune to changes on deletion of variables than the validation parameters like Q2 and R2pred. Finally, a new parameter rm2 has been suggested to indicate external predictability of QSAR models.

There are various models to predict financial time series like the RMB exchange rate. In this paper, considering the complex characteristics of RMB exchange rate, we build a nonlinear combination model of the autoregressive fractionally integrated moving average (ARFIMA) model, the support vector machine (SVM) model, and the back-propagation neural network (BPNN) model to forecast the RMB exchange rate. The basic idea of the nonlinear combination model (NCM) is to make the prediction more effective by combining different models’ advantages, and the weight of the combination model is determined by a nonlinear weighted mechanism. The RMB exchange rate against US dollar (RMB/USD) and the RMB exchange rate against Euro (RMB/EUR) are used as the empirical examples to evaluate the performance of NCM. The results show that the prediction performance of the nonlinear combination model is better than the single models and the linear combination models, and the nonlinear combination model is suitable for the prediction of the special time series, such as the RMB exchange rate.

We introduce an action recognition approach based on Partial Least Squares (PLS) and Support Vector Machines (SVM). We extract very high dimensional feature vectors representing spatio-temporal properties of actions and use multiple PLS regressors to find relevant features that distinguish amongst action classes. Finally, we use a multi-class SVM to learn and classify those relevant features. We applied our approach to INRIA's IXMAS dataset. Experimental results show that our method is superior to other methods applied to the IXMAS dataset.

The application of near infrared spectroscopy for quantitative analysis of cotton-polyester textile was investigated in the present work. A total of 214 cotton-polyester fabric samples, covering the range from 0% to 100% cotton were measured and analyzed. Partial least squares and least-squares support vector machine models with all variables as input data were established. Furthermore, successive projection algorithm was used to select effective wavelengths and establish the successive projection algorithm-least-squares support vector machine models, with the comparison of two other effective wavelength selection methods: loading weights analysis and regression coefficient analysis. The calibration and validation results show that the successive projection algorithm-least-squares support vector machine model outperformed not only the partial least squares and least-squares support vector machine models with all variables as inputs, but also the least-squares support vector machine models with loading weights analysis and regression coefficient analysis effective wavelength selection. The root mean squared error of calibration and root mean squared error of prediction values of the successive projection algorithm-least-squares support vector machine regression model with the optimal performance were 0.77% and 1.17%, respectively. The overall results demonstrated that near infrared spectroscopy combined with least-squares support vector machine and successive projection algorithm could provide a simple, rapid, economical and non-destructive method for determining the composition of cotton-polyester textiles.

Virtual metrology of semiconductor PVD process based on combination of tree-based ensemble model

A novel nonlinear ensemble forecasting model incorporating GLAR and ANN for foreign exchange rates

Health monitoring and prognostics of axial piston pump is very helpful for the safety of aerial hydraulic system. Directing to the difficulties of measuring the wear loss between the valve plate and cylinder barrel, this paper presents a hybrid prediction method based on EMD (Empirical Mode Decomposition) and SVM (Support Vector Machine), in which EMD is used to get the pump's health state and PSO (Particle Swarm Optimization)-SVM is used to make a prediction for the pump's remaining useful lifetime. Oil-return flow was selected to indicate the wear condition of pump with several IMFs. SVM was trained by these IMFs, and then be used to predict the oil-return flow one-step ahead or multi-step ahead. In the SVM's training process, PSO was used to search the optimal parameter of SVM's kernel function. Applications show that hybrid prediction method has higher prediction precision and could be applied for the remaining useful lifetime prognostics.

In semiconductor wafer fabrication, etching is one of the most critical processes, by which a material layer is selectively removed. Because of difficulty to correct a mistake caused by over etching, it is critical that etch should be performed correctly. This paper proposes a new approach for etch endpoint detection of small open area wafers. The traditional endpoint detection technique uses a few manually selected wavelengths, which are adequate for large open areas. As the integrated circuit devices continue to shrink in geometry and increase in device density, detecting the endpoint for small open areas presents a serious challenge to process engineers. In this work, a high-resolution optical emission spectroscopy (OES) sensor is used to provide the necessary sensitivity for detecting subtle endpoint signal. Partial Least Squares (PLS) method is used to analyze the OES data which reduces dimension of the data and increases gap between classes. Support Vector Machine (SVM) is employed to detect endpoint using the data after PLS. SVM classifies normal etching state and after endpoint state. Two data sets from OES are used in training PLS and SVM. The other data sets are used to test the performance of the model. The results show that the trained PLS and SVM hybrid algorithm model detects endpoint accurately.

Breast cancer is a highly prevalent malignant tumor among women worldwide, and early accurate diagnosis is crucial for improving prognosis. Traditional diagnostic methods have limitations such as high rates of misdiagnosis and missed diagnosis, while existing machine learning studies lack targeted strategies for screening nuclear morphological features. This study is the first to adopt Pearson correlation coefficient for targeted screening of breast cancer nuclear morphological features, retaining 25 features with an absolute correlation coefficient greater than 0.2 with diagnostic results; these features achieve a cumulative explained variance of 92.5% and accurately match the core pathological indicator of "nuclear pleomorphism". To address the class imbalance in the original dataset (569 cases, including 357 benign and 212 malignant cases), 143 synthetic samples were generated using the SMOTE algorithm, constructing a balanced dataset of 712 cases. After dividing the training and testing sets via 7:3 stratified sampling, a diagnostic model was built using a radial basis function (RBF) kernel support vector machine (SVM) and compared with a random forest model. Results showed that the SVM achieved an accuracy of 97.2%, precision of 98.1%, and recall of 96.3% on the testing set, with a 10-fold cross-validation average accuracy of 0.968 (with a fluctuation of 0.026); in contrast, the random forest model yielded corresponding indicators of 95.3%, 96.2%, 94.4%, and a 10-fold cross-validation average accuracy of 0.934 (with a fluctuation of 0.022), demonstrating that SVM significantly outperformed random forest. Additionally, the SVM achieved an AUC value of 0.968 on the ROC curve, showing excellent performance in the low false-positive range. Studies have shown that the SVM model based on features selected via the Pearson correlation coefficient exhibits significant advantages in breast cancer diagnosis. Compared with traditional diagnostic methods, such as mammography which has a misdiagnosis rate of 10%-15%, this model can reduce the risk of misdiagnosis by approximately 70%. Meanwhile, the model can effectively capture the clinical significance of microscopic nuclear features, not only providing a reliable tool for accurate breast cancer diagnosis but also promoting the in-depth integration of machine learning and pathological diagnosis.

Paraffin odor intensity is an important quality indicator when a paraffin inspection is performed. Currently, paraffin odor level assessment is mainly dependent on an artificial sensory evaluation. In this paper, we developed a paraffin odor analysis system to classify and grade four kinds of paraffin samples. The original feature set was optimized using Principal Component Analysis (PCA) and Partial Least Squares (PLS). Support Vector Machine (SVM), Random Forest (RF), and Extreme Learning Machine (ELM) were applied to three different feature data sets for classification and level assessment of paraffin. For classification, the model based on SVM, with an accuracy rate of 100%, was superior to that based on RF, with an accuracy rate of 98.33–100%, and ELM, with an accuracy rate of 98.01–100%. For level assessment, the R2 related to the training set was above 0.97 and the R2 related to the test set was above 0.87. Through comprehensive comparison, the generalization of the model based on ELM was superior to those based on SVM and RF. The scoring errors for the three models were 0.0016–0.3494, lower than the error of 0.5–1.0 measured by industry standard experts, meaning these methods have a higher prediction accuracy for scoring paraffin level.

BackgroundThe challenges in cancer diagnosis underline the need for continued research and development of new diagnostic tools and methods. This study aims to explore an effective, noninvasive, and convenient diagnostic tool using urine based near-infrared spectroscopy (NIRS) analysis combined with machine learning algorithm.MethodsUrine samples were collected from a total of 327 participants, including 181 cancer cases and 146 healthy controls. These participants were randomly spit into train set (n = 218) and test set (n = 109). NIRS analysis (4,000 ∼10,000 cm−1) was performed for each sample in both train and test sets. Five pretreatment methods, including Savitzky-Golay (SG) smoothing, multiplicative scatter correction (MSC), baseline removal (BSL) with fitting polynomials to be used as baselines, the first derivative (DERIV1), and the second derivative (DERIV2), and combination with “scaling” and “center”, were investigated. Then partial least-squares (PLS) and linear support-vector machine (SVM) classification models were established, and prediction performance was evaluated in test set.ResultsNIRS had greatly overlapping in peaks, and PCA analysis failed in separation between cancers and healthy controls. In modeling with urine based NIRS data, PLS model showed its highest prediction accuracy of 0.780, with DERIV2, “scaling” and “center” pretreatment, while linear SVM displayed its best prediction accuracy of 0.844, with raw NIRS. With optimization in SVM, the prediction accuracy could improve to 0.862, when the top 262 features were involved as variables.DiscussionThis pilot study combining urine based NIRS analysis and machine learning is effective and convenient that might facilitate in cancer diagnosis, encouraging further evaluation with a large-size multi-center study.

As a laboratory proximal sensing technique, the capability of visible and near-infrared (Vis-NIR) diffused reflectance spectroscopy with partial least squares (PLS) regression to determine soil properties has previously been demonstrated. However, the evaluation of the soil phosphorus (P) content—a major nutrient constraint for crop production in the tropics—is still a challenging task. PLS regression with waveband selection can improve the predictive ability of a calibration model, and a genetic algorithm (GA) has been widely applied as a suitable method for selecting wavebands in laboratory calibrations. To develop a laboratory-based proximal sensing method, this study investigated the potential to use GA-PLS regression analyses to estimate oxalate-extractable P in upland and lowland soils from laboratory Vis-NIR reflectance data. In terms of predictive ability, GA-PLS regression was compared with iterative stepwise elimination PLS (ISE-PLS) regression and standard full-spectrum PLS (FS-PLS) regression using soil samples collected in 2015 and 2016 from the surface of upland and lowland rice fields in Madagascar (n = 103). Overall, the GA-PLS model using first derivative reflectance (FDR) had the best predictive accuracy (R2 = 0.796) with a good prediction ability (residual predictive deviation (RPD) = 2.211). Selected wavebands in the GA-PLS model did not perfectly match wavelengths of previously known absorption features of soil nutrients, but in most cases, the selected wavebands were within 20 nm of previously known wavelength regions. Bootstrap procedures (N = 10,000 times) using selected wavebands also confirmed the improvements in accuracy and robustness of the GA-PLS model compared to those of the ISE-PLS and FS-PLS models. These results suggest that soil oxalate-extractable P can be predicted from Vis-NIR spectroscopy and that GA-PLS regression has the advantage of tuning optimum bands for PLS regression, contributing to a better predictive ability.

Fast discrimination of rice varieties plays a key role in the rice processing industry and benefits the management of rice in the supermarket. In order to discriminate rice varieties in a fast and nondestructive way, hyperspectral technology and several classification algorithms were used in this study. The hyperspectral data of 250 rice samples of 5 varieties were obtained using FieldSpec®3 spectrometer. Multiplication Scatter Correction (MSC) was used to preprocess the raw spectra. Principal Component Analysis (PCA) was used to reduce the dimension of raw spectra. To investigate the influence of different linear and non-linear classification algorithms on the discrimination results, K-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Least Square Support Vector Machine (LS-SVM) were used to develop the discrimination models respectively. Then the performances of these three multivariate classification methods were compared according to the discrimination accuracy. The number of Principal Components (PCs) and K parameter of KNN, kernel function of SVM or LS-SVM, were optimized by cross-validation in corresponding models. One hundred and twenty five rice samples (25 of each variety) were chosen as calibration set and the remaining 125 rice samples were prediction set. The experiment results showed that, the optimal PCs was 8 and the cross-validation accuracy of KNN (K = 2), SVM, LS-SVM were 94.4, 96.8 and 100%, respectively, while the prediction accuracy of KNN (K = 2), SVM, LS-SVM were 89.6, 93.6 and 100%, respectively. The results indicated that LS-SVM performed the best in the discrimination of rice varieties.

The incorporation of prior knowledge into the Support Vector Machine (SVM) architecture is a problem which if solved can lead to much more accurate classifiers in the near future. This result could be particularly effective in the classification of remote sensing imagery, where an abundance of information is available prior to classification. The most evident method to feed prior knowledge into the SVM algorithm is through the SVM kernel function. This paper proposes several composite kernel functions designed specifically for land cover classification of remote sensing imagery. These kernels make use of the spectral signature information, inherently available in remote sensing imagery. The results achieved from these kernels are very much satisfactory and surpass all previous results produced by classical kernels.