Comparison of machine learning regression algorithms for foot placement prediction

Accurate prediction of food crop yield is of great significance for global food security and regional trade stability. Since remote sensing data collected from unmanned aerial vehicle (UAV) platforms have the features of flexibility and high resolution, these data can be used as samples to develop regional regression models for accurate prediction of crop yield at a field scale. The primary objective of this study was to construct regional prediction models for winter wheat yield based on multi-spectral UAV data and machine learning methods. Six machine learning methods including Gaussian process regression (GPR), support vector machine regression (SVR) and random forest regression (RFR) were used for the construction of the yield prediction models. Ten vegetation indices (VIs) extracted from canopy spectral images of winter wheat acquired from a multi-spectral UAV at five key growth stages in Xuzhou City, Jiangsu Province, China in 2021 were selected as the variables of the models. In addition, in situ measurements of wheat yield were obtained in a destructive sampling manner for prediction algorithm modeling and validation. Prediction results of single growth stages showed that the optimal model was GPR constructed from extremely strong correlated VIs (ESCVIs) at the filling stage (R2 = 0.87, RMSE = 49.22 g/m2, MAE = 42.74 g/m2). The results of multiple stages showed GPR achieved the highest accuracy (R2 = 0.88, RMSE = 49.18 g/m2, MAE = 42.57 g/m2) when the ESCVIs of the flowering and filling stages were used. Larger sampling plots were adopted to verify the accuracy of yield prediction; the results indicated that the GPR model has strong adaptability at different scales. These findings suggest that using machine learning methods and multi-spectral UAV data can accurately predict crop yield at the field scale and deliver a valuable application reference for farm-scale field crop management.

The process of extracting meaningful rules from big and complex data is called data mining. Data mining has an increasing popularity in every field today. Data units are established in customer-oriented industries such as marketing, finance and telecommunication to work on the customer churn and acquisition, in particular. Among the data mining methods, classification algorithms are used in studies conducted for customer acquisition to predict the potential customers of the company in question in the related industry. In this study, bank marketing data set in UCI Machine Learning Data Set was used by creating models with the same classification algorithms in different data mining programs. Accuracy, precision and f- measure criteria were used to test performances of the classification models. When creating the classification models, the test and training data sets were randomly divided by the holdout method to evaluate the performance of the data set. The data set was divided into training and test data sets with the 60-40%, 75­25% and 80-20% separation ratios. Data mining programs used for these processes are the R, Knime, RapidMiner and WEKA. And, classification algorithms commonly used in these platforms are the k-nearest neighbor (k-nn), Naive Bayes, and C4.5 decision tree.

A surrogate model assisted with a subdomain model for permanent-magnet machine

This focus issue of Physiological Measurement follows the 38th Annual International Computing in Cardiology (CinC) Conference, hosted in Hangzhou, China in September 2011 by Zhejiang University. Each year, the NIH-sponsored PhysioNet resource (http://physionet.org/) runs an open competition lasting several months, aimed at encouraging the development of solutions to an unsolved or poorly solved problem in biomedicine, in most cases making use of relevant clinical and experimental data provided freely by PhysioNet. Participants in these annual challenges discuss their diverse approaches to the Challenge problems during dedicated scientific sessions at CinC. The topics of these PhysioNet/CinC Challenges range from physiologic signal processing and analysis to forecasting and modelling clinically important events and processes.

In this study, we reviewed methods, sensors and study areas in estimation of vegetation biophysical (VBph) and biochemical (VBch) parameters using satellite optical remote sensing, in terms of (1) their performance and (2) extent of their use. Performance of parametric regression methods is dependent on the type of sensor, vegetation parameter, vegetation cover and biome of study area. Artificial neural network, partial least-squares regression, random forest, support vector machine and Gaussian process regression (GPR) had acceptable performances in the estimations. Performance of physically based methods depends on selecting appropriate radiative transfer model, inversion method and cost function, optimal solution (e.g. mean of some of the best lookup-table solutions) and adding appropriate Gaussian noise. In recent years, the use of active learning techniques (especially Euclidean distance-based diversity), to optimize training datasets against field datasets, with machine learning regression algorithms (MLRAs) (especially GPR) has led to acceptable performances by hybrid methods in estimation of VBph and VBch parameters using real and/or simulated (Re/Si) satellite hyperspectral (and multispectral) data with dimensionality reduction techniques in the spectral domain (especially principal component analysis with 20 principal components). In overall, Re/Si satellite optical sensors data containing red-edge (mostly for VBch parameters) and/or short-wave infrared (mostly for VBph parameters) spectral band(s) performed well. Emerging trend observed in recent years in the estimations by using Re/Si medium spatial resolution sensors data with relatively good spectral resolutions (e.g. precursore iperspettrale della missione applicativa, environmental mapping and analysis program and Sentinel-2 multispectral instrument) and non-parametric regression (especially MLRAs) and hybrid methods.

IFN-signaling gene (ISG) expression scores are potential markers of inflammation with significance from cancer to genetic syndromes. In Aicardi Goutières Syndrome (AGS), a disorder of abnormal DNA and RNA metabolism, this score has potential as a diagnostic biomarker, although the approach to ISG calculation has not been standardized or validated. To optimize ISG calculation and validate ISG as a diagnostic biomarker, mRNA levels of 36 type I IFN response genes were quantified from 997 samples (including 334 AGS), and samples were randomized into training and test data sets. An independent validation cohort (n = 122) was also collected. ISGs were calculated using all potential combinations up to 6 genes. A 4-gene approach (IFI44L, IFI27, USP18, IFI6) was the best-performing model (AUC of 0.8872 [training data set], 0.9245 [test data set]). The majority of top-performing gene combinations included IFI44L. Performance of IFI44L alone was 0.8762 (training data set) and 0.9580 (test data set) by AUC. The top approaches were able to discriminate individuals with genetic interferonopathy from control samples. This study validates the context of use for the ISG score as a diagnostic biomarker and underscores the importance of IFI44L in diagnosis of genetic interferonopathies.

The role of pertinently diversified and balanced training as well as testing data sets in achieving the true performance of classifiers in predicting the antifreeze proteins

Organic-rich substrates from organic waste effluents are ideal sources for hydrogen production based on the circular economy concept. In this study, a data-driven approach was employed in modeling hydrogen production from palm oil mill effluents and activated sludge waste. Seven models built on support vector machine (SVM) and Gaussian process regression (GPR) were employed for the modeling of the hydrogen production from the waste sources. The SVM was incorporated with linear kernel function (LSVM), quadratic kernel function (QSVM), cubic kernel function (CSVM), and Gaussian fine kernel function (GFSVM). While the GPR was incorporated with the rotational quadratic kernel function (RQGPR), squared exponential kernel function (SEGPR), and exponential kernel function (EGPR). The model performance revealed that the SVM-based models did not show impressive performance in modeling the hydrogen production from the palm oil mill effluent, as indicated by the R2 of −0.01, 0.150, and 0.143 for LSVM, QSVM, and CSVM, respectively. Similarly, the SVM-based models did not perform well in modeling the hydrogen production from activated sludge, as evidenced by R2 values of 0.040, 0.190, and 0.340 for LSVM, QSVM, and CSVM, respectively. On the contrary, the SEGPR, RQGPR, SEGPR, and EGPR models displayed outstanding performance in modeling the prediction of hydrogen production from both oil palm mill effluent and activated sludge, with over 90% of the datasets explaining the variation in the model output. With the R2 > 0.9, the predicted hydrogen production was consistent with the SEGPR, RQGPR, SEGPR, and EGPR with minimized prediction errors. The level of importance analysis revealed that all the input parameters are relevant in the production of hydrogen. However, the influent chemical oxygen demand (COD) concentration and the medium temperature significantly influenced the hydrogen production from palm oil mill effluent, whereas the pH of the medium and the temperature significantly influenced the hydrogen production from the activated sludge.

Weld bead geometry is an important characteristic of the weld joints in evaluating its quality. However, weld geometry is characterized by three parameters namely weld width, weld height and weld depth. The measurement system of three parameters is often time consuming, particularly for intelligent robots used in the welding processes. Hence, in this paper, we introduced a penetration coefficient (PC) that effectively minimizes the complexities involved in existing measurement system of weld geometry. Further, various machine learning approaches are used to predict the penetration coefficient. Cold metal transfer welded AA6061 sheets are chosen to obtain the data of penetration coefficient. Linear regression (LR), support vector machine (SVM) regression and Gaussian process regression (GPR) models and artificial neural network (ANN) model are used for predictive modelling. The statistical performance factors of the models reveal the superior performance of ANN model. The lowest mean absolute error of 0.15 is observed for ANN followed by the SVM (0.31), GPR (0.39) and LR (0.41).

Improvement in artificial neural network-based estimation of grid connected photovoltaic power output

A major objective of this project was to apply GE’s commercial machine learning and data analytics toolsets to large-scale, real-world, anonymized Phasor Measurement Unit (PMU) datasets in order to extract signatures, correlated and/or causal factors, and precursor patterns associated with significant power system phenomena. The project had a particular emphasis on extraction of insights relevant to asset health monitoring, real-time load modeling and cybersecurity monitoring. Additionally, the team was directed to undertake a comprehensive data quality analysis for the provided datasets and encouraged to estimate the ‘machine-learning readiness’ of the datasets by documenting any major obstacles to the application of commercial machine learning algorithms. To accomplish the aforementioned objectives, the project team’s work centered around the identification of key event signatures and application of the identified event signatures for event detection and event classification. The industry-validated, semi-supervised machine learning strategy employed for event signature identification involved several major tasks, including data-preprocessing, generation of an overabundance of features, normal data identification, normality modeling, and event signature identification through a methodical, quantitative ranking of features in order of relevance to each studied event type. Throughout the project, data quality issues and mitigation techniques were investigated. In this report, insights are provided regarding the readiness of the provided synchrophasor datasets for application of machine learning and data analytics. The methodologies employed for this technical strategy are summarized in this report. With regards to data preprocessing and feature generation, the provided Training and Test Datasets were ingested into GE’s big data environment. Subsequently, the team applied bad data cleansing and data imputation scripts, event detection scripts, and application programming interfaces (APIs) to the datasets for convenient data access. The project team completed development and validation of dozens of physics-based, statistics-based and transformation-based feature functions used for the extraction of over 60 synchrophasor features. Using a new parallel feature generation technology developed on this project, over 60 features have been rapidly generated for the full two years’ worth of Training and Test Dataset data associated with both the Eastern and Western interconnects. Even accommodating for temporal down-sampling inherent to the feature extraction procedure, this parallel feature generation activity resulted in a massive feature set with a storage requirement approximately equal to that of the raw training dataset itself. With regards to normal data identification and normality modeling, a normality model was built using the feature data extracted from the Training Dataset and iteratively refined subsequent to incremental adjustments and expansions of the Training Dataset feature data. With respect to event characterization and signature identification, an event signature identification pipeline was developed and used in conjunction with the normality model to identify over 15 event signatures for key event categories within the Training Dataset. The identified event signatures were used to characterize hundreds of key events in terms of relative severity, duration, and location of the event. An investigation was undertaken to identify correlated and causal factors involved in transformer events. A separate investigation into temporal trends in ring-down analysis results was undertaken to determine possible associations between system dynamics and various other factors such as loading, season or year. To validate the identified event signatures, additional work was undertaken to develop signature-based anomaly detection and classification tools suitable for convenient application to the synchrophasor datasets. The anomaly detection and classification tools, suitable for online application, were then applied to the entirety of the Eastern Interconnect Training and Test Datasets. Performance of the event detection and classification tools was evaluated upon receipt of the Test Dataset event logs (i.e., the labels for events contained in the Test Dataset), and promising results were obtained despite several challenges (documented herein) associated with application of supervised or semi-supervised machine learning methods to large-scale, anonymized datasets. Finally, the detection and classification tools were used to detect, classify, and characterize thousands of new events not included in the original event logs provided by the DOE within both the Training and Test Datasets.

Constant power loads (CPLs) in the DC microgrids will lead to the instability of the bus voltage, so the power variation range needs to be limited. In this paper, a based on machine learning critical value prediction method is proposed for CPLs. Firstly, Pearson correlation analysis is used to find the factors that have effects on CPLs critical value in terms of droop coefficient and bus voltage. Then, support vector machine and Gaussian process regression prediction model of CPLs critical value are established. Finally, different scenarios of DC microgrid are established to verify the proposed algorithms. The results show that machine learning algorithms can accurately predict the critical value of CPLs, and compared with support vector machine, Gaussian process regression method has higher prediction accuracy and universality.

The current exponential increase of spatiotemporally explicit data streams from satellitebased Earth observation missions offers promising opportunities for global vegetation monitoring. Intelligent sampling through active learning (AL) heuristics provides a pathway for fast inference of essential vegetation variables by means of hybrid retrieval approaches, i.e., machine learning regression algorithms trained by radiative transfer model (RTM) simulations. In this study we summarize AL theory and perform a brief systematic literature survey about AL heuristics used in the context of Earth observation regression problems over terrestrial targets. Across all relevant studies it appeared that: (i) retrieval accuracy of AL-optimized training data sets outperformed models trained over large randomly sampled data sets, and (ii) Euclidean distance-based (EBD) diversity method tends to be the most efficient AL technique in terms of accuracy and computational demand. Additionally, a case study is presented based on experimental data employing both uncertainty and diversity AL criteria. Hereby, a a simulated training data base by the PROSAIL-PRO canopy RTM is used to demonstrate the benefit of AL techniques for the estimation of total leaf carotenoid content (Cxc) and leaf water content (Cw). Gaussian process regression (GPR) was incorporated to minimize and optimize the training data set with AL. Training the GPR algorithm on optimally AL-based sampled data sets led to improved variable retrievals compared to training on full data pools, which is further demonstrated on a mapping example. From these findings we can recommend the use of AL-based sub-sampling procedures to select the most informative samples out of large training data pools. This will not only optimize regression accuracy due to exclusion of redundant information, but also speed up processing time and reduce final model size of kernel-based machine learning regression algorithms, such as GPR. With this study we want to encourage further testing and implementation of AL sampling methods for hybrid retrieval workflows. AL can contribute to the solution of regression problems within the framework of operational vegetation monitoring using satellite imaging spectroscopy data, and may strongly facilitate data processing for cloud-computing platforms.

Rapid Detection of Clostridium difficile Toxins in Stool by Raman Spectroscopy

The aim of this study is to present a method of assessing eight types of mood that is optimized to every individual on the basis of the heart rate variability (HRV) data which, to eliminate the influence of the inter-individual variability, are measured in a long time period during daily life. Eight types of mood are happiness, tension, fatigue, anxiety, depression, anger, vigor, and confusion. HRV and body accelerations were recorded from nine normal subjects for two months of normal daily life. Fourteen HRV indices were calculated with the HRV data at 512 seconds prior to the time of every mood level report. Data to be analyzed were limited to those with body accelerations of 30 mG (0.294 m/s2) and lower. Further, the differences from the reference values in the same time zone were calculated with both the mood score (Δmood) and HRV index values (ΔHRVI). The multiple linear regression model that estimates Δmood from the scores for principal components of ΔHRVI were then constructed for each individual. The data were divided into training data set and test data set in accordance with the 2-fold cross validation method. Multiple linear regression coefficients were determined using the training data set, and with the optimized model its generalization capability was checked using the test data set. The model was most effective on estimating tension compared with other seven types of mood. The subjects' mean Pearson correlation coefficient was 0.52 with the training data set and 0.40 with the test data set. We proposed a method of assessing mood that is optimized to every individual based on HRV data measured over a long period of daily life.