Collaborative Implementation of Hardware-Oriented GBDT Compress Algorithm Based on DSP+FPGA

With the large-scale development of wind energy, wind power forecasting plays a key role in power dispatching in the electric power grid, as well as in the operation and maintenance of wind farms. The most important technology for wind power forecasting is forecasting wind speed. The current mainstream methods for wind speed forecasting involve the combination of mesoscale numerical meteorological models with a post-processing system. Our work uses the WRF model to obtain the numerical weather forecast and the gradient boosting decision tree (GBDT) algorithm to improve the near-surface wind speed post-processing results of the numerical weather model. We calculate the feature importance of GBDT in order to find out which feature most affects the post-processing wind speed results. The results show that, after using about 300 features at different height and pressure layers, the GBDT algorithm can output more accurate wind speed forecasts than the original WRF results and other post-processing models like decision tree regression (DTR) and multi-layer perceptron regression (MLPR). Using GBDT, the root mean square error (RMSE) of wind speed can be reduced from 2.7–3.5 m/s in the original WRF result by 1–1.5 m/s, which is better than DTR and MLPR. While the index of agreement (IA) can be improved by 0.10–0.20, correlation coefficient be improved by 0.10–0.18, Nash–Sutcliffe efficiency coefficient (NSE) be improved by −0.06–0.6. It also can be found that the feature which most affects the GBDT results is the near-surface wind speed. Other variables, such as forecast month, forecast time, and temperature, also affect the GBDT results.

Precipitable water vapor (PWV) facilitates the exchange of water and energy between the Earth's surface and the surrounding atmosphere. PWV can be retrieved by employing the water vapor absorption channels in Landsat 8. Unfortunately, the conventional split-window covariance-variance ratio (SWCVR) method is susceptible to various factors, resulting in low retrieval accuracy and availability. Therefore, we improve the traditional SWCVR model and propose a new PWV retrieval method using ensemble machine learning to address these limitations. The new method uses Gradient Boosting Decision Tree (GBDT) to establish the model between brightness temperature, GNSS-derived PWV, and related surface parameters. The results of the test set show that the improved SWCVR model has an RMSE, Bias, and availability of 0.4947 g/cm2, 0.0276 g/cm2, and 29.6%, respectively. By contrast, the GBDT model's corresponding values are 0.2870 g/cm2, −0.0094 g/cm2, and 67.9%, respectively. Compared with SWCVR, the GBDT improves the RMSE and availability by 41.99% and 38.3%, respectively. The GBDT algorithm is significantly better than the SWCVR model in low altitude and complex surface coverage areas. From a temporal perspective, the advantages of the GBDT method are more apparent in the summer. Finally, the SWCVR and GBDT models are externally validated using measured PWV data from the sun photometer and radiosonde, and the RMSE is 0.5148 g/cm2 and 0.3775 g/cm2, respectively. Based on the findings, we know that the accuracy of GBDT has significantly improved against the SWCVR.

Cancer is the second leading cause of disease-related death worldwide, and machine learning-based identification of novel biomarkers is crucial for improving early detection and treatment of various cancers. A key challenge in applying machine learning to high-dimensional data is deriving important features in an interpretable manner to provide meaningful insights into the underlying biological mechanismsWe developed a class-based directional feature importance (CLIFI) metric for decision tree methods and demonstrated its use for The Cancer Genome Atlas proteomics data. The CLIFI metric was incorporated into four algorithms, Random Forest (RF), LAtent VAriable Stochastic Ensemble of Trees (LAVASET), and Gradient Boosted Decision Trees (GBDTs), and a new extension incorporating the LAVA step into GBDTs (LAVABOOST). Both LAVA methods incorporate topological information from protein interactions into the decision function.The different models' performance in classifying 28 cancers resulted in F1-scores of 92.6% (RF), 92.0% (LAVASET), 89.3% (LAVABOOST) and 85.7% (GBDT), with no method outperforming all others for individual cancer type prediction. The CLIFI metric enables visualisation of the model's decision-making functions. The resulting CLIFI value distributions indicated heterogeneity in the expression of several proteins (MYH11, ERα, BCL2) across different cancer types (including brain glioma, breast, kidney, thyroid and prostate cancer) aligning with the original raw expression data.In conclusion, we have developed an integrated, directional feature importance metric for multi-class decision tree-based classification models that facilitates interpretable feature importance assessment. The CLIFI metric can be combined with incorporating topological information into the decision functions of models to introduce inductive bias, enhancing interpretability.

Machine learning-based landslide susceptibility assessment with optimized ratio of landslide to non-landslide samples

In recent studies, researchers have focused on using various modalities to recognize emotions for different applications. A major challenge is identifying emotions correctly with only electrocardiograms (ECG) as the modality. The main objective is to reduce costs by using single-modality ECG signals to predict human emotional states. This paper presents an emotion recognition approach utilizing the heart rate variability features obtained from ECG with feature selection techniques (exhaustive feature selection (EFS) and Pearson’s correlation) to train the classification models. Seven machine learning (ML) models: multi-layer perceptrons (MLP), Support Vector Machine (SVM), Decision Tree (DT), Gradient Boosting Decision Tree (GBDT), Logistic Regression, Adaboost and Extra Tree classifier are used to classify emotional state. Two public datasets, DREAMER and SWELL are used for evaluation. The results show that no particular ML works best for all data. For DREAMER with EFS, the best models to predict valence, arousal, and dominance are Extra Tree (74.6%), MLP and DT (74.6%), and GBDT and DT (69.8%), respectively. Extra tree with Pearson’s correlation are the best method for the ECG SWELL dataset and provide 100% accuracy. The usage of Extra tree classifier and feature selection technique contributes to the improvement of the model accuracy. Moreover, the Friedman test proved that ET is as good as other classification models for predicting human emotional state and ranks highest. Doi: 10.28991/ESJ-2023-07-01-011 Full Text: PDF

Cyberspace is a concept describing a widespread, interconnected digital technology with numerous users. New standards add more concerns with tremendous information gathered from various network sources, which can be utilized for focused cyber-attacks. Digital attacks in networks are getting more complex and subsequently introducing expanding difficulties in precisely distinguishing network intrusions. Inability to forestall the network intrusions could debase the validity of safety administrations, for example, information privacy, trustworthiness, and accessibility. In this work, we center around developing a Network Intrusion Detection System which is implemented utilizing Machine Learning Techniques. IDS dependent on ML techniques are successful and exact in distinguishing varied networks intrusions. Also, many of the ML-based IDS experience the ill effects of an increment in false-positive rate, leading to lower precision and accuracy. Consequently, we present an exploration of the UNSW-NB15 intrusion detection dataset that will be utilized for preparing the machine learning models. In our tests, we carry out various ML approaches namely, Support Vector Machine (SVM), Logistic Regression (LR), Random Forest Classifier (RF), Decision Tree (DT), Gradient boosted decision trees (GBDT). The results demonstrated that the hyperparameter optimization technique using Grid Search and Random Search methods and evaluation using k-fold cross-validation gives better optimization of model parameters and henceforth brings about better execution.KeywordsCross-validationCyberattackDecision tree (DT)Gradient boosted decision trees (GBDT)Hyperparameter TuningLogistic regression (LR)Network intrusion detection system (IDS)Random forest classifier (RF)Support vector machine (SVM)

BackgroundTongue diagnosis is an important research field of TCM diagnostic technology modernization. The quality of tongue images is the basis for constructing a standard dataset in the field of tongue diagnosis. To establish a standard tongue image database in the TCM industry, we need to evaluate the quality of a massive number of tongue images and add qualified images to the database. Therefore, an automatic, efficient and accurate quality control model is of significance to the development of intelligent tongue diagnosis technology for TCM.MethodsMachine learning methods, including Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), Adaptive Boosting Algorithm (Adaboost), Naïve Bayes, Decision Tree (DT), Residual Neural Network (ResNet), Convolution Neural Network developed by Visual Geometry Group at University of Oxford (VGG), and Densely Connected Convolutional Networks (DenseNet), were utilized to identify good-quality and poor-quality tongue images. Their performances were made comparisons by using metrics such as accuracy, precision, recall, and F1-Score.ResultsThe experimental results showed that the accuracy of the three deep learning models was more than 96%, and the accuracy of ResNet-152 and DenseNet-169 was more than 98%. The model ResNet-152 obtained accuracy of 99.04%, precision of 99.05%, recall of 99.04%, and F1-score of 99.05%. The performances were better than performances of other eight models. The eight models are VGG-16, DenseNet-169, SVM, RF, GBDT, Adaboost, Naïve Bayes, and DT. ResNet-152 was selected as quality-screening model for tongue IQA.ConclusionsOur research findings demonstrate various CNN models in the decision-making process for the selection of tongue image quality assessment and indicate that applying deep learning methods, specifically deep CNNs, to evaluate poor-quality tongue images is feasible.

Distribution network topology identification based on gradient boosting decision tree and attribute weighted naive Bayes

In order to improve the detection accuracy of the data tampering attack of the Power Cyber-physical System, a method of Power CPS attack detection based on Gradient Boosting Decision Tree is proposed in this paper from the angle of artificial intelligence. Firstly, by setting up the iForest anomaly value equation of physical system and IDS Shannon entropy function of information system, the feature extraction of data tamper detection is realized. Then, the CART decision tree is taken as the base learner of the detection classification, the target is minimized by the loss function, and the high intensity attack detection model is designed by iterative combination. Finally, the three-dimensional adaptive chaotic FOA algorithm is proposed for dynamic optimization of model parameters, an attack detection model under training optimal parameters. The analysis of the example shows that show that the proposed method can effectively detect power CPS data tampering attacks and has excellent detection precision.

Concrete carbonation depth prediction model based on a gradient-boosting decision tree and different metaheuristic algorithms

The weak classifier ensemble algorithms based on the decision tree model, mainly include bagging (e.g., fandom forest-RF) and boosting (e.g., gradient boosting decision tree, eXtreme gradient boosting), the former reduces the variance for the overall generalization error reduction while the latter focuses on reducing the overall bias to that end. Because of its straightforward idea, it is prevalent in MPM (mineral prospectivity mapping). However, an inevitable problem in the application of such methods is the hyperparameters tuning which is a laborious and time-consuming task. The selection of hyperparameters suitable for a specific task is worth investigating. In this paper, a tree Parzen estimator-based GBDT (gradient boosting decision tree) model (TPE-GBDT) was introduced for hyperparameters tuning (e.g., loss criterion, n_estimators, learning_rate, max_features, subsample, max_depth, min_impurity_decrease). Then, the geological data of the gold deposit in the Xiong ‘ershan area was used to create training data for MPM and to compare the TPE-GBDT and random search-GBDT training results. Results showed that the TPE-GBDT model can obtain higher accuracy than random search-GBDT in a shorter time for the same parameter space, which proves that this algorithm is superior to random search in principle and more suitable for complex hyperparametric tuning. Subsequently, the validation measures, five-fold cross-validation, confusion matrix and success rate curves were employed to evaluate the overall performance of the hyperparameter optimization models. The results showed good scores for the predictive models. Finally, according to the maximum Youden index as the threshold to divide metallogenic potential areas and non-prospective areas, the high metallogenic prospect area (accounts for 10.22% of the total study area) derived by the TPE-GBDT model contained > 90% of the known deposits and provided a preferred range for future exploration work.

This survey examines interpretability methods used or proposed for Gradient Boosting Decision Trees, which are advanced machine learning algorithms based on decision trees. The studies analyzed were gathered using synonyms for "explainability" combined with synonyms for "method," as well as synonyms for "Gradient Boosting Decision Trees." The proposed or applied approaches are classified by their techniques and described in detail. Among these methods, we recommend using SHAP values to rank features based on their relevance, as this approach aligns well with the structure of Gradient Boosting Decision Trees. Additionally, we suggest considering inTrees, RULECOSI+, and Tree Space Prototypes when applicable.

In recent years, the vulnerability attack on the industrial control system appears more organized and diverse. In this paper, we focus on power user electric energy data acquisition system and its communication protocol, namely 376.1 master station communication protocol. The system is an important infrastructure in national economy and people’s livelihood. To efficiently discover abnormal behaviors during its communication, we propose a terminal access data anomaly detection model based on gradient boosting decision tree (GBDT). Firstly, through analyzing the characteristics of the communication protocol and different kinds of terminal access data, we construct a high-quality multidimensional feature set. Then we choose GBDT as the abnormal access data detection model. The experimental result shows that the detection model has a high detection accuracy and outperforms its counterparts.

Decision forests, particularly Gradient Boosting Decision Trees (GBDT), are popular due to their high prediction performance and computational efficiency, making them suitable for embedded systems with circuit size and available energy constraints. In this study, we propose a new lightweight GBDT inference acceleration mechanism through the hardware and algorithm co-design. First, we present LoADPack, a hardware-friendly GBDT algorithm that enhances memory access locality. LoADPack obtains trees where the features and thresholds used across the entire ensemble are regular regardless of a branching direction by unifying some nodes and aligning the memory access patterns. Furthermore, we present DF-BETA, a resource-efficient accelerator for the LoADPack algorithm. DF-BETA utilizes MSB-first bit-serial computation to enable early determination of comparison calculations of 32-bit floating-point numbers, optimizing the operation for determining a branch direction. The hardware complexity and computation termination speed vary with the granularity of bit-serial computation. Therefore, we conduct design space exploration of DF-BETA to identify the optimal configuration. Our findings reveal that using 4-bit-serial comparators minimizes circuit size while achieving the leading throughput. Compared to running unconstrained GBDT on a typical accelerator with 32-bit bit-parallel comparators, our accelerator achieves 1.6 times higher throughput on average while maintaining comparable accuracy.

Prediction model for daily reference crop evapotranspiration based on hybrid algorithm and principal components analysis in Southwest China