Robust Watermarking on Gradient Boosting Decision Trees

Distribution network topology identification based on gradient boosting decision tree and attribute weighted naive Bayes

Radiomics and machine learning have been extensively utilized in the realm of urinary stones, particularly in forecasting stone treatment outcomes. The objective of this study was to integrate clinical variables and radiomic features to develop a machine learning model for predicting the stone-free rate (SFR) following percutaneous nephrolithotomy (PCNL).A total of 212 eligible patients who underwent PCNL surgery at the Second Affiliated Hospital of Nanchang University were included in a retrospective analysis. Preoperative clinical variables and non-contrast-enhanced CT images of all patients were collected, and radiomic features were extracted after delineating the stone ROI. Univariate analysis was conducted to identify clinical variables strongly correlated with the stone-free rate after PCNL, and the least absolute shrinkage and selection operator algorithm (lasso regression) was utilized to screen radiomic features. Four supervised machine learning algorithms, including Logistic Regression, Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Gradient Boosting Decision Tree (GBDT), were employed. The clinical variables with strong correlation and screened radiomic features were integrated into the four machine learning algorithms to construct a prediction model, and the receiver operating curve was plotted. The area under the receiver operating curve (AUC), the accuracy rate, the specificity, etc., were used to evaluate the predictive performance of the four models.After analyzing postoperative statistics, the stone-free rate following the procedure was found to be 70.3% (n = 149). Among the various clinical variables examined, factors, such as stone number, stone diameter, stone CT value, stone location, and history of stone surgery, were identified as statistically significant in relation to the stone-free rate after PCNL. A total of 121 radiomic features were extracted, and through lasso regression, 7 features most closely associated with the stone-free rate post-PCNL were identified. The predictive accuracy of different models (Logistic Regression, RF, XGBoost, and GBDT) for determining the stone-free rate after PCNL was evaluated, yielding accuracies of 78.1%, 76.6%, 75.0%, and 73.4%, respectively. The corresponding area under the curve AUC (95%CI) were 0.85 (0.83-0.89), 0.81 (0.76-0.85), 0.82 (0.78-0.85), and 0.77 (0.73-0.81), positioning these models among the top performers in logistic regression prediction. In terms of predictive importance scores, the key factors identified by the logistic regression model were number of stone, zone percentage, stone diameter, and surface area. Similarly, the RF model highlighted number of stone, stone CT value, stone diameter, and surface area as the top predictors.Among the four machine learning models, the logistic regression model demonstrated the highest accuracy and discrimination ability in predicting the stone-free rate following PCNL. In comparison to XGBoost and GBDT, RF also exhibited superior accuracy and a certain level of discrimination ability. However, based on the performance of all four models, logistic regression is more likely to aid in clinical decision-making by assisting clinicians in diagnosing PCNL in patients. This enables us to effectively predict the presence of residual stones post-surgery and ultimately select patients who are suitable candidates for PCNL.

Precipitable water vapor (PWV) facilitates the exchange of water and energy between the Earth's surface and the surrounding atmosphere. PWV can be retrieved by employing the water vapor absorption channels in Landsat 8. Unfortunately, the conventional split-window covariance-variance ratio (SWCVR) method is susceptible to various factors, resulting in low retrieval accuracy and availability. Therefore, we improve the traditional SWCVR model and propose a new PWV retrieval method using ensemble machine learning to address these limitations. The new method uses Gradient Boosting Decision Tree (GBDT) to establish the model between brightness temperature, GNSS-derived PWV, and related surface parameters. The results of the test set show that the improved SWCVR model has an RMSE, Bias, and availability of 0.4947 g/cm2, 0.0276 g/cm2, and 29.6%, respectively. By contrast, the GBDT model's corresponding values are 0.2870 g/cm2, −0.0094 g/cm2, and 67.9%, respectively. Compared with SWCVR, the GBDT improves the RMSE and availability by 41.99% and 38.3%, respectively. The GBDT algorithm is significantly better than the SWCVR model in low altitude and complex surface coverage areas. From a temporal perspective, the advantages of the GBDT method are more apparent in the summer. Finally, the SWCVR and GBDT models are externally validated using measured PWV data from the sun photometer and radiosonde, and the RMSE is 0.5148 g/cm2 and 0.3775 g/cm2, respectively. Based on the findings, we know that the accuracy of GBDT has significantly improved against the SWCVR.

Comparison of machine learning and deep learning models for the assessment of rondo wine grape quality with a hyperspectral camera

In recent years, the vulnerability attack on the industrial control system appears more organized and diverse. In this paper, we focus on power user electric energy data acquisition system and its communication protocol, namely 376.1 master station communication protocol. The system is an important infrastructure in national economy and people’s livelihood. To efficiently discover abnormal behaviors during its communication, we propose a terminal access data anomaly detection model based on gradient boosting decision tree (GBDT). Firstly, through analyzing the characteristics of the communication protocol and different kinds of terminal access data, we construct a high-quality multidimensional feature set. Then we choose GBDT as the abnormal access data detection model. The experimental result shows that the detection model has a high detection accuracy and outperforms its counterparts.

Decision forests, particularly Gradient Boosting Decision Trees (GBDT), are popular due to their high prediction performance and computational efficiency, making them suitable for embedded systems with circuit size and available energy constraints. In this study, we propose a new lightweight GBDT inference acceleration mechanism through the hardware and algorithm co-design. First, we present LoADPack, a hardware-friendly GBDT algorithm that enhances memory access locality. LoADPack obtains trees where the features and thresholds used across the entire ensemble are regular regardless of a branching direction by unifying some nodes and aligning the memory access patterns. Furthermore, we present DF-BETA, a resource-efficient accelerator for the LoADPack algorithm. DF-BETA utilizes MSB-first bit-serial computation to enable early determination of comparison calculations of 32-bit floating-point numbers, optimizing the operation for determining a branch direction. The hardware complexity and computation termination speed vary with the granularity of bit-serial computation. Therefore, we conduct design space exploration of DF-BETA to identify the optimal configuration. Our findings reveal that using 4-bit-serial comparators minimizes circuit size while achieving the leading throughput. Compared to running unconstrained GBDT on a typical accelerator with 32-bit bit-parallel comparators, our accelerator achieves 1.6 times higher throughput on average while maintaining comparable accuracy.

Machine learning-based landslide susceptibility assessment with optimized ratio of landslide to non-landslide samples

With the large-scale development of wind energy, wind power forecasting plays a key role in power dispatching in the electric power grid, as well as in the operation and maintenance of wind farms. The most important technology for wind power forecasting is forecasting wind speed. The current mainstream methods for wind speed forecasting involve the combination of mesoscale numerical meteorological models with a post-processing system. Our work uses the WRF model to obtain the numerical weather forecast and the gradient boosting decision tree (GBDT) algorithm to improve the near-surface wind speed post-processing results of the numerical weather model. We calculate the feature importance of GBDT in order to find out which feature most affects the post-processing wind speed results. The results show that, after using about 300 features at different height and pressure layers, the GBDT algorithm can output more accurate wind speed forecasts than the original WRF results and other post-processing models like decision tree regression (DTR) and multi-layer perceptron regression (MLPR). Using GBDT, the root mean square error (RMSE) of wind speed can be reduced from 2.7–3.5 m/s in the original WRF result by 1–1.5 m/s, which is better than DTR and MLPR. While the index of agreement (IA) can be improved by 0.10–0.20, correlation coefficient be improved by 0.10–0.18, Nash–Sutcliffe efficiency coefficient (NSE) be improved by −0.06–0.6. It also can be found that the feature which most affects the GBDT results is the near-surface wind speed. Other variables, such as forecast month, forecast time, and temperature, also affect the GBDT results.

GBDT (Gradient Boosting Decision Tree) is an algorithm that builds multiple decision trees by iteratively updating. Due to its strong generalization ability and fast operation speed, it is widely used in the realization of classification and regression. However, the GBDT classification model occupies large storage resources; its application on the mobile multimedia platform is greatly limited. To solve this problem, we come up with a new GBDT compression algorithm for hardware implementation [1]. And based on this algorithm, we designed a DSP+FPGA system to achieve four classifications of six-dimensional samples. DSP is responsible for classification control, while the FPGA implements the classification calculation process. This classification system not only occupies small hardware resources but also guarantee accuracy and speed.

A robust model for diagnosing water stress of winter wheat by combining UAV multispectral and thermal remote sensing

Prediction model for daily reference crop evapotranspiration based on hybrid algorithm and principal components analysis in Southwest China

Big data analytics (BDA) is a system based method with an aim to recognize and examine different designs, patterns and trends under the big dataset. In this paper, BDA is used to visualize and trends the prediction where exploratory data analysis examines the crime data. “A successive facts and patterns have been taken in following cities of California, Washington and Florida by using statistical analysis and visualization”. The predictive result gives the performance using Keras Prophet Model, LSTM and neural network models followed by prophet model which are the existing methods used to find the crime data under BDA technique. But the crime actions increases day by day which is greater task for the people to overcome the challenging crime activities. Some ignored the essential rate of influential aspects. To overcome these challenging problems of big data, many studies have been developed with limited one or two features. “This paper introduces a big data introduces to analyze the influential aspects about the crime incidents, and examine it on New York City. The proposed structure relates the dynamic machine learning algorithms and geographical information system (GIS) to consider the contiguous reasons of crime data. Recursive feature elimination (RFE) is used to select the optimum characteristic data. Exploitation of gradient boost decision tree (GBDT), logistic regression (LR), support vector machine (SVM) and artificial neural network (ANN) are related to develop the optimum data model. Significant impact features were then reviewed by applying GBDT and GIS”. The experimental results illustrates that GBDT along with GIS model combination can identify the crime ranking with high performance and accuracy compared to existing method.”

Cancer is the second leading cause of disease-related death worldwide, and machine learning-based identification of novel biomarkers is crucial for improving early detection and treatment of various cancers. A key challenge in applying machine learning to high-dimensional data is deriving important features in an interpretable manner to provide meaningful insights into the underlying biological mechanismsWe developed a class-based directional feature importance (CLIFI) metric for decision tree methods and demonstrated its use for The Cancer Genome Atlas proteomics data. The CLIFI metric was incorporated into four algorithms, Random Forest (RF), LAtent VAriable Stochastic Ensemble of Trees (LAVASET), and Gradient Boosted Decision Trees (GBDTs), and a new extension incorporating the LAVA step into GBDTs (LAVABOOST). Both LAVA methods incorporate topological information from protein interactions into the decision function.The different models' performance in classifying 28 cancers resulted in F1-scores of 92.6% (RF), 92.0% (LAVASET), 89.3% (LAVABOOST) and 85.7% (GBDT), with no method outperforming all others for individual cancer type prediction. The CLIFI metric enables visualisation of the model's decision-making functions. The resulting CLIFI value distributions indicated heterogeneity in the expression of several proteins (MYH11, ERα, BCL2) across different cancer types (including brain glioma, breast, kidney, thyroid and prostate cancer) aligning with the original raw expression data.In conclusion, we have developed an integrated, directional feature importance metric for multi-class decision tree-based classification models that facilitates interpretable feature importance assessment. The CLIFI metric can be combined with incorporating topological information into the decision functions of models to introduce inductive bias, enhancing interpretability.

BackgroundSurgical site infection (SSI) is one of the most common types of health care–associated infections. It increases mortality, prolongs hospital length of stay, and raises health care costs. Many institutions developed risk assessment models for SSI to help surgeons preoperatively identify high-risk patients and guide clinical intervention. However, most of these models had low accuracies.ObjectiveWe aimed to provide a solution in the form of an Artificial intelligence–based Multimodal Risk Assessment Model for Surgical site infection (AMRAMS) for inpatients undergoing operations, using routinely collected clinical data. We internally and externally validated the discriminations of the models, which combined various machine learning and natural language processing techniques, and compared them with the National Nosocomial Infections Surveillance (NNIS) risk index.MethodsWe retrieved inpatient records between January 1, 2014, and June 30, 2019, from the electronic medical record (EMR) system of Rui Jin Hospital, Luwan Branch, Shanghai, China. We used data from before July 1, 2018, as the development set for internal validation and the remaining data as the test set for external validation. We included patient demographics, preoperative lab results, and free-text preoperative notes as our features. We used word-embedding techniques to encode text information, and we trained the LASSO (least absolute shrinkage and selection operator) model, random forest model, gradient boosting decision tree (GBDT) model, convolutional neural network (CNN) model, and self-attention network model using the combined data. Surgeons manually scored the NNIS risk index values.ResultsFor internal bootstrapping validation, CNN yielded the highest mean area under the receiver operating characteristic curve (AUROC) of 0.889 (95% CI 0.886-0.892), and the paired-sample t test revealed statistically significant advantages as compared with other models (P<.001). The self-attention network yielded the second-highest mean AUROC of 0.882 (95% CI 0.878-0.886), but the AUROC was only numerically higher than the AUROC of the third-best model, GBDT with text embeddings (mean AUROC 0.881, 95% CI 0.878-0.884, P=.47). The AUROCs of LASSO, random forest, and GBDT models using text embeddings were statistically higher than the AUROCs of models not using text embeddings (P<.001). For external validation, the self-attention network yielded the highest AUROC of 0.879. CNN was the second-best model (AUROC 0.878), and GBDT with text embeddings was the third-best model (AUROC 0.872). The NNIS risk index scored by surgeons had an AUROC of 0.651.ConclusionsOur AMRAMS based on EMR data and deep learning methods—CNN and self-attention network—had significant advantages in terms of accuracy compared with other conventional machine learning methods and the NNIS risk index. Moreover, the semantic embeddings of preoperative notes improved the model performance further. Our models could replace the NNIS risk index to provide personalized guidance for the preoperative intervention of SSIs. Through this case, we offered an easy-to-implement solution for building multimodal RAMs for other similar scenarios.

Glass forming ability prediction of bulk metallic glasses based on fused strategy