Topological embedding and directional feature importance in ensemble classifiers for multi-class classification

Machine learning-based landslide susceptibility assessment with optimized ratio of landslide to non-landslide samples

BackgroundTongue diagnosis is an important research field of TCM diagnostic technology modernization. The quality of tongue images is the basis for constructing a standard dataset in the field of tongue diagnosis. To establish a standard tongue image database in the TCM industry, we need to evaluate the quality of a massive number of tongue images and add qualified images to the database. Therefore, an automatic, efficient and accurate quality control model is of significance to the development of intelligent tongue diagnosis technology for TCM.MethodsMachine learning methods, including Support Vector Machine (SVM), Random Forest (RF), Gradient Boosting Decision Tree (GBDT), Adaptive Boosting Algorithm (Adaboost), Naïve Bayes, Decision Tree (DT), Residual Neural Network (ResNet), Convolution Neural Network developed by Visual Geometry Group at University of Oxford (VGG), and Densely Connected Convolutional Networks (DenseNet), were utilized to identify good-quality and poor-quality tongue images. Their performances were made comparisons by using metrics such as accuracy, precision, recall, and F1-Score.ResultsThe experimental results showed that the accuracy of the three deep learning models was more than 96%, and the accuracy of ResNet-152 and DenseNet-169 was more than 98%. The model ResNet-152 obtained accuracy of 99.04%, precision of 99.05%, recall of 99.04%, and F1-score of 99.05%. The performances were better than performances of other eight models. The eight models are VGG-16, DenseNet-169, SVM, RF, GBDT, Adaboost, Naïve Bayes, and DT. ResNet-152 was selected as quality-screening model for tongue IQA.ConclusionsOur research findings demonstrate various CNN models in the decision-making process for the selection of tongue image quality assessment and indicate that applying deep learning methods, specifically deep CNNs, to evaluate poor-quality tongue images is feasible.

Glass forming ability prediction of bulk metallic glasses based on fused strategy

Radiomics and machine learning have been extensively utilized in the realm of urinary stones, particularly in forecasting stone treatment outcomes. The objective of this study was to integrate clinical variables and radiomic features to develop a machine learning model for predicting the stone-free rate (SFR) following percutaneous nephrolithotomy (PCNL).A total of 212 eligible patients who underwent PCNL surgery at the Second Affiliated Hospital of Nanchang University were included in a retrospective analysis. Preoperative clinical variables and non-contrast-enhanced CT images of all patients were collected, and radiomic features were extracted after delineating the stone ROI. Univariate analysis was conducted to identify clinical variables strongly correlated with the stone-free rate after PCNL, and the least absolute shrinkage and selection operator algorithm (lasso regression) was utilized to screen radiomic features. Four supervised machine learning algorithms, including Logistic Regression, Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Gradient Boosting Decision Tree (GBDT), were employed. The clinical variables with strong correlation and screened radiomic features were integrated into the four machine learning algorithms to construct a prediction model, and the receiver operating curve was plotted. The area under the receiver operating curve (AUC), the accuracy rate, the specificity, etc., were used to evaluate the predictive performance of the four models.After analyzing postoperative statistics, the stone-free rate following the procedure was found to be 70.3% (n = 149). Among the various clinical variables examined, factors, such as stone number, stone diameter, stone CT value, stone location, and history of stone surgery, were identified as statistically significant in relation to the stone-free rate after PCNL. A total of 121 radiomic features were extracted, and through lasso regression, 7 features most closely associated with the stone-free rate post-PCNL were identified. The predictive accuracy of different models (Logistic Regression, RF, XGBoost, and GBDT) for determining the stone-free rate after PCNL was evaluated, yielding accuracies of 78.1%, 76.6%, 75.0%, and 73.4%, respectively. The corresponding area under the curve AUC (95%CI) were 0.85 (0.83-0.89), 0.81 (0.76-0.85), 0.82 (0.78-0.85), and 0.77 (0.73-0.81), positioning these models among the top performers in logistic regression prediction. In terms of predictive importance scores, the key factors identified by the logistic regression model were number of stone, zone percentage, stone diameter, and surface area. Similarly, the RF model highlighted number of stone, stone CT value, stone diameter, and surface area as the top predictors.Among the four machine learning models, the logistic regression model demonstrated the highest accuracy and discrimination ability in predicting the stone-free rate following PCNL. In comparison to XGBoost and GBDT, RF also exhibited superior accuracy and a certain level of discrimination ability. However, based on the performance of all four models, logistic regression is more likely to aid in clinical decision-making by assisting clinicians in diagnosing PCNL in patients. This enables us to effectively predict the presence of residual stones post-surgery and ultimately select patients who are suitable candidates for PCNL.

This paper carries on research surrounding the influences produced by dimensionality reduction on machine learning classification effect. Firstly, paper constructs the analysis architecture of data dimension reduction classification, combines the two different unsupervised dimension reduction methods, locally linear embedding (LLE) and principal component analysis (PCA) with the five machine learning classification methods: Gradient Boosting Decision Tree (GBDT), Random Forest, Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Logistic Regression. And then uses the handwritten digital identification dataset to analyze the classification performance of these five classification methods on different dimension datasets by different dimensionality reduction methods. The analysis shows that using the appropriate dimensionality reduction method for dimensionality reduction classification can effectively improve the classification accuracy; the dimensionality reduction classification effect of non-linear dimensionality reduction method is generally better than the linear dimensionality reduction method; different machine learning classification algorithms have significant differences in the sensitivity of dimensions.

Alzheimer's disease (AD) is one of the most common neurodegenerative diseases and its onset is significantly associated with genetic factors. Being the capabilities of high specificity and accuracy, genetic testing has been considered as an important technique for AD diagnosis. In this paper, we presented an improved deep learning (DL) algorithm, namely differential genes screening TabNet (DGS-TabNet) for AD binary and multi-class classifications. For performance evaluation, our proposed approach was compared with three novel DLs of multi-layer perceptron (MLP), neural oblivious decision ensembles (NODE), TabNet as well as five classical machine learnings (MLs) including decision tree (DT), random forests (RF), gradient boosting decision tree (GBDT), light gradient boosting machine (LGBM) and support vector machine (SVM) on the public data set of gene expression omnibus (GEO). Moreover, the biological interpretability of global important genetic features implemented for AD classification was revealed by the Kyoto encyclopedia of genes and genomes (KEGG) and gene ontology (GO). The results demonstrated that our proposed DGS-TabNet achieved the best performance with an accuracy of 93.80% for binary classification, and with an accuracy of 88.27% for multi-class classification. Meanwhile, the gene pathway analyses demonstrated that there existed two most important global genetic features of AVIL and NDUFS4 and those obtained 22 feature genes were partially correlated with AD pathogenesis. It was concluded that the proposed DGS-TabNet could be used to detect AD-susceptible genes and the biological interpretability of susceptible genes also revealed the potential possibility of being AD biomarkers.

Forest carbon stock is an important indicator reflecting the structure of forest ecosystems and forest quality, and an important parameter for evaluating the carbon sequestration capacity and carbon balance of forests. It is of great significance to study forest carbon stock in the context of current global climate change. To explore the application ability of multi-loaded, high-resolution satellite data in the estimation of subtropical forest carbon stock, this paper takes Huangfu Mountain National Forest Park in Chuzhou City as the study area, extracts remote sensing features such as spectral features, texture features, backscattering coefficient, and other remote sensing features based on multi-loaded, high-resolution satellite data, and carries out correlation analyses with the carbon stock of different species of trees and different age groups of forests. Regression models for different tree species were established for different data sources, and the optimal modeling factors for multi-species were determined. Then, three algorithms, namely, multiple stepwise regression, random forest, and gradient-enhanced decision tree, were used to estimate carbon stocks of multi-species, and the predictive ability of different estimation models on carbon stocks was analyzed using the coefficient of determination (R2) and the root mean square error (RMSE) as indexes. The following conclusions were drawn: for the feature factors, the texture features of the GF-2 image, the new red edge index of the GF-6 image, the radar intensity coefficient sigma, and radar brightness coefficient beta of the GF-3 image have the best correlation with the carbon stock; for the algorithms, the random forest and gradient-boosting decision tree have the better effect of fitting and predicting the carbon stock of multi-tree species, among which gradient-boosting decision tree has the best effect, with an R2 of 0.902 and an RMSE of 10.261 t/ha. In summary, the combination of GF-2, GF-3, and GF-6 satellite data and gradient-boosting decision tree obtains the most accurate estimation results when estimating forest carbon stocks of complex tree species; multi-load, high-resolution satellite data can be used in the inversion of subtropical forest parameters to estimate the carbon stocks of subtropical forests. The multi-loaded, high-resolution satellite data have great potential for application in the field of subtropical forest parameter inversion.

Sepsis is a leading cause of mortality in the intensive care unit. Early prediction of sepsis can reduce the overall mortality rate and cost of sepsis treatment. Some studies have predicted mortality and development of sepsis using machine learning models. However, there is a gap between the creation of different machine learning algorithms and their implementation in clinical practice.This study utilized data from the Medical Information Mart for Intensive Care III. We established and compared the gradient boosting decision tree (GBDT), logistic regression (LR), k-nearest neighbor (KNN), random forest (RF), and support vector machine (SVM).A total of 3937 sepsis patients were included, with 34.3% mortality in the Medical Information Mart for Intensive Care III group. In our comparison of 5 machine learning models (GBDT, LR, KNN, RF, and SVM), the GBDT model showed the best performance with the highest area under the receiver operating characteristic curve (0.992), recall (94.8%), accuracy (95.4%), and F1 score (0.933). The RF, SVM, and KNN models showed better performance (area under the receiver operating characteristic curve: 0.980, 0.898, and 0.877, respectively) than the LR (0.876).The GBDT model showed better performance than other machine learning models (LR, KNN, RF, and SVM) in predicting the mortality of patients with sepsis in the intensive care unit. This could be used to develop a clinical decision support system in the future.

Random forest is a type of ensemble-based machine learning method that has been applied to a variety of machine learning tasks in recent years. This paper proposes an evolutionary approach to generate an oblique random forest for regression problems. More specifically, our method induces an oblique random forest by transforming the original feature space to a new feature space through the evolutionary feature construction method. To speed up the searching process, the proposed method evaluates each set of features based on a decision tree rather than a random forest. In order to obtain a random forest, we archive top-performing features and corresponding trees during the search. In this way, both the features and the forest can be constructed simultaneously in a single run. The proposed evolutionary forest is applied to 117 benchmark problems with different characteristics and compared with some state-of-the-art regression methods, including several variants of the random forest and gradient boosted decision trees. The experimental results suggest that the proposed method outperforms the existing random forest and gradient boosted decision tree methods.

This study aims to employ machine learning algorithms to analyze the axial bearing capacity of rubberized alkali-activated concrete filled steel tubes. A dataset encompassing 327 synthesized instances and seven input features is adopted for training and testing six machine learning models, including Decision Tree, Random Forest, Extremely Randomized Trees, Adaptive Boosting, Gradient Boosting Decision Trees (GBDT), and eXtreme Gradient Boosting Trees (XGBoost). The SHapley Additive exPlanation algorithm is employed to elucidate the prediction process of machine learning models and to analyze the influence of each parameter on axial bearing capacity. Comparison of evaluating metrics shows that GBDT and XGBoost models achieve highest accuracy and generalization capabilities when their Coefficient of Determination values surpassing 0.98 and Mean Absolute Percent Error remaining below 3%. Moreover, the explanation analysis of machine learning models reveals that diameter/width of the cross section, rubber content, yielding strength and thickness of steel tubes are critical variables that affect the axial bearing capacity, while compressive strength of alkali-activated concrete, specimen height, and shape of cross section show negligible impact. Besides, GBDT model overemphasizes the effect of specimen height and might lead a conservative prediction for specimens with smaller heights. Finally, compressive strength of alkali-activated concrete and diameter/width, thickness, and yielding strength of steel tubes are positively correlated with axial bearing capacity, and the increase of rubber content in alkali-activated concrete leads to the decrease of capacity.

This study aimed to construct a high-performance prediction and diagnosis model for type 2 diabetic retinopathy (DR) and identify key correlates of DR. This study utilized a cross-sectional dataset of 3,000 patients from the People’s Liberation Army General Hospital in 2021. Logistic regression was used as the baseline model to compare the prediction performance of the machine learning model and the related factors. The recursive feature elimination cross-validation (RFECV) algorithm was used to select features. Four machine learning models, support vector machine (SVM), decision tree (DT), random forest (RF), and gradient boost decision tree (GBDT), were developed to predict DR. The models were optimized using grid search to determine hyperparameters, and the model with superior performance was selected. Shapley-additive explanations (SHAP) were used to analyze the important correlation factors of DR. Among the four machine learning models, the optimal model was GBDT, with predicted accuracy, precision, recall, F1-measure, and AUC values of 0.7883, 0.8299, 0.7539, 0.7901, and 0.8672, respectively. Six key correlates of DR were identified, including rapid micronutrient protein/creatinine measurement, 24-h micronutrient protein, fasting C-peptide, glycosylated hemoglobin, blood urea, and creatinine. The logistic model had 27 risk factors, with an AUC value of 0.8341. A superior prediction model was constructed that identified easily explainable key factors. The number of correlation factors was significantly lower compared to traditional statistical methods, leading to a more accurate prediction performance than the latter.

Accurate prediction of in-hospital mortality for patients with severe community-acquired pneumonia (SCAP) complicated by respiratory failure admitted to the intensive care unit (ICU) remains a critical challenge. This study aimed to develop and validate a machine learning (ML) model to predict this risk and compare its performance with conventional scoring systems. In this retrospective study, data from 164 patients with SCAP and respiratory failure admitted to the ICU between January 2017 and January 2024 were analyzed. Patients were randomly divided into a training set (n = 114) and a validation (test) set (n = 50). Forty-five clinical features collected at admission were used as candidate predictors. The Least Absolute Shrinkage and Selection Operator (LASSO) regression was employed for feature selection. Six ML models, including Gradient Boosting Decision Tree (GBDT), Random Forest (RF), eXtreme Gradient Boosting (XGBoost), Decision Tree(DT),Support Vector Machine (SVM), and Logistic Regression(LR), were constructed and evaluated.Use SHAP analysis to assess the contribution of each feature in a machine learning model. Construct a nomogram using the top six most influential features. The GBDT model demonstrated the best predictive performance, achieving an area under the receiver operating characteristic curve (AUC) of 0.83 (95% CI: 0.757-0.927) in the internal validation set, significantly outperforming the Acute Physiology and Chronic Health Evaluation II (APACHE-II, AUC = 0.70). Calibration curves demonstrated good agreement between predicted and observed mortality risks, particularly across the mid-probability range. Decision curve analysis indicated that the model provided a higher net benefit than "treat-all" and "treat-none" strategies across a broad range of threshold probabilities. SHapley Additive exPlanations (SHAP) analysis identified lactate, D-dimer, temperature, albumin, Prothrombin Time and Fraction of Inspired Oxygen as the six most influential predictors of in-hospital mortality. Based on these key predictors, we further developed a simplified nomogram to facilitate bedside risk estimation. The GBDT ML model, developed from routinely available clinical data, provides a highly accurate and clinically interpretable tool for predicting in-hospital mortality in SCAP patients with respiratory failure. It outperforms traditional severity scores and holds promise for assisting clinicians in risk stratification and early intervention.

Pulse wave velocity (PWV) can evaluate potential atherosclerosis (AS) and ultrafast pulse wave velocity (ufPWV) is a new technique to accurately assess PWV. However, few studies have examined the predictive value of ufPWV for AS risk. We aimed to establish a classification model for AS risk diagnosis based on ufPWV, so that AS can be diagnosed and prevented in advance. We collected imaging data, as well as clinical and laboratory data. A total of 613 patients with 20 attributes were admitted in this study. There were 392 patients with hyperlipidemia (AS risk group) and 221 healthy adults as the control group. In order to build AS risk prediction models, we considered decision tree, five different ensemble learning (EL) models [random forest (RF), adaptive boosting (AdaBoost), gradient boosting decision tree (GBDT), extreme gradient boosting (XGBoost) and light gradient boosting machine (LGBM)] and two different feature selection methods [statistical analysis and RF]. Accuracy and the area under the ROC curve (AUC) were used as the main criterion for model evaluation. In the prediction of AS risk with statistical analysis as the feature selection method, the performances of XGBoost (accuracy: 0.851; AUC: 0.884) and RF (accuracy: 0.844; AUC: 0.889) were better than other models. Besides, in the prediction of AS risk with RF as the feature selection method, the performances of LGBM (accuracy: 0.870; AUC: 0.903) and XGBoost (accuracy: 0.857; AUC: 0.903) were better than other models. In conclusions, EL models with RF as the feature selection method might provide accurate results in predicting AS risk. Besides, ufPWV, especially PWV of left common carotid artery at the end of systole, was an important feature in the AS risk prediction models.

The objective of this paper is to predict the future driving risk of crash-involved drivers in Kunshan, China. A systematic machine learning framework is proposed to deal with three critical technical issues: 1. defining driving risk; 2. developing risky driving factors; 3. developing a reliable and explicable machine learning model. High-risk (HR) and low-risk (LR) drivers were defined by five different scenarios. A number of features were extracted from seven-year crash/violation records. Drivers’ two-year prior crash/violation information was used to predict their driving risk in the subsequent two years. Using a one-year rolling time window, prediction models were developed for four consecutive time periods: 2013–2014, 2014–2015, 2015–2016, and 2016–2017. Four tree-based ensemble learning techniques were attempted, including random forest (RF), Adaboost with decision tree, gradient boosting decision tree (GBDT), and extreme gradient boosting decision tree (XGboost). A temporal transferability test and a follow-up study were applied to validate the trained models. The best scenario defining driving risk was multi-dimensional, encompassing crash recurrence, severity, and fault commitment. GBDT appeared to be the best model choice across all time periods, with an acceptable average precision (AP) of 0.68 on the most recent datasets (i.e., 2016–2017). Seven of nine top features were related to risky driving behaviors, which presented non-linear relationships with driving risk. Model transferability held within relatively short time intervals (1–2 years). Appropriate risk definition, complicated violation/crash features, and advanced machine learning techniques need to be considered for risk prediction task. The proposed machine learning approach is promising, so that safety interventions can be launched more effectively.

BackgroundSurgical site infection (SSI) is one of the most common types of health care–associated infections. It increases mortality, prolongs hospital length of stay, and raises health care costs. Many institutions developed risk assessment models for SSI to help surgeons preoperatively identify high-risk patients and guide clinical intervention. However, most of these models had low accuracies.ObjectiveWe aimed to provide a solution in the form of an Artificial intelligence–based Multimodal Risk Assessment Model for Surgical site infection (AMRAMS) for inpatients undergoing operations, using routinely collected clinical data. We internally and externally validated the discriminations of the models, which combined various machine learning and natural language processing techniques, and compared them with the National Nosocomial Infections Surveillance (NNIS) risk index.MethodsWe retrieved inpatient records between January 1, 2014, and June 30, 2019, from the electronic medical record (EMR) system of Rui Jin Hospital, Luwan Branch, Shanghai, China. We used data from before July 1, 2018, as the development set for internal validation and the remaining data as the test set for external validation. We included patient demographics, preoperative lab results, and free-text preoperative notes as our features. We used word-embedding techniques to encode text information, and we trained the LASSO (least absolute shrinkage and selection operator) model, random forest model, gradient boosting decision tree (GBDT) model, convolutional neural network (CNN) model, and self-attention network model using the combined data. Surgeons manually scored the NNIS risk index values.ResultsFor internal bootstrapping validation, CNN yielded the highest mean area under the receiver operating characteristic curve (AUROC) of 0.889 (95% CI 0.886-0.892), and the paired-sample t test revealed statistically significant advantages as compared with other models (P<.001). The self-attention network yielded the second-highest mean AUROC of 0.882 (95% CI 0.878-0.886), but the AUROC was only numerically higher than the AUROC of the third-best model, GBDT with text embeddings (mean AUROC 0.881, 95% CI 0.878-0.884, P=.47). The AUROCs of LASSO, random forest, and GBDT models using text embeddings were statistically higher than the AUROCs of models not using text embeddings (P<.001). For external validation, the self-attention network yielded the highest AUROC of 0.879. CNN was the second-best model (AUROC 0.878), and GBDT with text embeddings was the third-best model (AUROC 0.872). The NNIS risk index scored by surgeons had an AUROC of 0.651.ConclusionsOur AMRAMS based on EMR data and deep learning methods—CNN and self-attention network—had significant advantages in terms of accuracy compared with other conventional machine learning methods and the NNIS risk index. Moreover, the semantic embeddings of preoperative notes improved the model performance further. Our models could replace the NNIS risk index to provide personalized guidance for the preoperative intervention of SSIs. Through this case, we offered an easy-to-implement solution for building multimodal RAMs for other similar scenarios.