Feature selection with a lexicographic social ranking method

In this paper, we focus on limitations in the use of the Shapley value within the field of eXplainable AI (XAI) through the lens of the axiomatic analysis and its implications in the realm of machine learning. As an alternative to the Shapley value, we analyse the properties of the lex-cel, a social ranking solution introduced inthe recent literature at the intersection between coalitional games and social choice theory, showing that axioms characterizing thelex-cel, under certain circumstances, are more suitable for ranking features in machine learning models, compared to those satisfied bythe Shapley value. Via experiments conducted on public datasets, we also show that the lex-cel outperforms a commonly employed feature selection algorithm based on the Shapley value, in particular with respect to the capacity of selecting less redundant features

Machine learning models with distinct Shapley value explanations decouple feature attribution and interpretation for chemical compound predictions

Objective: To investigate the feasibility of a machine learning (ML) model based on radiomic features to identify active giant cell arteritis (GCA) in the aorta and differentiate it from atherosclerosis in follow-up [18F]FDG-PET/CT images for therapy monitoring. Methods: To train the ML model, 64 [18F]FDG-PET scans of 34 patients with proven GCA and 34 control subjects with type 2 diabetes mellitus were retrospectively included. The aorta was delineated into the ascending, arch, descending, and abdominal aorta. From each segment, 95 features were extracted. All segments were randomly split into a training/validation (n = 192; 80%) and test set (n = 46; 20%). In total, 441 ML models were trained, using combinations of seven feature selection methods, seven classifiers, and nine different numbers of features. The performance was assessed by area under the curve (AUC). The best performing ML model was compared to the clinical report of nuclear medicine physicians in 19 follow-up scans (7 active GCA, 12 inactive GCA). For explainability, an occlusion map was created to illustrate the important regions of the aorta for the decision of the ML model. Results: The ten-feature model with ANOVA as the feature selector and random forest classifier demonstrated the highest performance (AUC = 0.92 ± 0.01). Compared with the clinical report, this model showed a higher PPV (0.83 vs. 0.80), NPV (0.85 vs. 0.79), and accuracy (0.84 vs. 0.79) in the detection of active GCA in follow-up scans. Conclusions: The current radiomics ML model was able to identify active GCA and differentiate GCA from atherosclerosis in follow-up [18F]FDG-PET/CT scans. This demonstrates the potential of the ML model as a monitoring tool in challenging [18F]FDG-PET scans of GCA patients.

Accurate prediction of energy consumption in district heating systems plays an important role in supporting effective and clean energy production and distribution in dense urban areas. Predictive models are needed for flexible and cost-effective operation of energy production and usage, e.g., using peak shaving or load shifting to compensate for heat losses in the pipeline. This helps to avoid exceedance of power plant capacity. The purpose of this study is to automate the process of building machine learning (ML) models to solve a short-term power demand prediction problem. The dataset contains a district heating network’s measured hourly power consumption and ambient temperature for 415 days. In this paper, we propose a hybrid evolutionary-based algorithm, named GA-SHADE, for the simultaneous optimization of ML models and feature selection. The GA-SHADE algorithm is a hybrid algorithm consisting of a Genetic Algorithm (GA) and success-history-based parameter adaptation for differential evolution (SHADE). The results of the numerical experiments show that the proposed GA-SHADE algorithm allows the identification of simplified ML models with good prediction performance in terms of the optimized feature subset and model hyperparameters. The main contributions of the study are (1) using the proposed GA-SHADE, ML models with varying numbers of features and performance are obtained. (2) The proposed GA-SHADE algorithm self-adapts during operation and has only one control parameter. There is no fine-tuning required before execution. (3) Due to the evolutionary nature of the algorithm, it is not sensitive to the number of features and hyperparameters to be optimized in ML models. In conclusion, this study confirms that each optimized ML model uses a unique set and number of features. Out of the six ML models considered, SVR and NN are better candidates and have demonstrated the best performance across several metrics. All numerical experiments were compared against the measurements and proven by the standard statistical tests.

Background: Osteoporosis is a systemic skeletal disorder marked by reduced bone density and microarchitectural deterioration, leading to increased fracture risk. While the dual-energy X-ray absorptiometry (DEXA) scan is the World Health Organization (WHO)-recommended diagnostic standard, its limitations necessitate alternative methods. Emerging magnetic resonance imaging (MRI) techniques, radiomics, and machine learning promise to enhance osteoporosis diagnosis through detailed analysis of lumbar MRI apparent diffusion coefficient (ADC) maps, potentially revolutionizing early detection and treatment strategies. Objectives: In this study, we are going to evaluate the performance of machine learning (ML) models using radiomics features of lumbar MRI ADC map for osteoporosis detection, and to identify significant features and their diagnostic thresholds. Specific performance metrics such as accuracy, sensitivity, specificity, and Area Under the receiver operating characteristic (ROC) Curve (AUC) were assessed. Patients and Methods: This retrospective study employed a cross-sectional design, with a total of 140 cases, including 21 with osteoporosis. The study's inclusion criteria consisted of concurrent lumbar MRI and DEXA within a year, while exclusion criteria included infectious or neoplastic lumbar lesions, fractures, instrumentation, significant osteodegenerative changes, cases where the first four lumbar vertebrae were not included in the imaging field, and absence of diffusion-weighted imaging. Manual segmentation of lumbar vertebrae from ADC maps was performed to create a comprehensive dataset, comprising 5,580 radiomics features per case. Subsequently, the top five features selected by fast correlation-based filter (FCBF) were used to test the performance of seven Machine Learning algorithms (k-Nearest neighbors, decision tree, random forest, logistic regression, support vector machine, naive bayes, and neural network). Statistical tests and ROC curve analysis were conducted to determine the significance and thresholds of these features. Results: The study included 140 cases, with 132 females (94.3%) and 8 males (5.7%), and a mean age of 65.32 ± 8.50 years. The mean BMI was 31.43 ± 5.53 kg/m² for females and 26 ± 3.59 kg/m² for males. In terms of demographic differences, no significant age difference was found between the osteoporotic and non-osteoporotic groups (P = 0.889). However, the osteoporotic group had significantly lower mean body weight (64.90 ± 10.13 kg vs. 74.68 ± 13.94 kg, P = 0.003) and BMI (27.40 ± 4.38 kg/m² vs. 31.77 ± 5.52 kg/m², P = 0.001) compared to the non-osteoporotic group. The median interval between DEXA and lumbar MRI was 1 month (range 0.1 - 11.87 months). The Neural Network model demonstrated the highest performance with an AUC of 0.616 and a classification accuracy of 0.764 using all features. The Naive Bayes model, using the top five features selected by FCBF, showed the highest performance with an AUC of 0.913, accuracy of 0.907, sensitivity of 0.667, and specificity of 0.95. All ML models’ performance were elevated by feature selection. Independent t-tests and Mann-Whitney U tests identified 521 and 670 significant features, respectively (P < 0.05). ROC analysis revealed 58 features with AUC values above 0.70. Conclusion: This study's findings suggest that ML models, particularly the Naive Bayes algorithm, can effectively use lumbar ADC map radiomics to diagnose osteoporosis. These findings could enhance early detection and treatment strategies, potentially improving patient outcomes and reducing the burden of osteoporotic fractures. This study also established threshold values for significant features.

The intrusion detection system has been widely studied and deployed by researchers for providing better security to computer networks. The increasing volume of attacks, com-bined with the rapid improvement of machine learning (ML) has made the collaboration of intrusion detection techniques with machine learning and deep learnings are a popular subject and a feasible approach for cyber threat protection. Machine learning usually involves the training process using huge sample data. Since the huge input data may cause a negative effect on the training and detection performance of the machine learning model, feature selection becomes a crucial technique to rule out the irrelevant and redundant features from the dataset. This study applied a feature selection approach for intrusion detection that incorporated state-of-the-art feature selection algorithms with attack characteristic feature to produce an optimized set of features for the machine learning algorithms, which was then used to train the machine learning model. CSECIC- IDS2018 dataset, the most recent benchmark dataset with a wide attack diversity and features have been used to create the efficient feature subset. The result of the experiment was produced using machine learning models with a decision tree classifier and analyzed with respect to the accuracy, precision, recall, and f1 score.

To enhance the performance of machine learning (ML) models for the post-embolization recanalization of cerebral aneurysms, we evaluated the impact of hemodynamic feature derivation and selection method on six ML algorithms. We utilized computational fluid dynamics (CFD) to simulate hemodynamics in 66 cerebral aneurysms from 65 patients, including 57 stable and nine recanalized aneurysms. We derived a total of 107 features for each aneurysm, encompassing four clinical features, 12 morphological features, and 91 hemodynamic features. To investigate the influence of feature derivation and selection methods on the ML models, we employed two derivation methods, simplified and fully derived, in combination with four selection methods: all features, statistically significant analysis, stepwise multivariate logistic regression analysis (stepwise-LR), and recursive feature elimination (RFE). Model performance was assessed using the area under the receiver operating characteristic curve (AUROC) and precision-recall curve (AUPRC) on both the training and testing datasets. The AUROC values on the testing dataset exhibited a wide-ranging spectrum, spanning from 0.373 to 0.863. Fully derived features and the RFE selection method demonstrated superior performance in intra-model comparisons. The multi-layer perceptron (MLP) model, trained with RFE-selected fully derived features, achieved the best performance on the testing dataset, with an AUROC value of 0.863 (95% CI: 0.684- 1.000). Our study demonstrated the importance of feature derivation and selection in determining the performance of ML models. This enabled the development of accurate decision-making models without the need to invade the patient.

Propofol is a widely used sedative-hypnotic agent for critically-ill patients requiring invasive mechanical ventilation (IMV). Despite its clinical benefits, propofol is associated with increased risks of hypertriglyceridemia. Early identification of patients at risk for propofol-associated hypertriglyceridemia is crucial for optimizing sedation strategies and preventing adverse outcomes. Machine learning (ML) models offer a promising approach for predicting individualized patient risks of propofol-associated hypertriglyceridemia. We propose the development of a ML model aimed at predicting the risk of propofol-associated hypertriglyceridemia in ICU patients receiving IMV. The study will utilize retrospective data from four Mayo Clinic sites. Nested cross-validation (CV) will be employed, with a 10-fold inner CV loop for model tuning and selection as well as an outer loop using leave-one-site-out CV for external validation. Feature selection will be conducted using Boruta and LASSO-penalized logistic regression. Data preprocessing steps include missing data imputation, feature scaling, and dimensionality reduction techniques. Six ML algorithms will be tuned and evaluated. Bayesian optimization will be used for hyperparameter selection. Global model explainability will be assessed using permutation importance, and local model explainability will be assessed using SHapley Additive exPlanations (SHAP). The proposed ML model aims to provide a reliable and interpretable tool for clinicians to predict the risk of propofol-associated hypertriglyceridemia in ICU patients. The final model will be deployed in a web-based clinical risk calculator. The model development process and performance measures obtained during nested cross-validation will be described in a study publication to be disseminated in a peer-reviewed journal. The proposed study has received ethics approval from the Mayo Clinic Institutional Review Board (IRB #23-007416). Robust external validation using a nested cross-validation (CV) framework will help assess the generalizability of models produced from the modeling pipeline across different hospital settings.A diverse set of machine learning (ML) algorithms and advanced hyperparameter tuning techniques will be employed to identify the most optimal model configuration.Integration of feature explainability will enhance the clinical applicability of the ML models by providing transparency in predictions, which can improve clinician trust and encourage adoption.Reliance on retrospective data may introduce biases due to inconsistent or erroneous data collection, and the computational intensity of the validation approach may limit replication and future model expansion in resource-constrained settings.

This report details the results of a three-fold investigation of sensitivity analysis (SA) for machine learning (ML) explainability (MLE): (1) the mathematical assessment of the fidelity of an explanation with respect to a learned ML model, (2) quantifying the trustworthiness of a prediction, and (3) the impact of MLE on the efficiency of end-users through multiple users studies. We focused on the cybersecurity domain as the data is inherently non-intuitive. As ML is being using in an increasing number of domains, including domains where being wrong can elicit high consequences, MLE has been proposed as a means of generating trust in a learned ML models by end users. However, little analysis has been performed to determine if the explanations accurately represent the target model and they themselves should be trusted beyond subjective inspection. Current state-of-the-art MLE techniques only provide a list of important features based on heuristic measures and/or make certain assumptions about the data and the model which are not representative of the real-world data and models. Further, most are designed without considering the usefulness by an end-user in a broader context. To address these issues, we present a notion of explanation fidelity based on Shapley values from cooperative game theory. We find that all of the investigated MLE explainability methods produce explanations that are incongruent with the ML model that is being explained. This is because they make critical assumptions about feature independence and linear feature interactions for computational reasons. We also find that in deployed, explanations are rarely used due to a variety of reason including that there are several other tools which are trusted more than the explanations and there is little incentive to use the explanations. In the cases when the explanations are used, we found that there is the danger that explanations persuade the end users to wrongly accept false positives and false negatives. However, ML model developers and maintainers find the explanations more useful to help ensure that the ML model does not have obvious biases. In light of these findings, we suggest a number of future directions including developing MLE methods that directly model non-linear model interactions and including design principles that take into account the usefulness of explanations to the end user. We also augment explanations with a set of trustworthiness measures that measure geometric aspects of the data to determine if the model output should be trusted.

Identification of high-risk genes and classification of acute myocardial infarction patients utilizing deep learning in a restricted cohort.

A machine-learning-based model for predicting the effective stiffness of precast concrete columns

Improving Machine Learning Models for Malware Detection Using Embedded Feature Selection Method

The increasing prevalence of malware presents a critical challenge to cybersecurity, emphasizing the need for robust detection methods. This study uses a binary tabular classification dataset to evaluate the impact of feature selection, feature scaling, and machine learning (ML) models on malware detection. The methodology involves experimenting with three feature scaling techniques (no scaling, normalization, and min-max scaling), three feature selection methods (no selection, Linear Discriminant Analysis (LDA), and Principal Component Analysis (PCA)), and twelve ML models, including traditional algorithms and ensemble methods. A publicly available dataset with 11,598 samples and 139 features is utilized, and model performance is assessed using metrics such as accuracy, precision, recall, F1-score, and AUC-ROC. Results reveal that the Light Gradient Boosting Machine (LGBM) achieves the highest accuracy of 97.16% when PCA and either min-max scaling or normalization are applied. Additionally, ensemble models consistently outperform traditional ML models, demonstrating their effectiveness in enhancing malware detection. These findings offer valuable insights into optimizing preprocessing and model selection strategies for developing reliable and efficient malware detection systems.

Pulsar detection has become an active research topic in radio astronomy recently. One of the essential procedures for pulsar detection is pulsar candidate sifting (PCS), a procedure for identifying potential pulsar signals in a survey. However, pulsar candidates are always class-imbalanced, as most candidates are non-pulsars such as RFI and only a tiny part of them are from real pulsars. Class imbalance can greatly affect the performance of machine learning (ML) models, resulting in a heavy cost as some real pulsars are misjudged. To deal with the problem, techniques of choosing relevant features to discriminate pulsars from non-pulsars are focused on, which is known as feature selection. Feature selection is a process of selecting a subset of the most relevant features from a feature pool. The distinguishing features between pulsars and non-pulsars can significantly improve the performance of the classifier even if the data are highly imbalanced. In this work, an algorithm for feature selection called the K-fold Relief-Greedy (KFRG) algorithm is designed. KFRG is a two-stage algorithm. In the first stage, it filters out some irrelevant features according to their K-fold Relief scores, while in the second stage, it removes the redundant features and selects the most relevant features by a forward greedy search strategy. Experiments on the data set of the High Time Resolution Universe survey verified that ML models based on KFRG are capable of PCS, correctly separating pulsars from non-pulsars even if the candidates are highly class-imbalanced.

Improving Clinical Decisions in IR: Interpretable Machine Learning Models for Predicting Ascites Improvement after Transjugular Intrahepatic Portosystemic Shunt Procedures