Machine learning & fairness: an integrated multicriteria approach for the evaluation of supervised classifiers

Background: Early identification of probable post-traumatic stress disorder (PTSD) can lead to early intervention and treatment.Aims: This study aimed to evaluate supervised machine learning (ML) classifiers for the identification of probable PTSD in those who are serving, or have recently served in the United Kingdom (UK) Armed Forces.Methods: Supervised ML classification techniques were applied to a military cohort of 13,690 serving and ex-serving UK Armed Forces personnel to identify probable PTSD based on self-reported service exposures and a range of validated self-report measures. Data were collected between 2004 and 2009.Results: The predictive performance of supervised ML classifiers to detect cases of probable PTSD were encouraging when compared to a validated measure, demonstrating a capability of supervised ML to detect the cases of probable PTSD. It was possible to identify which variables contributed to the performance, including alcohol misuse, gender and deployment status. A satisfactory sensitivity was obtained across a range of supervised ML classifiers, but sensitivity was low, indicating a potential for false negative diagnoses.Conclusions: Detection of probable PTSD based on self-reported measurement data is feasible, may greatly reduce the burden on public health and improve operational efficiencies by enabling early intervention, before manifestation of symptoms.

This paper proposes a robustness analysis based on Multiple Criteria Decision Aiding (MCDA). The ensuing model was used to assess the implementation of green chemistry principles in the synthesis of silver nanoparticles. Its recommendations were also compared to an earlier developed model for the same purpose to investigate concordance between the models and potential decision support synergies. A three-phase procedure was adopted to achieve the research objectives. Firstly, an ordinal ranking of the evaluation criteria used to characterize the implementation of green chemistry principles was identified through relative ranking analysis. Secondly, a structured selection process for an MCDA classification method was conducted, which ensued in the identification of Stochastic Multi-Criteria Acceptability Analysis (SMAA). Lastly, the agreement of the classifications by the two MCDA models and the resulting synergistic role of decision recommendations were studied. This comparison showed that the results of the two models agree between 76% and 93% of the simulation set-ups and it confirmed that different MCDA models provide a more inclusive and transparent set of recommendations. This integrative research confirmed the beneficial complementary use of MCDA methods to aid responsible development of nanosynthesis, by accounting for multiple objectives and helping communication of complex information in a comprehensive and traceable format, suitable for stakeholders and/or decision-makers with diverse backgrounds.

Regulating inflammatory microglia presents a promising strategy for treating neurodegenerative and autoimmune disorders, yet effective therapeutic agents delivery to these cells remains a challenge. This study investigates modified lipid nanoparticles (LNP) for mRNA delivery to hyperactivated microglia, particularly those with pro-inflammatory characteristics, utilizing supervised machine learning (ML) classifiers. We developed and screened a library of 216 LNP formulations with varying lipid compositions, N/P ratios, and hyaluronic acid (HA) modifications. The transfection efficiency of eGFP mRNA was assessed in the BV-2 murine microglia cell line under different immunological states, including resting and activated conditions (LPS-activated and IL4/IL13-activated). ML-guided morphometric analysis tracked the phenotypes of various microglia subtypes before and after transfection. Four supervised ML classifiers were investigated to predict transfection efficiency and phenotypic changes based on LNP design parameters. The Multi-Layer Perceptron (MLP) neural network emerged as the best-performing model, achieving weighted F1-scores ≥0.8. While it accurately predicted responses from LPS-activated and resting cells, it struggled with IL4/IL13-activated cells. The MLP model was validated by predicting the performance of four unseen LNP formulations delivering eGFP mRNA to LPS-activated BV2 cells. HA-LNP2 emerged as optimal formulation for delivering target IL10 mRNA, effectively suppressing inflammatory phenotypes, evidenced by shifts in cell morphology, increased IL10 expression, and reduced TNF-α levels. We also evaluated HA-LNP2 on LPS-activated human iPSC-derived microglia, confirming its efficacy in modulating inflammatory responses. This study highlights the potential of tailored LNP design and ML techniques to enhance mRNA therapy for neuroinflammatory disorders by leveraging carrier’s immunogenic properties to modulate microglial responses.

We can state undoubtedly that Bangla language is rich enough to work with and implement various Natural Language Processing (NLP) tasks. Though it needs proper attention, hardly NLP field has been explored with it. In this age of digitalization, large amount of Bangla news contents are generated in online platforms. Some of the contents are inappropriate for the children or aged people. With the motivation to filter out news contents easily, the aim of this work is to perform document level sentiment analysis (SA) on Bangla online news. In this respect, the dataset is created by collecting news from online Bangla newspaper archive. Further, the documents are manually annotated into positive and negative classes. Composite process technique of “Pipeline” class including Count Vectorizer, transformer (TF-IDF) and machine learning (ML) classifiers are employed to extract features and to train the dataset. Six supervised ML classifiers (i.e. Multinomial Naive Bayes (MNB), K-Nearest Neighbor (K-NN), Random Forest (RF), (C4.5) Decision Tree (DT), Logistic Regression (LR) and Linear Support Vector Machine (LSVM)) are used to analyze the best classifier for the proposed model. There has been very few works on SA of Bangla news. So, this work is a small attempt to contribute in this field. This model showed remarkable efficiency through better results in both the validation process of percentage split method and 10-fold cross validation. Among all six classifiers, RF has outperformed others by 99% accuracy. Even though LSVM has shown lowest accuracy of 80%, it is also considered as good output. However, this work has also exhibited surpassing outcome for recent and critical Bangla news indicating proper feature extraction to build up the model.

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

Malware Detection using conventional methods is incompetent to detect new and generic malware. For the investigation of a variety of malware, there were no ready-made machine learning datasets available for malware detection. So we generated our dataset by downloading a variety of malware files from the world's famous malware projects. By performing unstructured data collection from the downloaded APK files and feature mining process the final dataset was generated with 16300 records and a total of 215 features. There was a need to evaluate the performance of the generated dataset with supervised machine learning classifiers. So in this paper, we propose a malware detection approach using different supervised machine learning classifiers. Here supervised algorithms, Feature Reduction Techniques, and Ensembling techniques are used to evaluate the performance of the generated dataset. Machine Learning classifiers are evaluated on the evaluation parameters like AUC, FPR, TPR, Cohen Kappa Score, Precision, and Accuracy. We also represented the results of classifiers using Bar plots of Accuracy and plotting the ROC curve. From the results of machine learning classifiers, the performance of the CatBoost Classifier is highest with Accuracy 93.15% having a value of ROC curve as 0.91 and Cohen Kappa Score as 81.56%.

Artificial intelligence (AI) is a tool thought to revolutionize the field of reproductive medicine in the years to come. Specifically, machine learning (ML), which is a subset of AI methods used to detect patterns and make predictions based on large datasets, has been used in ART to predict implantation outcome, embryo transfer strategies or adverse outcomes for IVF treatments (for a review see, Wang et al., 2019). However, ML methods have not yet been widely adopted in medically assisted reproduction (MAR). This slow uptake could be due to a lack of in-depth interdisciplinary communication between ML experts and clinicians/embryologists, as well as some uncertainty on how ML models could be generalized for different populations. The February edition of the ESHRE Journal Club discussed a paper from Yland et al. (2022) where the authors compared different ML algorithms to predict the chance of pregnancy in couples actively trying to conceive (without undergoing MAR). By using data from the Pregnancy Study Online (PRESTO), a web-based prospective cohort study of 4133 couples in North America (Wise et al., 2015), the authors analysed 163 potential variables and used supervised ML classification algorithms to identify the strength of each variable in predicting pregnancy outcomes of couples across the fertility spectrum (infertile: <12 menstrual cycles; subfertile: within 6 menstrual cycles; and fecundability: the average probability of pregnancy per menstrual cycle). The authors were able to predict pregnancy with a discrimination as high as 71.2% and identify the variables that most consistently predicted conception. These variables included lifestyle and reproductive characteristics such as age, BMI, history of infertility, daily use of vitamins or folic acid, intercourse timing, diet quality and reduced stress. ESHRE Journal Club discussion focused on promoting interdisciplinarity among ML and reproductive medicine specialists. It hosted 50 participants on Twitter together with experts Michelle Perugini, Vajira Thambawita and authors Jennifer Yland, Yannis Paschalidis and Lauren Wise. The discussion resulted in 902 tweets and around 1 million impressions over a 24-h period.

Freshwater ecosystems play a crucial role in maintaining ecological balance and supporting biodiversity. Assessing water quality and identifying pollution factors affected by seasonal changes is of utmost importance. The traditional approach for identifying pollution sources and seasonal water quality assessment is often labor-intensive, time-consuming, and complex. In response, our study introduces a hybrid receptor, multivariate statistical and machine learning (ML) based framework as an alternative. Positive matrix factorization (PMF) and receptor model were used to identify key pollutant sources, while Principal Component Analysis (PCA) was used to corroborate pollutant sources. Additionally, various supervised ML classification algorithms are used to predict and identify pollution sources on full physicochemical datasets and reduced in situ parameter scenarios. ML model stability and generalization capacity were evaluated using learning curves, cross-validation method. The findings indicate significant seasonal variability in water quality index, ranging from 38.5 (poor) to 78.6 (good). PMF and PCA revealed three dominant pollution factors, such as biogeochemical processes, sediment resuspension/nutrient influx, and anthropogenic pollution, affecting the seasonal water quality by 15-30%, 25-45%, and 40-70%, respectively. Model performance indicators using confusion matrices, ROC-AUC, and other evaluation metrics, for full dataset identified XGBoost as the best model, achieving the highest accuracy of 96.3%, followed by CatBoosting (93.6%), Random Forest (93.6%), Support Vector Classifier (91.8%), K-Nearest Neighbors (87.2%) and Decision Tree (84.3%). However, with the in situ data the ML models show stable and consistent classification accuracy in the range of approximately 70-81%. Overall, the modelling framework provides an effective approach for capturing seasonal variability and pollution source attribution, contributing to improved ecological understanding and predictive management of freshwater system.

To develop and internally validate a machine learning (ML) model that identifies older outpatients with MCI using routine electronic health record (EHR) data. We conducted a retrospective cross-sectional study of community outpatients aged ≥60 years in Zhejiang, China. Structured EHR predictors included demographics, comorbidities/medications, lifestyle, and visit patterns. The outcome was adjudicated MCI based on cognitive screening (MoCA plus supplemental tests). Supervised ML classifiers were compared using 10-fold cross-validation and an independent held-out test set; class imbalance was addressed with SMOTE. Performance was assessed by the area under the ROC curve (AUC) and by sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), accuracy, and F1 score. The test set included ~640 patients (≈20% MCI). Gradient Boosting performed best: cross-validation mean AUC 0.855 (SD 0.031) and accuracy 0.862 (SD 0.013); test AUC 0.850, accuracy 0.833, and F1 0.402. At the default threshold, sensitivity was 0.286 and specificity 0.967 (PPV 0.679; NPV 0.847). Prioritizing sensitivity (~0.82) lowered specificity (~0.64). At a high-sensitivity threshold of 0.159, the model achieved a sensitivity of 0.802 with a specificity of 0.751 (PPV 0.441; NPV 0.939). Important predictors included older age, female sex, lower education, smaller family size, and higher depression scores. An ML model using routine outpatient EHR can discriminate MCI in older adults (AUC ≈ 0.85), supporting potential for automated, low-cost screening in primary care. Using the predicted probabilities generated in this analysis, we assessed calibration and conducted a decision-curve analysis. While the model shows good discrimination and calibration, external validation is still required to confirm clinical utility and refine operating thresholds.

Chapter 6 - Cardiovascular disease prediction using machine learning models and ensemble technique

Wireless Sensor Networks (WSNs) are the key underlying technology of the Internet of Things (IoT); however, these networks are energy constrained. Security has become a major challenge with the significant increase in deployed sensors, necessitating effective detection and mitigation approaches. Machine learning (ML) is one of the most effective methods for building cyber-attack detection systems. This paper presents a lightweight ensemble-based ML approach, Weighted Score Selector (WSS), for detecting cyber-attacks in WSNs. The proposed approach is implemented using a blend of supervised ML classifiers, in which the most effective classifier is promoted dynamically for the detection process to gain higher detection performance quickly. We compared the performance of the proposed approach to three classical ensemble techniques: Boosting-based, Bagging-based, and Stacking-based. The performance comparison was conducted in terms of accuracy, probability of false alarm, probability of detection, probability of misdetection, model size, processing time, and average prediction time per sample. We applied two independent feature selection techniques. We utilized the simulation-based labeled dataset, WSN-DS, that comprises samples of four internal network-layer Denial of Service attack types: Grayhole, Blackhole, Flooding, and TDMA scheduling, in addition to normal traffic. The simulation revealed promising results for our proposed approach.

Introduction: The 30–50% non-response rate to cardiac resynchronization therapy (CRT) calls for improved patient selection and optimized pacing lead placement. The study aimed to develop a novel technique using patient-specific cardiac models and machine learning (ML) to predict an optimal left ventricular (LV) pacing site (ML-PS) that maximizes the likelihood of LV ejection fraction (LVEF) improvement in a given CRT candidate. To validate the approach, we evaluated whether the distance DPS between the clinical LV pacing site (ref-PS) and ML-PS is associated with improved response rate and magnitude.Materials and methods: We reviewed retrospective data for 57 CRT recipients. A positive response was defined as a more than 10% LVEF improvement. Personalized models of ventricular activation and ECG were created from MRI and CT images. The characteristics of ventricular activation during intrinsic rhythm and biventricular (BiV) pacing with ref-PS were derived from the models and used in combination with clinical data to train supervised ML classifiers. The best logistic regression model classified CRT responders with a high accuracy of 0.77 (ROC AUC = 0.84). The LR classifier, model simulations and Bayesian optimization with Gaussian process regression were combined to identify an optimal ML-PS that maximizes the ML-score of CRT response over the LV surface in each patient.Results: The optimal ML-PS improved the ML-score by 17 ± 14% over the ref-PS. Twenty percent of the non-responders were reclassified as positive at ML-PS. Selection of positive patients with a max ML-score >0.5 demonstrated an improved clinical response rate. The distance DPS was shorter in the responders. The max ML-score and DPS were found to be strong predictors of CRT response (ROC AUC = 0.85). In the group with max ML-score > 0.5 and DPS< 30 mm, the response rate was 83% compared to 14% in the rest of the cohort. LVEF improvement in this group was higher than in the other patients (16 ± 8% vs. 7 ± 8%).Conclusion: A new technique combining clinical data, personalized heart modelling and supervised ML demonstrates the potential for use in clinical practice to assist in optimizing patient selection and predicting optimal LV pacing lead position in HF candidates for CRT.

Magnetic resonance imaging (MRI) is playing an important role in the classification of breast tumors. MRI can be used to obtain multiparametric (mp) information, such as structural, hemodynamic, and physiological information. Quantitative analysis of mp-MRI data has shown potential in improving the accuracy of breast tumor classification. In general, a large set of quantitative and texture features can be generated depending upon the type of methodology used. A suitable combination of selected quantitative and texture features can further improve the accuracy of tumor classification. Machine learning (ML) classifiers based upon features derived from MRI data have shown potential in tumor classification. There is a need for further research studies on selecting an appropriate combination of features and evaluating the performance of different ML classifiers for accurate classification of breast tumors. The objective of the current study was to develop and optimize an ML framework based upon mp-MRI features for the characterization of breast tumors (malignant vs. benign and low- vs. high-grade). This study included the breast mp-MRI data of 60 female patients with histopathology results. A total of 128 features were extracted from the mp-MRI tumor data followed by features selection. Five ML classifiers were evaluated for tumor classification using 10-fold crossvalidation with 10 repetitions. The support vector machine (SVM) classifier based on optimum features selected using a wrapper method with an adaptive boosting (AdaBoost) technique provided the highest sensitivity (0.96 ± 0.03), specificity (0.92 ± 0.09), and accuracy (94% ± 2.91%) in the classification of malignant versus benign tumors. This method also provided the highest sensitivity (0.94 ± 0.07), specificity (0.80 ± 0.05), and accuracy (90% ± 5.48%) in the classification of low- versus high-grade tumors. These findings suggest that the SVM classifier outperformed other ML methods in the binary classification of breast tumors.

As technology advances so does its adoption in industry and our everyday lives. We use technology to better our daily tasks whether at home or in the workplace. In the workplace there are many benefits of implementing new technologies to improve processes and products while at the same time being cautious about associated costs. Costs associated with incorporating new digital technologies include but are not limited to hardware and software, system integration, testing, and employee training. Now that we have entered the era of the Fourth Industrial Revolution, there are many opportunities for improvements in manufacturing that involve Smart Digital Technologies. Quality Inspection (QI) is a common process in the manufacturing industry. This process serves to identify and locate defects based on preassigned product features. It can be used to find defect causes during the manufacturing process. QI is an established process that is often conducted manually. However, with the opportunity afforded by core Industry 4.0 technologies like Machine Learning (ML), the QI process is increasingly automated. A literature review on the topic of Image Based QI shows that today the designs of fixed single camera based QI systems inspect only one side of a product. In case multiple sides of a product need to be inspected, more datapoints are acquired via either multiple cameras or moving the camera system which affects the length of process, task complexity, cost, and data storage. This thesis explores the application of a mirror system in the design of a QI station to overcome this current limitation and potentially lower the number of processing tasks and increase the inspection ability of one automated station. The images acquired for the QI experiment present a simultaneous, multi-side (five sides) view of a product with similar accuracy. The experiment is set up with three major parts: 1) a physical setup of a mirror-enhanced camera system for part QI; 2) a set of test parts with two types of simulated common surface defects (additional material and scratches) to create the data set for analysis, and 3) a Python-based ML model to analyze the data set and evaluate the system’s ability to correctly identify errors on multiple sides of the specimens. The results show that the setup successfully identified the majority of surface errors on all sides of the part with an accuracy of at least 85% when applying an Artificial Neural Network based supervised ML classification approach. This

The detection accuracy and model building time of machine learning (ML) classifiers are vital aspects for an intrusion detection system (IDS) to predict attacks in real life. Recently, researchers have introduced feature selection methods to increase the detection accuracy and minimize the model building time of a limited number of ML classifiers. Therefore, identifying more ML classifiers with very high detection accuracy and the lowest possible model building time is necessary. In this study, the authors tested six supervised classifiers on a full NSL-KDD training dataset (a benchmark record for Internet traffic) using 10-fold cross-validation in the Weka tool with and without feature selection/reduction methods. The authors aimed to identify more options to outperform and secure classifiers with the highest detection accuracy and lowest model building time. The results show that the feature selection/reduction methods, including the wrapper method in combination with the discretize filter, the filter method in combination with the discretize filter, and the discretize filter, can significantly decrease model building time without compromising detection accuracy. The suggested ML algorithms and feature selection/reduction methods are automated pattern recognition approaches to detect network attacks, which are within the scope of the Symmetry journal.