Anomaly Detection for Insider Threats: An Objective Comparison of Machine Learning Models and Ensembles

<p>Ensemble learning is one of the most powerful extensions for improving upon individual machine learning models. Rather than a single model being used, several models are trained and the predictions combined to make a more informed decision. Such combinations will ideally overcome the shortcomings of any individual member of the ensemble. Most ma- chine learning competition winners feature an ensemble of some sort, and there is also sound theoretical proof to the performance of certain ensem- bling schemes. The benefits of ensembling are clear in both theory and practice. Despite the great performance, ensemble learning is not a trivial task. One of the main difficulties is designing appropriate ensembles. For exam- ple, how large should an ensemble be? What members should be included in an ensemble? How should these members be weighted? Our first contribution addresses these concerns using a strongly-typed population- based search (genetic programming) to construct well-performing ensem- bles, where the entire ensemble (members, hyperparameters, structure) is automatically learnt. The proposed method was found, in general, to be significantly better than all base members and commonly used compari- son methods trialled. With automatically designed ensembles, there is a range of applica- tions, such as competition entries, forecasting and state-of-the-art predic- tions. However, often these applications also require additional prepro- cessing of the input data. Above the ensemble considers only the original training data, however, in many machine learning scenarios a pipeline is required (for example performing feature selection before classification). For the second contribution, a novel automated machine learning method is proposed based on ensemble learning. This method uses a random population-based search of appropriate tree structures, and as such is em- barrassingly parallel, an important consideration for automated machine learning. The proposed method is able to achieve equivalent or improved results over the current state-of-the-art methods and does so in a fraction of the time (six times as fast). Finally, while complex ensembles offer great performance, one large limitation is the interpretability of such ensembles. For example, why does a forest of 500 trees predict a particular class for a given instance? In an effort to explain the behaviour of complex models (such as ensem- bles), several methods have been proposed. However, these approaches tend to suffer at least one of the following limitations: overly complex in the representation, local in their application, limited to particular fea- ture types (i.e. categorical only), or limited to particular algorithms. For our third contribution, a novel model agnostic method for interpreting complex black-box machine learning models is proposed. The method is based on strongly-typed genetic programming and overcomes the afore- mentioned limitations. Multi-objective optimisation is used to generate a Pareto frontier of simple and explainable models which approximate the behaviour of much more complex methods. We found the resulting rep- resentations are far simpler than existing approaches (an important con- sideration for interpretability) while providing equivalent reconstruction performance. Overall, this thesis addresses two of the major limitations of existing ensemble learning, i.e. the complex construction process and the black- box models that are often difficult to interpret. A novel application of ensemble learning in the field of automated machine learning is also pro- posed. All three methods have shown at least equivalent or improved performance than existing methods.</p>

Sub-Saharan Africa continues to shoulder the heaviest burden of malaria. The 2024 World Health Organisation (WHO) malaria report highlighted that Africa contributed an alarming 94% of the global cases and 95% of the deaths. In the WHO African region, progress towards elimination and management of malaria is hindered by weak health systems and a lack of traditional diagnostic methods such as microscopy and malaria Rapid Diagnostic Tests (mRDTs). The primary aim of this paper is to develop a machine learning (ML) ensemble model for malaria diagnosis using clinical and demographic data, tailored for resource-limited settings. A retrospective study was conducted using 637 patient records from Gutu Mission Hospital and Gweru Provincial Hospital in Zimbabwe. Clinical symptoms (fever, chills, abdominal pain, headache and diarrhea) and demographic features (age, gender, residence and travel history) were analysed. Data preprocessing included handling class imbalance using the Synthetic Minority Oversampling Technique (SMOTE) and feature selection using Recursive Feature Elimination (RFE). Seven individual ML models, including Logistic regression (LR), Random Forest (RF), Decision Trees (DT), Gradient Boosting (GB), K-Nearest Neighbour (KNN), Naive Bayes (NB) and XGBoost were trained and evaluated on the malaria dataset. The individual models were further combined to build, train and evaluate ensemble models such as Bagging, Stacking, Soft Voting and AdaBoost. Model performance was assessed using accuracy, precision, confusion matrices, recall, F1 score, and AUC-ROC metrics. Clinical symptoms (chills: p-value=0.001, fever: p=0.003, diarrhea: p=0.01, abdominal pain: p< 0.001) were statistically significant predictors of malaria. Of the demographic factors, only travel history (p=0.02) showed a significant association with malaria. Among the seven individual ML models, GB achieved the highest predictive performance (Accuracy =0.94), followed by RF (Accuracy =0.94) and XGBoost (Accuracy =0.93). The stacking ensemble model outperformed all individual ML models and other ensemble models (bagging, soft voting and AdaBoost), achieving accuracy =0.96, precision =0.95, recall =0.98, F1 score =0.96 and AUC-ROC =0.98. This study demonstrates that ML, particularly ensemble models, can be used to improve malaria diagnosis significantly. The integration of these models into a web-based application could provide a scalable and accessible diagnostic tool for healthcare workers in resource-limited settings.

Machine learning methods may have the potential to significantly accelerate drug discovery. However, the increasing rate of new methodological approaches being published in the literature raises the fundamental question of how models should be benchmarked and validated. We reanalyze the data generated by a recently published large-scale comparison of machine learning models for bioactivity prediction and arrive at a somewhat different conclusion. We show that the performance of support vector machines is competitive with that of deep learning methods. Additionally, using a series of numerical experiments, we question the relevance of area under the receiver operating characteristic curve as a metric in virtual screening. We further suggest that area under the precision–recall curve should be used in conjunction with the receiver operating characteristic curve. Our numerical experiments also highlight challenges in estimating the uncertainty in model performance via scaffold-split nested cross validation.

Student’s mental health problems have been explored previously in higher education literature in various contexts including empirical work involving quantitative and qualitative methods. Nevertheless, comparatively few research could be found, aiming for computational methods that learn information directly from data without relying on set parameters for a predetermined equation as an analytical method. This study aims to investigate the performance of Machine learning (ML) models used in higher education. ML models considered are Naïve Bayes, Support Vector Machine, K-Nearest Neighbor, Logistic Regression, Stochastic Gradient Descent, Decision Tree, Random Forest, XGBoost (Extreme Gradient Boosting Decision Tree), and NGBoost (Natural) algorithm. Considering the factors of mental health illness among students, we follow three phases of data processing: segmentation, feature extraction, and classification. We evaluate these ML models against classification performance metrics such as accuracy, precision, recall, F1 score, and predicted run time. The empirical analysis includes two contributions: 1. It examines the performance of various ML models on a survey-based educational dataset, inferring a significant classification performance by a tree-based XGBoost algorithm; 2. It explores the feature importance [variables] from the datasets to infer the significant importance of social support, learning environment, and childhood adversities on a student’s mental health illness.

Crop yield prediction is of great importance for decision making, yet it remains an ongoing scientific challenge. Interactions among different genetic, environmental, and management factors and uncertainty in input values are making crop yield prediction complex. Building upon a previous work in which we coupled crop modeling with machine learning (ML) models to predict maize yields for three US Corn Belt states, here, we expand the concept to the entire US Corn Belt (12 states). More specifically, we built five new ML models and their ensemble models, considering the scenarios with and without crop modeling variables. Additional input values in our models are soil, weather, management, and historical yield data. A unique aspect of our work is the spatial analysis to investigate causes for low or high model prediction errors. Our results indicated that the prediction accuracy increases by coupling crop modeling with machine learning. The ensemble model overperformed the individual ML models, having a relative root mean square error (RRMSE) of about 9% for the test years (2018, 2019, and 2020), which is comparable to previous studies. In addition, analysis of the sources of error revealed that counties and crop reporting districts with low cropland ratios have high RRMSE. Furthermore, we found that soil input data and extreme weather events were responsible for high errors in some regions. The proposed models can be deployed for large-scale prediction at the county level and, contingent upon data availability, can be utilized for field level prediction.

Designing an electrical motor is a complex process that needs to deal with the non-linearity phenomena caused by the saturation of the iron at high magnetic field strength, the multi-physical nature of the investigated system and with requirements that may come into conflict. This paper proposes to use geometric parametric models to evaluate the multi-physical performances of electrical machines and build a machine learning model that is able to predict multi-physical characteristics of electrical machines from input geometrical parameters. The focus of this work is to accurately estimate the electromagnetic characteristics, motor losses and stator natural frequencies, using the developed machine learning model, at the early-design stage of the electrical motor, when the information about the housing is not available and to include the model in optimisation loops, to speed-up the computational time. Three individual machine learning models are built for each physics analysed, a model for the torque and back electromotive force harmonic orders, one model for motor losses and another one for natural frequencies of the mode-shapes. The necessary data is obtained by varying the geometrical parameters of 2D electromagnetic and 3D structural motor parametric models. The accuracy of different machine learning regression algorithms are compared to obtain the best model for each physics involved. Ultimately, the developed multi-attribute model is integrated in an optimisation routine, to compare the computational time with the classical finite element analysis (FEA) optimisation approach.

Abdominal IgA vasculitis without purpura (IgAVNP) shares clinical manifestations with acute uncomplicated appendicitis (AUA), and traditional diagnostic methods have limitations. Therefore, more accurate diagnostic tools are needed for differentiation. A retrospective analysis was conducted on 702 children aged 2-14 years who were diagnosed between 2011 and 2020. Variables were selected using LASSO regression, controlling for confounding factors. Eight machine learning (ML) models were constructed and validated using independent data from 2021-2024. LASSO regression identified constipation, fatigue, hematochezia, as well as PLT, ALB, and AST as key predictive factors. Comparison of ML models revealed that logistic regression (LR) performed the best in the test set (AUC = 0.942, Brier score (BS) = 0.0680), demonstrating high clinical utility, sensitivity, and negative predictive value. The independent validation set further confirmed the stable performance of LR (AUC = 0.942, BS = 0.062). A nomogram based on LR provided a clear visual assessment of disease risk. The LR model, based on clinical symptoms and laboratory indicators, effectively distinguishes IgAVNP from AUA, with high diagnostic accuracy and significant clinical value. This study effectively distinguishes IgAVNP from AUA using machine learning models, particularly logistic regression, reducing the risk of misdiagnosis. The key factors identified and the nomogram tool can assist in clinical decision-making, optimizing diagnostic and treatment.

Remote sensing estimation of rice chlorophyll content based on UAV image feature selection and PSO-optimized ensemble learning

Optimization of in vitro organogenesis of Glossostigma elatinoides (Benth) Hook. f. was targeted in this study. The experiment was designed with the help of design of experiment (DOE) of response surface methodology (RSM) model. Murashige and Skoog (MS) basal salt, sucrose, and agar were used as input factors and a total of 15 runs were used for optimization. Results were analyzed by ANOVA and response surface regression analysis (RSRA) followed by prediction and validation via different machine learning (ML) models. Results of ANOVA revealed the impact of different combinations on output parameters. Results of RSRA illustrated the relationship between input and output parameters. Pareto chart analysis showed the significant impact of MS on clump diameter, fresh wt., and dry wt. Normal plot analysis illustrated the positive impact of MS on all output parameters and increased proportionally with MS concentration. Results of heatmap and network analysis also demonstrated the significance of MS on all output parameters. Comparison of ML models depicted the better performance of multilayer perceptron (MLP) model for rooting (R 2 = 0.957), fresh wt (R 2 = 0.806), and dry wt (R 2 = 0.812). Conversely, the support vector regression (SVR) model demonstrated superior prediction for clump diameter (R 2 = 0.809). Among the tested models, the SVR model showed the weakest performance, aside from clump diameter, while LightGBM achieved scores close to those of the RF and MLP models across all metrics. The findings clearly indicate that the adopted protocol is well-suited for the effective commercial propagation of the aquatic G. elatinoides plant.

A new intelligent multilayer framework for insider threat detection

Enhancing estimation of water quality index using stacking machine learning techniques: The case of Southern Bug River.

Laser powder bed fusion (LPBF) is a widely used Additive Manufacturing (AM) process involving a high-powered laser to fuse metal powder and build up successive layers to create complex structures. Process parameter selection significantly impacts the dimensional characteristics of the melt pool morphology, including its depth and width, which is vital to the final print quality. Accurately predicting the melt-pool depth is critical for ensuring the quality of the final product. Various methods for different materials have been investigated, for example, thermal image analysis, analytical methods like scaling law, machine learning, and so on. However, most of the promising results are only studies for one material, and it is not necessarily fully applied to another material. In this paper, our study involves utilizing machine learning models to make predictions of melt-pool depth in the LPBF process and compare their performance to an analytical approach based on scaling law with Nickel alloys IN625 and IN718. We trained three different machine learning models which are Gaussian Process Regression (GPR), Support Vector Machines (SVM), and Neural Networks (NN), and evaluated their performance using R-squared error and Root Mean Squared Error (RMSE) against the performance of the analytical scaling law. Our results show that the machine learning models outperform the scaling law approach significantly. This study demonstrates the potential of machine learning for improving the accuracy of melt-pool depth prediction in LPBF and provides insights into the strengths and limitations of different machine learning models for this task.

Motivated to address the inconsistency between the essential i.i.d. assumption in machine learning theory and the data heterogeneity in real-world applications, we propose a novel calibrated ensemble (CE) algorithm to facilitate learning with diverse data subgroups. Unlike the traditional ensemble framework in which each learner is trained independently using the entire dataset, our method exploits the strengths of various machine learning models by training them simultaneously and forming model-ergonomic data subgroups as part of the training process. Consequently, each learner is calibrated to a unique subset of data based on their individualized predictive strength. Clinically, we can interpret each model as an expert specializing in treating patients with particular disease manifestations. We evaluate the CE model in our motivating domain of identifying lupus patients with severe SLE flares using 1541 clinical encounters in the Mass General Brigham (MGB) Lupus Cohort. Our experimental results demonstrate the efficacy of our CE model across seven performance evaluation metrics compared to five individual machine learning models and regular ensemble approaches. We further utilize ANOVA and Tukey HSD post-hoc statistical analysis to discover characteristic features of individual model clusters for clinical interpretations.

Scrutinization of land subsidence rate using a supportive predictive model: Incorporating radar interferometry and ensemble soft-computing

IntroductionPatients with sepsis who present to an emergency department (ED) have highly variable underlying disease severity, and can be categorized from low to high risk. Development of a risk stratification tool for these patients is important for appropriate triage and early treatment. The aim of this study was to develop machine learning models predicting 31-day mortality in patients presenting to the ED with sepsis and to compare these to internal medicine physicians and clinical risk scores.MethodsA single-center, retrospective cohort study was conducted amongst 1,344 emergency department patients fulfilling sepsis criteria. Laboratory and clinical data that was available in the first two hours of presentation from these patients were randomly partitioned into a development (n = 1,244) and validation dataset (n = 100). Machine learning models were trained and evaluated on the development dataset and compared to internal medicine physicians and risk scores in the independent validation dataset. The primary outcome was 31-day mortality.ResultsA number of 1,344 patients were included of whom 174 (13.0%) died. Machine learning models trained with laboratory or a combination of laboratory + clinical data achieved an area-under-the ROC curve of 0.82 (95% CI: 0.80–0.84) and 0.84 (95% CI: 0.81–0.87) for predicting 31-day mortality, respectively. In the validation set, models outperformed internal medicine physicians and clinical risk scores in sensitivity (92% vs. 72% vs. 78%;p<0.001,all comparisons) while retaining comparable specificity (78% vs. 74% vs. 72%;p>0.02). The model had higher diagnostic accuracy with an area-under-the-ROC curve of 0.85 (95%CI: 0.78–0.92) compared to abbMEDS (0.63,0.54–0.73), mREMS (0.63,0.54–0.72) and internal medicine physicians (0.74,0.65–0.82).ConclusionMachine learning models outperformed internal medicine physicians and clinical risk scores in predicting 31-day mortality. These models are a promising tool to aid in risk stratification of patients presenting to the ED with sepsis.