Comparison of Supervised Machine Learning Approaches for Enhanced File Malware Detection: Techniques and Evaluations

In recent years, a significant amount of research has focused on analyzing the effectiveness of machine learning (ML) models for malware detection. These approaches have ranged from methods such as decision trees and clustering to more complex approaches like support vector machine (SVM) and deep neural networks. In particular, neural networks have proven to be very effective in detecting complex and advanced malware. This, however, comes with a caveat. Neural networks are notoriously complex. Therefore, the decisions that they make are often just accepted without questioning why the model made that specific decision. The black box characteristic of neural networks has challenged researchers to explore methods to explain black-box models such as SVM and neural networks and their decision-making process. Transparency and explainability give the experts and malware analysts assurance and trustworthiness about the ML models’ decisions. In addition, it helps in generating comprehensive reports that can be used to enhance cyber threat intelligence sharing. As such, this much needed analysis drives our work in this paper to explore the explainability and interpretability of ML models in the field of online malware detection. In this paper, we used the Shapley Additive exPlanations (SHAP) explainability technique to achieve efficient performance in interpreting the outcome of different ML models such as SVM Linear, SVM-RBF (Radial Basis Function), Random Forest (RF), Feed-Forward Neural Net (FFNN), and Convolutional Neural Network (CNN) models trained on an online malware dataset. To explain the output of these models, explainability techniques such as KernalSHAP, TreeSHAP, and DeepSHAP are applied to the obtained results.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

In recent years, infections and damage caused by malware have increased at exponential rates. At the same time, machine learning (ML) techniques have shown tremendous promise in many domains, often out performing human efforts by learning from large amounts of data. Results in the open literature suggest that ML is able to provide similar results for malware detection, achieving greater than 99% classification accuracy [49]. However, the same detection rates when applied in deployed settings have not been achieved. Malware is distinct from many other domains in which ML has shown success in that (1) it purposefully tries to hide, leading to noisy labels and (2) often its behavior is similar to benign software only differing in intent, among other complicating factors. This report details the reasons for the difficultly of detecting novel malware by ML methods and offers solutions to improve the detection of novel malware. We propose to detect malware by detecting behaviors commonly exhibited by malware such as DLL injection, and process hollowing. This is based on the assumption that there is a set of behaviors that are common to most malware samples and detecting them will generalize to novel malware. Additionally, detected behaviors point analysts toward appropriate handling and mitigation strategies, which is not the case with a binary benign/malicious classification. A behavior labeling method was developed and was used to label an existing malware dataset. Results show that detecting malicious behaviors is much more difficult than simply classifying malware and goodware?achieving 80% accuracy compared to reported 99% accuracy from classifying malware and goodware. This drop is due to several reasons which are detailed in the report. We also propose to evaluate the performance of detecting novel malware by holding out a malware family for testing and training on the other families. Traditional ML evaluation will shuffle the data and then split the data into training and testing. Our method addresses the use-case when novel malware families are encountered and they require more than just a malicious or benign designation. Our results suggest that this type of evaluation is much more difficult than traditional methods and provides more realistic results, albeit, significantly worse. For our behavior detection, accuracy decreases from 80% to 68% across all behaviors when holding out a malware family from training. We show that the degradation in performance is because each malware family has distinct characteristics resulting in high extrapolations by an ML model. Here, an ML model should return an "I do not know" response and request further analysis from an analyst. We run a number of experiments that compare novel malware families to the training data using different feature representations including a genomics-inspired distance measure and features extracted by deep learning. Generally, held-out families are significantly different from the training data, resulting in unpredictable results. This has been observed generally in the ML community [22, 9]. We empirically demonstrate this in the domain of malware detection. In an attempt to improve the detection of malware behaviors, we examine the impact that additional synthetic data has on the performance of an ML model in detecting behaviors in novel malware families. We find that while synthetic data does improve the performance of ML models, often simpler methods perform better than more complicated ones. Two generative modeling techniques were examined to produce synthetic malware samples such that the behaviors present are able to be specified externally. The difficulty is due to finer grained analysis of the executable and modifying the problem from a binary classification problem to a multi-label problem. The addition of synthetic data increases the overall accuracy from 68% to 70%. While far less accurate than measures presented in academic analyses, we believe that this is more representative of real-world performance and allows models to be properly placed within a malware detection system. We suggest that in highly dynamic environments ML pipelines should determine whether an ML model is competent in the area of new data and should involve mechanisms to improve over time with a human in the loop.

Improve clinical feature-based bladder cancer survival prediction models through integration with gene expression profiles and machine learning techniques

Underwriting decisions by insurance companies make a significant contribution to their profitability. Machine Learning (ML) techniques in underwriting decision making have saved time and improved operational efficiencies. A user-friendly cause-and-effect explanation of model’s predictions is useful to stakeholders, financial institutions and regulators. This research performed comparative analysis between tree-based classifiers such as Decision Tree, Random Forest and XGBoost. The study focused on enhancing risk assessment capabilities for life insurance companies using predictive analytics by classifying the insurance risk based on the historical data and propose the appropriate model to assess risk. Its purpose also included incorporating mechanisms that can aid in user friendly interpretation of ML models. Of all the models created as part of this research, the XGBoost classifier performed the best when compared to other classifiers, with an AUC value of 0.86 and F1-score above 0.56 on the validation set. The Random Forest classifier got AUC value of .84 and f1 score of .53 on the validation dataset. The results indicate the importance and advantages of tree -based models. These models i.e., XGBoost, decision tree and random forest are one of the best alternate techniques after the advent and popularity of the new age techniques in the machine learning such as neural networks, deep learning etc. The research also provides an insight on the interpretability of these conventional techniques by way of ‘SHAP’ or shapley values and ‘Feature Importance’ or ‘Variable Importance’. SHAP was used on complex models such as XGBoost and neural networks whereas Feature Importance is used in supervised learning methods such as Logistic Regression and tree- based models such as Decision Tree and Random Forest. Overall, the study was able to propose XGBoost as the most accurate model for Insurance risk classification and predictions.

For effective management of water quantity and quality, it is absolutely essential to estimate the pollution level of the existing surface water. This case study aims to evaluate the performance of twelve machine learning (ML) models, including five boosting-based algorithms (adaptive boosting, gradient boosting, histogram-based gradient boosting, light gradient boosting, and extreme gradient boosting), three decision tree-based algorithms (decision tree, extra trees, and random forest), and four ANN-based algorithms (multilayer perceptron, radial basis function, deep feed-forward neural network, and convolutional neural network), in estimating the surface water quality of the La Buong River in Vietnam. Water quality data at four monitoring stations alongside the La Buong River for the period 2010–2017 were utilized to calculate the water quality index (WQI). Prediction performance of the ML models was evaluated by using two efficiency statistics (i.e., R2 and RMSE). The results indicated that all twelve ML models have good performance in predicting the WQI but that extreme gradient boosting (XGBoost) has the best performance with the highest accuracy (R2 = 0.989 and RMSE = 0.107). The findings strengthen the argument that ML models, especially XGBoost, may be employed for WQI prediction with a high level of accuracy, which will further improve water quality management.

Harmful Algal Blooms (HABs), driven by environmental pollution, pose significant threats to water quality, public health, and aquatic ecosystems. This study aims to enhance the prediction of HABs in Lake Erie, part of the Great Lakes system, by utilizing ensemble machine learning (ML) models coupled with explainable artificial intelligence (XAI) for interpretability. Using water quality data from 2013 to 2020, various physical, chemical, and biological parameters were analyzed to predict chlorophyll-a (Chl-a) concentrations, a proxy for algal blooms. The study employed multiple ensemble ML models, including Random Forest (RF), Deep Forest (DF), Gradient Boosting (GB), and XGBoost, and compared their performance against individual models such as Support Vector Machine (SVM), Decision Tree (DT), and Multi-Layer Perceptron (MLP). The findings reveal that ensemble models, particularly XGBoost and Deep Forest (DF), achieve superior predictive accuracy with R² values of 0.8517 and 0.8544, respectively. The application of SHapley Additive exPlanations (SHAP) provided insights into the relative importance of input features, identifying Particulate Organic Nitrogen (PON), Particulate Organic Carbon (POC), and Total Phosphorus (TP) as critical factors influencing Chl-a concentrations. This research demonstrates the effectiveness of integrating ensemble ML models with XAI to improve HAB prediction accuracy and interpretability. The results support the development of proactive water quality management strategies and highlight the potential of advanced ML techniques in environmental monitoring.

This study aims to analyze the performance of machine learning models for detecting Internet of Things malware utilizing a recent IoT dataset. Experiments on the IoT dataset were conducted with nine well-known machine learning techniques, consisting of Logistic Regression (LR), Naive Bayes (NB), Decision Tree (DT), k-Nearest Neighbors (KNN), Support Vector Machines (SVM), Neural Networks (NN), Random Forest (RF), Bagging (BG), and Stacking (ST). The results show that the proposed model attains 100% accuracy in detecting IoT malware for DT, SVM, RF, BG; about 99.9% percent for LR, NB, KNN, NN; and only 28.16% for ST classifier. This study also shows higher performance than other proposed machine learning models evaluated on the same dataset. Therefore, the results of this study can help both the researchers and application developers in designing and building intelligent malware detection systems for IoT devices.

This study explores how machine learning (ML) models can predict the compressive strength of concrete when soap factory wastewater is used as a partial replacement for mixing water. Using a dataset with key mix parameters like cement content, water-to-cement ratio, aggregates, and curing age, five ML models—Gradient Boosting, Random Forest, Decision Tree, k-Nearest Neighbors (kNN), and Linear Regression—were tested. Among them, Gradient Boosting performed best, with the highest accuracy (R² = 0.868) and lowest error. Random Forest also showed strong results, while Decision Tree was moderately effective. In contrast, kNN and Linear Regression struggled to capture the complex relationships in the data. These findings highlight the potential of ML in optimizing concrete mix design and promoting sustainable construction practices.

For the various climatic zones of India, machine learning (ML) models are created in the current work to forecast monthly-average diffuse solar radiation (DSR). The long-term solar radiation data are taken from Indian Meteorological Department (IMD), Pune, provided for 21 cities that span all of India’s climatic zones. The diffusion coefficient and diffuse fraction are the two groups of ML models with dual input parameters (sunshine ratio and clearness index) that are built and compared (each category has seven models). To create ML models, two well-known ML techniques, random forest (RF) and k-nearest neighbours (KNN), are used. The proposed ML models are compared with well-known models that are found in the literature. The ML models are ranked according to their overall and within predictive power using the Global Performance Indicator (GPI). It is discovered that KNN models generally outperform RF models. The results reveal that in diffusion coefficient models perform well than diffuse fraction models. Moreover, functional form 2 is the best followed by form 6. The ML models created here can be effectively used to accurately forecast DSR in various climates.

In recent years, the microbiome has been recognized as an important factor associated with cardiovascular disease (CVD), which is the leading cause of human mortality worldwide. Disparities in gut microbial compositions between individuals with and without CVD were reported, whereby, we hypothesized that utilizing such microbiome-based data for training with supervised machine learning (ML) models could be exploited as a new strategy for evaluation of cardiovascular health. To test our hypothesis, we analyzed the metagenomics data extracted from the American Gut Project. Specifically, 16S rRNA reads from stool samples of 478 CVD and 473 non-CVD control samples were analyzed using five supervised ML algorithms: random forest (RF), support vector machine with radial kernel (svmRadial), decision tree (DT), elastic net (ENet) and neural networks (NN). Thirty-nine differential bacterial taxa (LEfSe: LDA > 2) were identified between CVD and non-CVD groups. ML classifications, using these taxonomic features, achieved an AUC (area under the receiver operating characteristic curve) of ~0.58 (RF). However, by choosing the top 500 high-variance features of operational taxonomic units (OTUs) for training ML models, an improved AUC of ~0.65 (RF) was achieved. Further, by limiting the selection to only the top 25 highly contributing OTU features to reduce the dimensionality of feature space, the AUC was further significantly enhanced to ~0.70 (RF). In summary, this study is the first to demonstrate the successful development of a ML model using microbiome-based datasets for a systematic diagnostic screening of CVD.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak

Malware detection plays a critical role in securing endpoints such as workstations, servers, cloud instances, and mobile devices. It involves identifying and mitigating malicious activities triggered by various forms of malware, including viruses, trojans, worms, ransomware, adware, and spyware. With the rapid proliferation of sophisticated and evasive malware variants, such as polymorphic and metamorphic malware, traditional signature-based and heuristic approaches are no longer sufficient. These conventional methods often fail to detect previously unseen or obfuscated threats, making the adoption of more intelligent and adaptive systems essential. In response to this challenge, this project explores the application of machine learning techniques to enhance malware detection and classification. Leveraging publicly available malware datasets, various supervised and unsupervised algorithms—such as Decision Trees, Random Forests, Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), Naïve Bayes, and Artificial Neural Networks (ANN)—will be employed and evaluated. Additionally, feature extraction and dimensionality reduction techniques such as Principal Component Analysis (PCA) will be applied to improve model performance. The objective is to assess and compare the models using metrics such as accuracy, precision, recall, F1-score, and confusion matrix analysis. This project aims to identify the most effective machine learning model for real-time, scalable, and accurate malware detection, contributing to improved cybersecurity practices and proactive threat mitigation. Keywords: Malware detection, machine learning, enterprise security, information gain, random forest, PE analysis, API calls.

Systematic literature review of machine learning based software development effort estimation models

Development of advanced hybrid computational model for description of molecular separation in liquid phase via polymeric membranes