A new intelligent multilayer framework for insider threat detection

Given the demand for developing the efficient Machine Learning (ML) classification models for healthcare data, and the potentiality of Bio-Inspired Optimization (BIO) algorithms to tackle the problem of high dimensional data, we investigate the range of ML classification models trained with the optimal subset of features of PD data set for efficient PD classification. We used two BIO algorithms, Genetic Algorithm (GA) and Binary Particle Swarm Optimization (BPSO), to determine the optimal subset of features of PD data set. The data set chosen for investigation comprises 756 observations (rows or records) taken over 755 attributes (columns or dimensions or features) from 252 PD patients. We employed MaxAbsolute feature scaling method to normalize the data and one hold cross-validation method to avoid biased results. Accordingly, the data is split in to training and testing set in the ratio of 70% and 30%. Subsequently, we employed GA and BPSO algorithms separately on 11 ML classifiers (Logistic Regression (LR), linear Support Vector Machine (lSVM), radial basis function Support Vector Machine (rSVM), Gaussian Naïve Bayes (GNB), Gaussian Process Classifier (GPC), k-Nearest Neighbor (kNN), Decision Tree (DT), Random Forest (RF), Multilayer Perceptron (MLP), Ada Boost (AB) and Quadratic Discriminant Analysis (QDA)), to determine the optimal subset of features (reduction of dimensionality) contributing to the highest classification accuracy. Among all the bio-inspired ML classifiers employed: GA-inspired MLP produced the maximum dimensionality reduction of 52.32% by selecting only 359 features and delivering 85.1% of the classification accuracy; GA-inspired AB delivered the maximum classification accuracy of 90.7% producing the dimensionality reduction of 41.43% by selecting only 441 features; And, BPSO-inspired GNB produced the maximum dimensionality reduction of 47.14% by selecting 396 features and delivering the classification accuracy of 79.3%; BPSOMLP delivered the maximum classification accuracy of 89% and produced 46.48% of the dimensionality reduction by selecting only 403 features.

Insider threats remain a persistent challenge in cybersecurity, as malicious or negligent insiders exploit legitimate access to compromise systems and data. This study presents a bibliometric review of 325 peer-reviewed publications from 2015 to 2025 to examine how machine learning (ML) and deep learning (DL) techniques for insider threat detection have evolved. The analysis investigates temporal publication trends, influential authors, international collaboration networks, thematic shifts, and algorithmic preferences. Results show a steady rise in research output and a transition from traditional ML models, such as decision trees and random forests, toward advanced DL methods, including long short-term memory (LSTM) networks, autoencoders, and hybrid ML–DL frameworks. Co-authorship mapping highlights China, India, and the United States as leading contributors, while keyword analysis underscores the increasing focus on behavior-based and eXplainable AI models. Symmetry emerges as a central theme, reflected in balancing detection accuracy with computational efficiency, and minimizing false positives while avoiding false negatives. The study recommends adaptive hybrid architectures, particularly Bidirectional LSTM–Variational Auto-Encoder (BiLSTM-VAE) models with eXplainable AI, as promising solutions that restore symmetry between detection accuracy and transparency, strengthening both technical performance and organizational trust.

The rapid expansion of cyber-physical systems (CPSs) has introduced new security challenges, leading to the emergence of various threats, attacks, and controls aimed at addressing security concerns in this evolving CPS landscape. However, a noticeable gap exists in the literature, particularly in the field of insider threat detection, which lacks a systematic review of CPS security. This study aims to comprehensively review and analyse relevant studies on insider threat detection in CPS. Employing a systematic protocol, we conducted an extensive search for pertinent articles across five prominent online databases: IEEE Xplore, Web of Science, Scopus, ACM, and ScienceDirect. The selection of these indices was based on their comprehensive coverage and the distinct relevance of their contents to our research topic. The results, guided by defined inclusion and exclusion criteria, yielded a final set of 69 included articles. Notably, 39.1 % of these articles focused on insider threat detection using specification-based methods, while 27.5 % addressed cryptography methods. Machine learning methods constituted 13.04 %, and the remaining 14.5 % included review and survey studies. Insider threats pose significant challenges in global cybersecurity, necessitating effective detection systems, methods, and tools for accurate and rapid identification. This study contributes distinct observations on the insider threat detection research topic in CPS, providing valuable insights for researchers and practitioners to expedite improvements and draw significant guidelines based on this comprehensive systematic review.

In recent years, data security incidents caused by insider threats in distributed file systems have attracted the attention of academia and industry. The most common way to detect insider threats is based on user profiles. Through analysis, we realize that based on existing user profiles are not efficient enough, and there are many false positives when a stable user profile has not yet been formed. In this work, we propose personalized user profiles and design an insider threat detection framework, which can intelligently detect insider threats for securing distributed file systems in real-time. To generate personalized user profiles, we come up with a time window-based clustering algorithm and a weighted kernel density estimation algorithm. Compared with non-personalized user profiles, both the Recall and Precision of insider threat detection based on personalized user profiles have been improved, resulting in their harmonic mean F1 increased to 96.52%. Meanwhile, to reduce the false positives of insider threat detection, we put forward operation recommendations based on user similarity to predict new operations that users will produce in the future, which can reduce the false positive rate (FPR). The FPR is reduced to 1.54% and the false positive identification rate (FPIR) is as high as 92.62%. Furthermore, to mitigate the risks caused by inaccurate authorization for users, we present user tags based on operation content and permission. The experimental results show that our proposed framework can detect insider threats more effectively and precisely, with lower FPR and high FPIR.

The rapid evolution of insider threats presents a significant challenge for cybersecurity systems, necessitating the development of advanced detection models capable of adapting to dynamic and sophisticated attack patterns. This chapter explores the integration of Hybrid Artificial Intelligence (AI) architectures combining Machine Learning (ML) and Deep Learning (DL) with emerging techniques such as Reinforcement Learning (RL) to enhance insider threat detection. The combination of ML and DL methodologies offers distinct advantages in handling large, complex datasets, while RL introduces the capability for real-time adaptation to evolving threat behaviors. By leveraging hybrid models, organizations can better detect subtle insider threats that traditional systems often fail to identify, improving accuracy and response times. Despite the promise, challenges such as model interpretability, data privacy, and the exploration-exploitation trade-off in RL systems must be addressed. The chapter discusses these hurdles and provides insights into how hybrid architectures can be optimized to overcome them, with a focus on scalability, security, and the ability to continuously learn from new data. Through a comprehensive review of the current research and methodologies, this chapter aims to highlight the potential of hybrid AI models in revolutionizing the way insider threats are detected and mitigated in real-time, offering a path toward more resilient and adaptive cybersecurity systems.

Insider threat represents a major cybersecurity challenge to companies and government agencies. The challenges in insider threat detection include unbalanced data, limited ground truth, and possible user behaviour changes. This research presents an unsupervised machine learning (ML) based anomaly detection approach for insider threat detection. We employ two ML methods with different working principles, specifically auto-encoder and isolation forest, and explore various representations of data with temporal information. Evaluation results show that the approach allows learning from unlabelled data under adversarial conditions for insider threat detection with a high detection and a low false positive rate. For example, 60% of malicious insiders are detected under 0.1% investigation budget. Furthermore, we explore the ability of the proposed approach to generalize for detecting unseen anomalous behaviours in different datasets, i.e. robustness. Comparisons with other work in the literature confirm the effectiveness of the proposed approach.

Social media has become an essential tool for travel planning, with tourists increasingly using it to research destinations, book accommodation, and make travel arrangements. However, little is known about how tourists use social media for travel planning and what factors influence their intentions to use social media for this purpose. This thesis aims to understand tourists’ intentions to use social media for travel planning. Specifically, it investigates the factors influencing tourists’ intentions to use social media for planning travel to Saudi Arabia. It develops a machine learning (ML) classification model to assist Saudi tourism SMEs in creating effective digital marketing strategies for social media platforms. A survey was conducted with 573 tourists interested in visiting Saudi Arabia, using the Design Science Research (DSR) approach. The findings support the tourist-based theoretical framework, showing that perceived usefulness (PU), perceived ease of use (PEOU), satisfaction (SAT), marketing-generated content (MGC), and user-generated content (UGC) significantly impact tourists’ intentions to use social media for travel planning. Tourists’ characteristics and visit characteristics influenced their intentions to use MGC but not UGC. The tourist-based ML classification model, developed using the LinearSVC algorithm, achieved an accuracy of 99% when evaluated using the K-Fold Cross-Validation (KF-CV) technique. The findings of this study have several implications for Saudi tourism SMEs. First, the results suggest that SMEs should focus on developing social media content that is perceived as useful, easy to use, and satisfying. Second, the findings suggest that SMEs should focus on using MGC in their social media marketing campaigns. Third, the results suggest that SMEs should tailor their social media marketing campaigns to the characteristics of their target tourists. This study contributes to the literature on tourism marketing and social media by providing a better understanding of how tourists use social media for travel planning. Saudi tourism SMEs can use the findings of this study to develop more effective digital marketing strategies for social media platforms.

Machine learning has achieved remarkable success across diverse domains. Nevertheless, concerns about interpretability in black-box models, especially within Deep Neural Networks (DNNs), have become pronounced in safety-critical fields like healthcare and finance. Classical machine learning (ML) classifiers, known for their higher interpretability, are preferred in these domains. Similar to DNNs, classical ML classifiers can exhibit bugs that could lead to severe consequences in practice. Test input prioritization has emerged as a promising approach to ensure the quality of an ML system, which prioritizes potentially misclassified tests so that such tests can be identified earlier with limited manual labeling costs. However, when applying to classical ML classifiers, existing DNN test prioritization methods are constrained from three perspectives: 1) Coverage-based methods are inefficient and time-consuming; 2) Mutation-based methods cannot be adapted to classical ML models due to mismatched model mutation rules; 3) Confidence-based methods are restricted to a single dimension when applying to binary ML classifiers, solely depending on the model’s prediction probability for one class. To overcome the challenges, we propose MLPrior, a test prioritization approach specifically tailored for classical ML models. MLPrior leverages the characteristics of classical ML classifiers (i.e., interpretable models and carefully engineered attribute features) to prioritize test inputs. The foundational principles are: 1) tests more sensitive to mutations are more likely to be misclassified, and 2) tests closer to the model’s decision boundary are more likely to be misclassified. Building on the first concept, we design mutation rules to generate two types of mutation features (i.e., <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">model mutation features</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">input mutation features</b> ) for each test. Drawing from the second notion, MLPrior generates <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute features</b> of each test based on its attribute values, which can indirectly reveal the proximity between the test and the decision boundary. For each test, MLPrior combines all three types of features of it into a final vector. Subsequently, MLPrior employs a pre-trained ranking model to predict the misclassification probability of each test based on its final vector and ranks tests accordingly. We conducted an extensive study to evaluate MLPrior based on 185 subjects, encompassing natural datasets, mixed noisy datasets, and fairness datasets. The results demonstrate that MLPrior outperforms all the compared test prioritization approaches, with an average improvement of 14.74%∼66.93% on natural datasets, 18.55%∼67.73% on mixed noisy datasets, and 15.34%∼62.72% on fairness datasets.

PurposeThis study aims to emphasize utilization of Predictive Six Sigma to achieve process improvements based on machine learning (ML) techniques embedded in define, measure, analyze, improve, control (DMAIC). With this aim, this study presents selection and utilization of ML techniques, including multiple linear regression (MLR), artificial neural network (ANN), random forests (RF), gradient boosting machines (GBM) and k-nearest neighbors (k-NN) in the analyze and improve phases of Six Sigma DMAIC.Design/methodology/approachA data set containing 320 observations with nine input and one output variables is used. To achieve the objective which was to decrease the number of fabric defects, five ML techniques were compared in terms of prediction performance and best tools were selected. Next, most important causes of defects were determined via these tools. Finally, parameter optimization was conducted for minimum number of defects.FindingsAmong five ML tools, ANN, GBM and RF are found to be the best predictors. Out of nine potential causes, “machine speed” and “fabric width” are determined as the most important variables by using these tools. Then, optimum values for “machine speed” and “fabric width” for fabric defect minimization are determined both via regression response optimizer and ANN surface optimization. Ultimately, average defect number was decreased from 13/roll to 3/roll, which is a considerable decrease attained through utilization of ML techniques in Six Sigma.Originality/valueAddressing an important gap in Six Sigma literature, in this study, certain ML techniques (i.e. MLR, ANN, RF, GBM and k-NN) are compared and the ones possessing best performances are used in the analyze and improve phases of Six Sigma DMAIC.

Organizations’ own personnel now have a greater ability than ever before to misuse their access to critical organizational assets. Insider threat detection is a key component in identifying rare anomalies in context, which is a growing concern for many organizations. Existing perimeter security mechanisms are proving to be ineffective against insider threats. As a prospective filter for the human analysts, a new deep learning based insider threat detection method that uses the Dempster-Shafer theory is proposed to handle both accidental as well as intentional insider threats via organization’s channels of communication in real time. The long short-term memory (LSTM) architecture together with multi-head attention mechanism is applied in this work to detect anomalous network behavior patterns. Furthermore, belief is updated with Dempster’s conditional rule and utilized to fuse evidence to achieve enhanced prediction. The CERT Insider Threat Dataset v6.2 is used to train the behavior model. Through performance evaluation, our proposed method is proven to be effective as an insider threat detection technique.

This research aims to identify the best machine learning (ML) classification techniques for classifying the flow regimes in vertical gas-liquid two-phase flow. Two-phase flow regime identification is crucial for many operations in the oil and gas industry. Processes such as flow assurance, well control, and production rely heavily on accurate identification of flow regimes for their respective systems' smooth functioning. The primary motivation for the proposed ML classification algorithm selection processes was drilling and well control applications in Deepwater wells. The process started with vertical two-phase flow data collection from literature and two different flow loops. One, a 140 ft. tall vertical flow loop with a centralized inner metal pipe and a larger outer acrylic pipe. Second, an 18-ft long flow loop, also with a centralized, inner metal drill pipe. After extensive experimental and historical data collection, supervised and unsupervised ML classification models such as Multi-class Support vector machine (MCSVM), K-Nearest Neighbor Classifier (KNN), K-means clustering, and hierarchical clustering were fit on the datasets to separate the different flow regions. The next step was fine-tuning the models' parameters and kernels. The last step was to compare the different combinations of models and refining techniques for the best prediction accuracy and the least variance. Among the different models and combinations with refining techniques, the 5- fold cross-validated KNN algorithm, with 37 neighbors, gave the optimal solution with a 98% classification accuracy on the test data. The KNN model distinguished five major, distinct flow regions for the dataset and a few minor regions. These five regions were bubbly flow, slug flow, churn flow, annular flow, and intermittent flow. The KNN-generated flow regime maps matched well with those presented by Hasan and Kabir (2018). The MCSVM model produced visually similar flow maps to KNN but significantly underperformed them in prediction accuracy. The MCSVM training errors ranged between 50% - 60% at normal parameter values and costs but went up to 99% at abnormally high values. However, their prediction accuracy was below 50% even at these highly overfitted conditions. In unsupervised models, both clustering techniques pointed to an optimal cluster number between 10 and 15, consistent with the 14 we have in the dataset. Within the context of gas kicks and well control, a well-trained, reliable two-phase flow region classification algorithm offers many advantages. When trained with well-specific data, it can act as a black box for flow regime identification and subsequent well-control measure decisions for the well. Further advancements with more robust statistical training techniques can render these algorithms as a basis for well-control measures in drilling automation software. On a broader scale, these classification techniques have many applications in flow assurance, production, and any other area with gas-liquid two-phase flow.

BackgroundKnee osteoarthritis (OA) is the most common form of OA and a leading cause of disability worldwide. Chronic pain and functional loss secondary to knee OA put patients at risk of developing depression, which can also impair their treatment response. However, no tools exist to assist clinicians in identifying patients at risk. Machine learning (ML) predictive models may offer a solution. We investigated whether ML models could predict the development of depression in patients with knee OA and examined which features are the most predictive.ObjectiveThe primary aim of this study was to develop and test an ML model to predict depression in patients with knee OA at 2 years and to validate the models using an external data set. The secondary aim was to identify the most important predictive features used by the ML algorithms.MethodsOsteoarthritis Initiative Study (OAI) data were used for model development and external validation was performed using Multicenter Osteoarthritis Study (MOST) data. Forty-two features were selected, which denoted routinely collected demographic and clinical data such as patient demographics, past medical history, knee OA history, baseline examination findings, and patient-reported outcome measures. Six different ML classification models were trained (logistic regression, least absolute shrinkage and selection operator [LASSO], ridge regression, decision tree, random forest, and gradient boosting machine). The primary outcome was to predict depression at 2 years following study enrollment. The presence of depression was defined using the Center for Epidemiological Studies Depression Scale. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1 score. The most important features were extracted from the best-performing model on external validation.ResultsA total of 5947 patients were included in this study, with 2969 in the training set, 742 in the test set, and 2236 in the external validation set. For the test set, the AUC ranged from 0.673 (95% CI 0.604-0.742) to 0.869 (95% CI 0.824-0.913), with an F1 score of 0.435 to 0.490. On external validation, the AUC varied from 0.720 (95% CI 0.685-0.755) to 0.876 (95% CI 0.853-0.899), with an F1 score of 0.456 to 0.563. LASSO modeling offered the highest predictive performance. Blood pressure, baseline depression score, knee pain and stiffness, and quality of life were the most predictive features.ConclusionsTo our knowledge, this is the first study to apply ML classification models to predict depression in patients with knee OA. Our study showed that ML models can deliver a clinically acceptable level of performance (AUC>0.7) in predicting the development of depression using routinely available demographic and clinical data. Further work is required to address the class imbalance in the training data and to evaluate the clinical utility of the models in facilitating early intervention and improved outcomes.

The concealment and confusion nature of insider threat makes it a challenging task for security analysts to identify insider threat from log data. To detect insider threat, we propose a novel gated recurrent unit (GRU) and multi-autoencoder based insider threat detection method, which is an unsupervised anomaly detection method. It takes advantage of the extremely unbalanced characteristic of insider threat data and constructs a normal behavior autoencoder with low reconfiguration error through multi-level filter behavior learning, and identifies the behavior data with high reconfiguration error as abnormal behavior. In order to achieve the high efficiency of calculation and detection, GRU and multi-head attention are introduced into the autoencoder. Use dataset v6.2 of the CERT insider threat as validation data and threat detection recall as evaluation metric. The experimental results show that the effect of the proposed method is obviously better than that of Isolation Forest, LSTM autoencoder and multi-channel autoencoders based insider threat detection methods, and it&#x0027;s an effective insider threat detection technology.

Improving autocoding performance of rare categories in injury classification: Is more training data or filtering the solution?

The insider threat has consistently been identified as a key threat to organizations and governments. Understanding the nature of insider threats and the related threat landscape can help in forming mitigation strategies, including non-technical means. In this paper, we survey and highlight challenges associated with the identification and detection of insider threats in both public and private sector organizations, especially those part of a nation’s critical infrastructure. We explore the utility of the cyber kill chain to understand insider threats, as well as understanding the underpinning human behavior and psychological factors. The existing defense techniques are discussed and critically analyzed, and improvements are suggested, in line with the current state-of-the-art cyber security requirements. Finally, open problems related to the insider threat are identified and future research directions are discussed.