Predictive Analytics for Insider Threats Using Multimodal Data (Log + Behavioural + Physical Security)

A new intelligent multilayer framework for insider threat detection

In recent years, data security incidents caused by insider threats in distributed file systems have attracted the attention of academia and industry. The most common way to detect insider threats is based on user profiles. Through analysis, we realize that based on existing user profiles are not efficient enough, and there are many false positives when a stable user profile has not yet been formed. In this work, we propose personalized user profiles and design an insider threat detection framework, which can intelligently detect insider threats for securing distributed file systems in real-time. To generate personalized user profiles, we come up with a time window-based clustering algorithm and a weighted kernel density estimation algorithm. Compared with non-personalized user profiles, both the Recall and Precision of insider threat detection based on personalized user profiles have been improved, resulting in their harmonic mean F1 increased to 96.52%. Meanwhile, to reduce the false positives of insider threat detection, we put forward operation recommendations based on user similarity to predict new operations that users will produce in the future, which can reduce the false positive rate (FPR). The FPR is reduced to 1.54% and the false positive identification rate (FPIR) is as high as 92.62%. Furthermore, to mitigate the risks caused by inaccurate authorization for users, we present user tags based on operation content and permission. The experimental results show that our proposed framework can detect insider threats more effectively and precisely, with lower FPR and high FPIR.

Organizations’ own personnel now have a greater ability than ever before to misuse their access to critical organizational assets. Insider threat detection is a key component in identifying rare anomalies in context, which is a growing concern for many organizations. Existing perimeter security mechanisms are proving to be ineffective against insider threats. As a prospective filter for the human analysts, a new deep learning based insider threat detection method that uses the Dempster-Shafer theory is proposed to handle both accidental as well as intentional insider threats via organization’s channels of communication in real time. The long short-term memory (LSTM) architecture together with multi-head attention mechanism is applied in this work to detect anomalous network behavior patterns. Furthermore, belief is updated with Dempster’s conditional rule and utilized to fuse evidence to achieve enhanced prediction. The CERT Insider Threat Dataset v6.2 is used to train the behavior model. Through performance evaluation, our proposed method is proven to be effective as an insider threat detection technique.

The concealment and confusion nature of insider threat makes it a challenging task for security analysts to identify insider threat from log data. To detect insider threat, we propose a novel gated recurrent unit (GRU) and multi-autoencoder based insider threat detection method, which is an unsupervised anomaly detection method. It takes advantage of the extremely unbalanced characteristic of insider threat data and constructs a normal behavior autoencoder with low reconfiguration error through multi-level filter behavior learning, and identifies the behavior data with high reconfiguration error as abnormal behavior. In order to achieve the high efficiency of calculation and detection, GRU and multi-head attention are introduced into the autoencoder. Use dataset v6.2 of the CERT insider threat as validation data and threat detection recall as evaluation metric. The experimental results show that the effect of the proposed method is obviously better than that of Isolation Forest, LSTM autoencoder and multi-channel autoencoders based insider threat detection methods, and it's an effective insider threat detection technology.

The rapid expansion of cyber-physical systems (CPSs) has introduced new security challenges, leading to the emergence of various threats, attacks, and controls aimed at addressing security concerns in this evolving CPS landscape. However, a noticeable gap exists in the literature, particularly in the field of insider threat detection, which lacks a systematic review of CPS security. This study aims to comprehensively review and analyse relevant studies on insider threat detection in CPS. Employing a systematic protocol, we conducted an extensive search for pertinent articles across five prominent online databases: IEEE Xplore, Web of Science, Scopus, ACM, and ScienceDirect. The selection of these indices was based on their comprehensive coverage and the distinct relevance of their contents to our research topic. The results, guided by defined inclusion and exclusion criteria, yielded a final set of 69 included articles. Notably, 39.1 % of these articles focused on insider threat detection using specification-based methods, while 27.5 % addressed cryptography methods. Machine learning methods constituted 13.04 %, and the remaining 14.5 % included review and survey studies. Insider threats pose significant challenges in global cybersecurity, necessitating effective detection systems, methods, and tools for accurate and rapid identification. This study contributes distinct observations on the insider threat detection research topic in CPS, providing valuable insights for researchers and practitioners to expedite improvements and draw significant guidelines based on this comprehensive systematic review.

This paper comprehensively examines the multifaceted motivations behind insider threats within organizations, elucidating driving forces such as financial gain, revenge, personal aspirations, ideological beliefs, coercion, and negligence. Understanding this spectrum is fundamental for crafting effective Counterintelligence strategies. The study delves into behavioral indicators crucial for identifying potential threats, emphasizing the significance of recognizing warning signs like unusual data access, unsanctioned software usage, escalated privilege requests, poor performance, disagreement with policies, and more. Furthermore, the role of Human Intelligence (HUMINT) in Counterintelligence (CI) and insider threat detection is explored, highlighting its qualitative contribution to understanding human behavior. Plus, through a hypothetical case study in the Shipping industry, the paper illustrates the direct application of HUMINT principles in fortifying security against insider threats, considering the unique challenges of this dynamic sector. The case study strategically employs employee interviews, psychological assessments, social network analysis, and trust-building initiatives to proactively identify and mitigate potential threats, in an industry reliant on seamless global supply chain operations.

Misuse or malicious access to critical assets of information systems by insiders usually causes significant loss to organizations. The issue of insider threat detection for information systems has received many researchers’ attention in both security and data mining fields, and a lot of related research results were presented. However, there are still many challenges in capturing the behavior difference between malicious insiders and normal users accurately, such as lack of labeled insider threats, the subtle and adaptive nature of insider threats, complexity, heterogeneity, sparsity of the underlying data, etc. To detect insider threats with large and complex audit data, a Multi-Edge Weight Relational Graph Neural Network method (MEWRGNN) for robust anomaly detection is proposed in this paper. Unlike most existing approaches, the MEWRGNN adopts several graph neural networks to capture the contextual relationship of user behaviors over a period of time, which is a critical factor for achieving accurate anomaly identification. The MEWRGNN achieves a certain degree of interpretability through ranking the contribution of different edge-representation features. Evaluation experimental results demonstrate that the MEWRGNN can learn a model from limited sample data sets, and achieve quick and accurate insider threat detection performance. In addition, other feature ranking results allow providing security analysts with understandable insights for investigating the detected insider threats.

The insider threat has consistently been identified as a key threat to organizations and governments. Understanding the nature of insider threats and the related threat landscape can help in forming mitigation strategies, including non-technical means. In this paper, we survey and highlight challenges associated with the identification and detection of insider threats in both public and private sector organizations, especially those part of a nation’s critical infrastructure. We explore the utility of the cyber kill chain to understand insider threats, as well as understanding the underpinning human behavior and psychological factors. The existing defense techniques are discussed and critically analyzed, and improvements are suggested, in line with the current state-of-the-art cyber security requirements. Finally, open problems related to the insider threat are identified and future research directions are discussed.

The rapid evolution of insider threats presents a significant challenge for cybersecurity systems, necessitating the development of advanced detection models capable of adapting to dynamic and sophisticated attack patterns. This chapter explores the integration of Hybrid Artificial Intelligence (AI) architectures combining Machine Learning (ML) and Deep Learning (DL) with emerging techniques such as Reinforcement Learning (RL) to enhance insider threat detection. The combination of ML and DL methodologies offers distinct advantages in handling large, complex datasets, while RL introduces the capability for real-time adaptation to evolving threat behaviors. By leveraging hybrid models, organizations can better detect subtle insider threats that traditional systems often fail to identify, improving accuracy and response times. Despite the promise, challenges such as model interpretability, data privacy, and the exploration-exploitation trade-off in RL systems must be addressed. The chapter discusses these hurdles and provides insights into how hybrid architectures can be optimized to overcome them, with a focus on scalability, security, and the ability to continuously learn from new data. Through a comprehensive review of the current research and methodologies, this chapter aims to highlight the potential of hybrid AI models in revolutionizing the way insider threats are detected and mitigated in real-time, offering a path toward more resilient and adaptive cybersecurity systems.

Insider threat represents a major cybersecurity challenge to companies and government agencies. The challenges in insider threat detection include unbalanced data, limited ground truth, and possible user behaviour changes. This research presents an unsupervised machine learning (ML) based anomaly detection approach for insider threat detection. We employ two ML methods with different working principles, specifically auto-encoder and isolation forest, and explore various representations of data with temporal information. Evaluation results show that the approach allows learning from unlabelled data under adversarial conditions for insider threat detection with a high detection and a low false positive rate. For example, 60% of malicious insiders are detected under 0.1% investigation budget. Furthermore, we explore the ability of the proposed approach to generalize for detecting unseen anomalous behaviours in different datasets, i.e. robustness. Comparisons with other work in the literature confirm the effectiveness of the proposed approach.

Insider threat detection is challenging due to the wide variety of possible attacks and the limited availability of real threat data for testing. Most previous anomaly detection studies have relied on synthetic threat data, such as the CERT insider threat dataset. However, several previous studies have used models that arguably introduce bias, such as the selective use of metrics, and reusing the same dataset with the prior knowledge of the answer labels. In this paper, we create and test a host of models following some guidelines of good conduct to produce what we believe to be a more objective comparison of these models. Our results indicate that majority voting ensembles are a simple and cost-effective way of boosting the quality of results from individual machine learning models, both on the CERT data and on a version augmented with additional attacks. We include a comparison of models with their hyperparameters optimized for different target metrics.KeywordsAnomaly detectionInsider threatMachine learningEnsembles

Insider threat represents a major cybersecurity challenge to companies, organizations, and government agencies. Insider threat detection involves many challenges, including unbalanced data, limited ground truth, and possible user behavior changes. This research presents an unsupervised learning based anomaly detection approach for insider threat detection. We employ four unsupervised learning methods with different working principles, and explore various representations of data with temporal information. Furthermore, different computational intelligence schemes are explored to combine these models to create anomaly detection ensembles for improving the detection performance. Evaluation results show that the approach allows learning from unlabelled data under challenging conditions for insider threat detection. Insider threats are detected with high detection and low false positive rates. For example, 60% of malicious insiders are detected under 0.1% investigation budget, and all malicious insiders are detected at less than 5% investigation budget. Furthermore, we explore the ability of the proposed approach to generalize for detecting new anomalous behaviors in different datasets, i.e., robustness. Finally, results demonstrate that a voting-based ensemble of anomaly detection can be used to improve detection performance as well as the robustness. Comparisons with the state-of-the-art confirm the effectiveness of the proposed approach.

Enhancing false negative and positive rates for efficient insider threat detection

Insider threats pose significant risks to enterprise systems, with malicious or negligent actions by authorized users leading to data breaches and financial losses. Traditional detection methods, such as rule-based systems and anomaly detection, struggle to capture the complex relationships and dynamic behaviors in enterprise networks. This research explores Graph Neural Networks (GNNs) to model enterprise systems as graphs, leveraging their ability to capture structural and relational patterns for enhanced insider threat detection. Using the CERT Insider Threat Dataset (r6.2), a comprehensive dataset simulating enterprise activities, this research develops a GNN-based framework to identify anomalous behaviors. The proposed model achieves superior performance compared to traditional machine learning approaches, with an accuracy of 96.8%, precision of 0.94, and recall of 0.93. Visualizations, including bar graphs and confusion matrices, illustrate the model’s effectiveness. This research contributes to enterprise security by providing a scalable, interpretable, and efficient solution for insider threat detection, with implications for real-time monitoring and forensic analysis.

The purpose of this chapter is to introduce the insider threat and discuss methods for preventing, detecting, and responding to the threat. Trusted insiders present one of the most significant risks to an organization. They possess elevated privileges when compared to external users, have knowledge about technical and non-technical control measures, and potentially can bypass security measures designed to prevent, detect, or react to unauthorized access. In this chapter, we define the insider threat and summarize various case studies of insider attacks in order to highlight the severity of the problem. We then discuss best practices for preventing, detecting, and mitigating insider attacks, to include application of risk management principles specific to the insider threat. Finally, we provide a survey of ongoing research into detecting irregular activities that are potentially harmful to an organization.