GRU and Multi-autoencoder based Insider Threat Detection for Cyber Security

A new intelligent multilayer framework for insider threat detection

Organizations’ own personnel now have a greater ability than ever before to misuse their access to critical organizational assets. Insider threat detection is a key component in identifying rare anomalies in context, which is a growing concern for many organizations. Existing perimeter security mechanisms are proving to be ineffective against insider threats. As a prospective filter for the human analysts, a new deep learning based insider threat detection method that uses the Dempster-Shafer theory is proposed to handle both accidental as well as intentional insider threats via organization’s channels of communication in real time. The long short-term memory (LSTM) architecture together with multi-head attention mechanism is applied in this work to detect anomalous network behavior patterns. Furthermore, belief is updated with Dempster’s conditional rule and utilized to fuse evidence to achieve enhanced prediction. The CERT Insider Threat Dataset v6.2 is used to train the behavior model. Through performance evaluation, our proposed method is proven to be effective as an insider threat detection technique.

In recent years, data security incidents caused by insider threats in distributed file systems have attracted the attention of academia and industry. The most common way to detect insider threats is based on user profiles. Through analysis, we realize that based on existing user profiles are not efficient enough, and there are many false positives when a stable user profile has not yet been formed. In this work, we propose personalized user profiles and design an insider threat detection framework, which can intelligently detect insider threats for securing distributed file systems in real-time. To generate personalized user profiles, we come up with a time window-based clustering algorithm and a weighted kernel density estimation algorithm. Compared with non-personalized user profiles, both the Recall and Precision of insider threat detection based on personalized user profiles have been improved, resulting in their harmonic mean F1 increased to 96.52%. Meanwhile, to reduce the false positives of insider threat detection, we put forward operation recommendations based on user similarity to predict new operations that users will produce in the future, which can reduce the false positive rate (FPR). The FPR is reduced to 1.54% and the false positive identification rate (FPIR) is as high as 92.62%. Furthermore, to mitigate the risks caused by inaccurate authorization for users, we present user tags based on operation content and permission. The experimental results show that our proposed framework can detect insider threats more effectively and precisely, with lower FPR and high FPIR.

The insider threat has consistently been identified as a key threat to organizations and governments. Understanding the nature of insider threats and the related threat landscape can help in forming mitigation strategies, including non-technical means. In this paper, we survey and highlight challenges associated with the identification and detection of insider threats in both public and private sector organizations, especially those part of a nation’s critical infrastructure. We explore the utility of the cyber kill chain to understand insider threats, as well as understanding the underpinning human behavior and psychological factors. The existing defense techniques are discussed and critically analyzed, and improvements are suggested, in line with the current state-of-the-art cyber security requirements. Finally, open problems related to the insider threat are identified and future research directions are discussed.

The rapid expansion of cyber-physical systems (CPSs) has introduced new security challenges, leading to the emergence of various threats, attacks, and controls aimed at addressing security concerns in this evolving CPS landscape. However, a noticeable gap exists in the literature, particularly in the field of insider threat detection, which lacks a systematic review of CPS security. This study aims to comprehensively review and analyse relevant studies on insider threat detection in CPS. Employing a systematic protocol, we conducted an extensive search for pertinent articles across five prominent online databases: IEEE Xplore, Web of Science, Scopus, ACM, and ScienceDirect. The selection of these indices was based on their comprehensive coverage and the distinct relevance of their contents to our research topic. The results, guided by defined inclusion and exclusion criteria, yielded a final set of 69 included articles. Notably, 39.1 % of these articles focused on insider threat detection using specification-based methods, while 27.5 % addressed cryptography methods. Machine learning methods constituted 13.04 %, and the remaining 14.5 % included review and survey studies. Insider threats pose significant challenges in global cybersecurity, necessitating effective detection systems, methods, and tools for accurate and rapid identification. This study contributes distinct observations on the insider threat detection research topic in CPS, providing valuable insights for researchers and practitioners to expedite improvements and draw significant guidelines based on this comprehensive systematic review.

Misuse or malicious access to critical assets of information systems by insiders usually causes significant loss to organizations. The issue of insider threat detection for information systems has received many researchers’ attention in both security and data mining fields, and a lot of related research results were presented. However, there are still many challenges in capturing the behavior difference between malicious insiders and normal users accurately, such as lack of labeled insider threats, the subtle and adaptive nature of insider threats, complexity, heterogeneity, sparsity of the underlying data, etc. To detect insider threats with large and complex audit data, a Multi-Edge Weight Relational Graph Neural Network method (MEWRGNN) for robust anomaly detection is proposed in this paper. Unlike most existing approaches, the MEWRGNN adopts several graph neural networks to capture the contextual relationship of user behaviors over a period of time, which is a critical factor for achieving accurate anomaly identification. The MEWRGNN achieves a certain degree of interpretability through ranking the contribution of different edge-representation features. Evaluation experimental results demonstrate that the MEWRGNN can learn a model from limited sample data sets, and achieve quick and accurate insider threat detection performance. In addition, other feature ranking results allow providing security analysts with understandable insights for investigating the detected insider threats.

Malicious insider activities threaten various govern-ment agencies and private organizations. This paper presents a novel approach to detect malicious behaviors. We propose the use of a granularity level to represent users' log data: textual session-based data samples. The user's behaviors are modeled using character embeddings and a deep learning model that consists of CNN and LSTM. Character embeddings are used to represent the input samples. Then, a convolution layer is used to capture local tri-gram features from the input samples, followed by an LSTM layer to consider the order of these given features (tri-grams). We conduct experiments using several variations of model architectures with no handcrafted features. The proposed model is evaluated with a subset of CERT Insider Threat dataset, r4.2. The result shows that performance improved with high precision and recall values.

Insider threat represents a major cybersecurity challenge to companies, organizations, and government agencies. Insider threat detection involves many challenges, including unbalanced data, limited ground truth, and possible user behavior changes. This research presents an unsupervised learning based anomaly detection approach for insider threat detection. We employ four unsupervised learning methods with different working principles, and explore various representations of data with temporal information. Furthermore, different computational intelligence schemes are explored to combine these models to create anomaly detection ensembles for improving the detection performance. Evaluation results show that the approach allows learning from unlabelled data under challenging conditions for insider threat detection. Insider threats are detected with high detection and low false positive rates. For example, 60% of malicious insiders are detected under 0.1% investigation budget, and all malicious insiders are detected at less than 5% investigation budget. Furthermore, we explore the ability of the proposed approach to generalize for detecting new anomalous behaviors in different datasets, i.e., robustness. Finally, results demonstrate that a voting-based ensemble of anomaly detection can be used to improve detection performance as well as the robustness. Comparisons with the state-of-the-art confirm the effectiveness of the proposed approach.

Insider threats are one of the most costly and difficult types of attacks to detect due to the fact that insiders have the right to access an organization’s network systems and understand its structure and security procedures, making it difficult to detect this type of behavior through traditional behavioral auditing. This paper proposes a method to leverage unsupervised outlier scores to enhance supervised insider threat detection by integrating the advantages of supervised and unsupervised learning methods and using multiple unsupervised outlier mining algorithms to extract from the underlying data useful representations, thereby enhancing the predictive power of supervised classifiers on the enhanced feature space. This novel approach provides superior performance, and our method provides better predictive power compared to other excellent abnormal detection methods. Using only 20% of the computing budget, our method achieved an accuracy of 86.12%. Compared with other anomaly detection methods, the accuracy increased by up to 12.5% under the same computing budget.

Enhancing false negative and positive rates for efficient insider threat detection

The purpose of this chapter is to introduce the insider threat and discuss methods for preventing, detecting, and responding to the threat. Trusted insiders present one of the most significant risks to an organization. They possess elevated privileges when compared to external users, have knowledge about technical and non-technical control measures, and potentially can bypass security measures designed to prevent, detect, or react to unauthorized access. In this chapter, we define the insider threat and summarize various case studies of insider attacks in order to highlight the severity of the problem. We then discuss best practices for preventing, detecting, and mitigating insider attacks, to include application of risk management principles specific to the insider threat. Finally, we provide a survey of ongoing research into detecting irregular activities that are potentially harmful to an organization.

This paper comprehensively examines the multifaceted motivations behind insider threats within organizations, elucidating driving forces such as financial gain, revenge, personal aspirations, ideological beliefs, coercion, and negligence. Understanding this spectrum is fundamental for crafting effective Counterintelligence strategies. The study delves into behavioral indicators crucial for identifying potential threats, emphasizing the significance of recognizing warning signs like unusual data access, unsanctioned software usage, escalated privilege requests, poor performance, disagreement with policies, and more. Furthermore, the role of Human Intelligence (HUMINT) in Counterintelligence (CI) and insider threat detection is explored, highlighting its qualitative contribution to understanding human behavior. Plus, through a hypothetical case study in the Shipping industry, the paper illustrates the direct application of HUMINT principles in fortifying security against insider threats, considering the unique challenges of this dynamic sector. The case study strategically employs employee interviews, psychological assessments, social network analysis, and trust-building initiatives to proactively identify and mitigate potential threats, in an industry reliant on seamless global supply chain operations.

The rapid evolution of insider threats presents a significant challenge for cybersecurity systems, necessitating the development of advanced detection models capable of adapting to dynamic and sophisticated attack patterns. This chapter explores the integration of Hybrid Artificial Intelligence (AI) architectures combining Machine Learning (ML) and Deep Learning (DL) with emerging techniques such as Reinforcement Learning (RL) to enhance insider threat detection. The combination of ML and DL methodologies offers distinct advantages in handling large, complex datasets, while RL introduces the capability for real-time adaptation to evolving threat behaviors. By leveraging hybrid models, organizations can better detect subtle insider threats that traditional systems often fail to identify, improving accuracy and response times. Despite the promise, challenges such as model interpretability, data privacy, and the exploration-exploitation trade-off in RL systems must be addressed. The chapter discusses these hurdles and provides insights into how hybrid architectures can be optimized to overcome them, with a focus on scalability, security, and the ability to continuously learn from new data. Through a comprehensive review of the current research and methodologies, this chapter aims to highlight the potential of hybrid AI models in revolutionizing the way insider threats are detected and mitigated in real-time, offering a path toward more resilient and adaptive cybersecurity systems.

This paper investigates anomaly detection techniques that have been successful for detecting external threats and applies them to the insider threat problem. The "insider threat" involves the actions of a trusted and privileged user who is inappropriately accessing or disseminating sensitive information or otherwise compromising information systems. In contrast, the "external threat" involves the actions of an outsider attempting to compromise or gain access to the information systems. Although approaches for automatically detecting external threat instances have been quite successful (i.e., intrusion detection systems), there is very little similar work for the insider threat. In the past, anomaly detection systems have proven useful for detecting external threat. Anomaly detection at the system call level offers a high degree of information assurance in terms of tamper-resistance and system activity coverage. Therefore, we investigate three system-call-based feature representations: n-grams of system call names, histograms of system call names, and individual system calls with associated parameters. We find that none of these representations consistently performs as well when dealing with the internal threat as previous results show for external threat detection. However, parameter-based features for certain system calls do show some sensitivity to detecting the insider threat, and we plan to explore and enhance this sensitivity in future work.

Insider threats pose significant risks to enterprise systems, with malicious or negligent actions by authorized users leading to data breaches and financial losses. Traditional detection methods, such as rule-based systems and anomaly detection, struggle to capture the complex relationships and dynamic behaviors in enterprise networks. This research explores Graph Neural Networks (GNNs) to model enterprise systems as graphs, leveraging their ability to capture structural and relational patterns for enhanced insider threat detection. Using the CERT Insider Threat Dataset (r6.2), a comprehensive dataset simulating enterprise activities, this research develops a GNN-based framework to identify anomalous behaviors. The proposed model achieves superior performance compared to traditional machine learning approaches, with an accuracy of 96.8%, precision of 0.94, and recall of 0.93. Visualizations, including bar graphs and confusion matrices, illustrate the model’s effectiveness. This research contributes to enterprise security by providing a scalable, interpretable, and efficient solution for insider threat detection, with implications for real-time monitoring and forensic analysis.