Detection of advanced persistent threat using machine-learning correlation analysis

Advanced Persistent Threat (APT) has become one of the most complicated and intractable cyber attack over the last decade. As APT attacks are conducted through series of actions that comprise social engineering, phishing, command and control servers, and remote desktop control, conventional anti-virus mechanisms become insufficient because they were designed to cope with traditional stand-alone malware attacks. Furthermore, data transmission from the compromised network to the APT actors is usually well disguised and embedded in normal transmission, exacerbating the detection of APT attacks to the point that even major anti-virus firms are not sure about the ratio of discovered APT attacks against real attacks. To make things worse, APT actors tend to be well-organized and potentially government-funded groups of hackers and professionals who are capable of developing and maintaining malware specifically made for their own purposes and interpret the stolen data. While most efforts in defending against APT attacks focus on related technologies, this research argues the importance of constructing a holistic understanding by analyzing the behaviors and changes of ATP attacks and actors. This research aims to understand the evolution of technologies and malware on the one hand and the behavioral changes of attacking groups. By doing so, this research is expected to contribute to constructing a clearer roadmap of APT attacks and actors that cyber security providers can use as reference.

This paper analyzes the process of Google Aurora attack, summarizes the typical steps of APT attacks. By analyzing the defects on traditional enterprise security architecture, this paper proposes a new security solution with a central analysis and control module. The module adopts a detection mode based on the storage, and it is divided into storage layer, analysis layer and presentation layer.

In recent years, extensive research has been conducted in Advanced Persistent Threat (APT) attack defence. However, most existing defence solutions can only identify and temporarily disrupt cyber attacks, seeking to deny the threat from the intranet, it’s difficult to defence against APT attacks. Attributing the APT organization is an excellent complement to the existing defence solutions, which not only can expose the attacker’s true identity, but also provide evidence to bring the attacker to justice. However, research on attributing APT Organization is still few, poses complex tasks because APT attacks are highly targeted, stealthy, persistent and organized. To answer thie question, we propose a Particle Swarm Optimization Multiclass Support Vector Machine (PSO-MSVM) approach to identify the organization behind complex APT attacks automatically. Firstly, we have collected a large amount of data on the traces of APT attack tools executed in the sandbox, and selected data closely related to APT organizations to construct the feature set. Secondly, based on the strategy of keeping the personal best (pbest) and global best (gbest) particles in the particle swarm algorithm away from the adaptation values generated by the misclassification information as they move, the particle positions are updated frequently to eventually obtain the optimal parameters (i.e., penalty parameter ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${C}$ </tex-math></inline-formula> ) and sigma parameter ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\sigma $ </tex-math></inline-formula> )) for MSVM, thus enabling the MSVM technique to accurately identify APT organizations. The results obtained from the PSO-MSVM approach showed the superiority of this technique in three different measures of accuracy, precision and F1,compared with other six classical methods.

Advanced Persistent Threat (APT) attacks have caused serious security threats and financial losses worldwide. Various real-time detection mechanisms that combine context information and provenance graphs have been proposed to defend against APT attacks. However, existing real-time APT detection mechanisms suffer from accuracy and efficiency issues due to inaccurate detection models and the growing size of provenance graphs. To address the accuracy issue, we propose a novel and accurate APT detection model that removes unnecessary phases and focuses on the remaining ones with improved definitions. To address the efficiency issue, we propose a state-based framework in which events are consumed as streams and each entity is represented in an FSA-like structure without storing historic data. Additionally, we reconstruct attack scenarios by storing just one in a thousand events in a database. Finally, we implement our design, called <small>Conan</small>, on Windows and conduct comprehensive experiments under real-world scenarios to show that <small>Conan</small> can accurately and efficiently detect all attacks within our evaluation. The memory usage and CPU efficiency of <small>Conan</small> remain constant over time (1-10 MB of memory and hundreds of times faster than data generation), making <small>Conan</small> a practical design for detecting both known and unknown APT attacks in real-world scenarios.

Advanced Persistent Threat (APT) attack detection and monitoring has attracted a lot of attention recently when this type of cyber-attacks is growing in both number and dangerous levels. In this paper, a new APT attack model, which is the combination of three different neural network layers including: Multi-layer Perceptron (MLP), Inference (I), and Graph Convolutional Networks (GCN) is proposed. The new model is named MIG for short. In this model, the MLP layer is in charge of aggregating and extracting properties of the IPs based on flow network in Network traffic, while the Inference layer is responsible for building IP information profiles by grouping and concatenating flow networks generated from the same IP. Finally, the GCN layer is used for analyzing and reconstructing IP features based on the behavior extraction process from IP information records. The APT attacks detection method based on network traffic using this MIG model is new, and has yet been proposed and applied anywhere. The novelty and uniqueness of this method is the combination of many different data mining techniques in order to calculate, extract and represent the relationship and the correlation between APT attack behaviors based on Network traffic. In MIG model, many meaningful anomalous properties and behaviors of APT attacks are synthesized and extracted, which help improve the performance of APT attack detection. The experimental results showed that the proposed method is meaningful in both theory and practice since the MIG model not only improves the ability to correctly detect APT attacks in network traffic but also minimizes false alarms.

Cyber security has become a matter of a global interest, and several attacks target industrial companies and governmental organizations. The advanced persistent threats (APTs) have emerged as a new and complex version of multi-stage attacks (MSAs), targeting selected companies and organizations. Current APT detection systems focus on raising the detection alerts rather than predicting APTs. Forecasting the APT stages not only reveals the APT life cycle in its early stages but also helps to understand the attacker’s strategies and aims. This paper proposes a novel intrusion detection system for APT detection and prediction. This system undergoes two main phases; the first one achieves the attack scenario reconstruction. This phase has a correlation framework to link the elementary alerts that belong to the same APT campaign. The correlation is based on matching the attributes of the elementary alerts that are generated over a configurable time window. The second phase of the proposed system is the attack decoding. This phase utilizes the hidden Markov model (HMM) to determine the most likely sequence of APT stages for a given sequence of correlated alerts. Moreover, a prediction algorithm is developed to predict the next step of the APT campaign after computing the probability of each APT stage to be the next step of the attacker. The proposed approach estimates the sequence of APT stages with a prediction accuracy of at least 91.80%. In addition, it predicts the next step of the APT campaign with an accuracy of 66.50%, 92.70%, and 100% based on two, three, and four correlated alerts, respectively.

Advanced persistent threat (APT) attack is a malicious attack type which has intentional and clear targets. This attack technique has become a challenge for information security systems of organizations, governments, and businesses. The approaches of using machine learning or deep learning algorithms to analyze signs and abnormal behaviors of network traffic for detecting and preventing APT attacks have become popular in recent years. However, the APT attack detection approach that uses behavior analysis and evaluation techniques is facing many difficulties due to the lack of typical data of attack campaigns. To handle this situation, recent studies have selected and extracted the APT attack behaviors which based on datasets are built from experimental tools. Consequently, these properties are few and difficult to obtain in practical monitoring systems. Therefore, although the experimental results show good detection, it does not bring high efficiency in practice. For above reasons, in this paper, a new method based on network traffic analysis using a combined deep learning model to detect APT attacks will be proposed. Specifically, individual deep learning networks such as multilayer perceptron (MLP), convolutional neural network (CNN), and long short-term memory (LSTM) will also be sought, built and linked into combined deep learning networks to analyze and detect signs of APT attacks in network traffic. To detect APT attack signals, the combined deep learning models are performed in two main stages including (i) extracting IP features based on flow: In this phase, we will analyze network traffic into networking flows by IP address and then use the combined deep learning models to extract IP features by network flow; (ii) classifying APT attack IPs: Based on IP features extracted in a task (i), the APT attack IPs and normal IPs will be identified and classified. The proposal of a combined deep learning model to detect APT attacks based on network traffic is a new approach, and there is no research proposed and applied yet. In the experimental section, combined deep learning models proved their superior abilities to ensure accuracy on all measurements from 93 to 98%. This is a very good result for APT attack detection based on network traffic.

With the wide use of Cyber-Physical Systems (CPS) in many applications, targets of advanced persistent threats (APTs) have been extended to the IoT and industrial control systems. Provenance graph analysis based on system audit logs has become a promising way for APT detection and investigation. However, we cannot afford to ignore that existing provenance-based APT detection systems lack the process–context information at system runtime, which seriously limits detection performance. In this paper, we proposed ConGraph, an approach for detecting APT attacks using provenance graphs combined with process context; we presented a module for collecting process context to detect APT attacks. This module collects file access behavior, network access behavior, and interactive relationship features of processes to enrich semantic information of the provenance graph. It was the first time that the provenance graph was combined with multiple process–context information to improve the detection performance of APT attacks. ConGraph extracts process activity features from the provenance graphs and submits the features to a CNN-BiLSTM model to detect underlying APT activities. Compared to some state-of-the-art models, our model raised the average precision rate, recall rate, and F-1 score by 13.12%, 25.61%, and 24.28%, respectively.

Advanced Persistent Threat (APT) attacks are a form of malicious, intentionally and clearly targeted attack. This attack technique is growing in both the number of recorded attacks and the extent of its dangers to organizations, businesses and governments. Therefore, the task of detecting and warning APT attacks in the real system is very necessary today. One of the most effective approaches to APT attack detection is to apply machine learning or deep learning to analyze network traffic. There have been a number of studies and recommendations to analyze network traffic into network flows and then combine with some classification or clustering methods to look for signs of APT attacks. In particular, recent studies often apply machine learning algorithms to spot the present of APT attacks based on network flow. In this paper, a new method based on deep learning to detect APT attacks using network flow is proposed. Accordingly, in our research, network traffic is analyzed into IP-based network flows, then the IP information is reconstructed from flow, and finally deep learning models are used to extract features for detecting APT attack IPs from other IPs. Additionally, a combined deep learning model using Bidirectional Long Short-Term Memory (BiLSTM) and Graph Convolutional Networks (GCN) is introduced. The new detection model is evaluated and compared with some traditional machine learning models, i.e. Multi-layer perceptron (MLP) and single GCN models, in the experiments. Experimental results show that BiLSTM-GCN model has the best performance in all evaluation scores. This not only shows that deep learning application on flow network analysis to detect APT attacks is a good decision but also suggests a new direction for network intrusion detection techniques based on deep learning.

Designing systems that can infiltrate networks, remain undetected for long periods, and extract sensitive information make Advanced Persistent Threats (APTs) some of the most sophisticated and powerful cyber-attacks. Most APTs circumvent conventional Intrusion Detection Systems (IDS) because of signature-based detection technology that relies on static patterns and is blind to new forms of threats. To bridge this gap, we put forth a comprehensive framework that applies machine learning through the incorporation of Random Forest, Isolation Forest, and Long Short-Term Memory (LSTM) networks aimed at APT detection. Random Forest, as a supervised learning algorithm, will extract known attack patterns while Isolation Forest, as an unsupervised algorithm, will capture anomalous network behavior corresponding to unknown attacks. With LSTM networks, complex multivariate network traffic patterns are analyzed to identify complex attack behaviors. The UNSW-NB15 dataset, which contains various attack scenarios as well as normal network traffic, is used for training and testing the models. The combination of these technologies is deployed on a scalable serverless infrastructure through A WS SageMaker and real-time data streams are processed using A WS Kinesis. The experimental findings show substantial improvements in accuracy, precision, and recall in both batch and real-time detection scenarios. This approach provides a comprehensive and flexible solution to counter evolving cyber threats with enhanced network security.

Detecting mobile advanced persistent threats based on large-scale DNS logs

Nowadays, countries face a multitude of electronic threats that have permeated almost all business sectors, be it private corporations or public institutions. Among these threats, advanced persistent threats (APTs) stand out as a well-known example. APTs are highly sophisticated and stealthy computer network attacks meticulously designed to gain unauthorized access and persist undetected threats within targeted networks for extended periods. They represent a formidable cybersecurity challenge for governments, corporations, and individuals alike. Recognizing the gravity of APTs as one of the most critical cybersecurity threats, this study aims to reach a deeper understanding of their nature and propose a multi-stage framework for automated APT detection leveraging time series data. Unlike previous models, the proposed approach has the capability to detect real-time attacks based on stored attack scenarios. This study conducts an extensive review of existing research, identifying its strengths, weaknesses, and opportunities for improvement. Furthermore, standardized techniques have been enhanced to enhance their effectiveness in detecting APT attacks. The learning process relies on datasets sourced from various channels, including journal logs, traceability audits, and systems monitoring statistics. Subsequently, an efficient APT detection and prevention system, known as the composition-based decision tree (CDT), has been developed to operate in complex environments. The obtained results demonstrate that the proposed approach consistently outperforms existing algorithms in terms of detection accuracy and effectiveess.

Advanced Persistent Threat (APT), unlike traditional hacking attempts, carries out specific attacks on a specific target to illegally collect information and data from it. These targeted attacks use special-crafted malware and infrequent activity to avoid detection, so that hackers can retain control over target systems unnoticed for long periods of time. In order to detect these stealthy activities, a large-volume of traffic data generated in a period of time has to be analyzed. We proposed a scalable solution, Ctracer to detect stealthy command and control channel in a large-volume of traffic data. APT uses multiple command and control (C&C) channel and change them frequently to avoid detection, but there are common signatures in those C&C sessions. By identifying common network signature, Ctracer is able to group the C&C sessions. Therefore, we can detect an APT and all the C&C session used in an APT attack. The Ctracer is evaluated in a large enterprise for four months, twenty C&C servers, three APT attacks are reported. After investigated by the enterprise's Security Operations Center (SOC), the forensic report shows that there is specific enterprise targeted APT cases and not ever discovered for over 120 days.

Malware homology analysis aims at detecting whether different malicious code originates from the same set of malicious code or is written by the same author or team, and whether it has intrinsic relevance and similarity. At the same time, the homology analysis of malicious code is also an important part of studying the groups behind different APT (Advanced Persistent Threat) attacks. At present, homology identification still relies on manual analysis and security experts' experience in the anti-malware industry. In addition, research on large-scale malicious code automated homology analysis is still insufficient. The method proposed in this paper is to solve the problem of large-scale malicious code homology automatic analysis, and hope to provide auxiliary information for discovering the group behind the APT attack. In this paper, we collected samples of different APT groups from public threat intelligence and proposed a novel approach to classify these samples into different APT groups to further analyze the homology of malware. We combined the CFG (Control Flow Graph) of the malicious code function and the disassembled code of the stripped malware to generate the embedding, i.e., a numeric vector, which formed a function feature database of the APT group, and presented a neural network model used for APT group classification. We have implemented our approach in a prototype system called MCrab. Our extensive evaluation showed that MCrab could produce high accuracy results, with few to no false positives. Our research also showed that deep learning can be successfully applied to malware homology analysis.

For years governments, organizations and companies have made great efforts to keep hackers, malware, cyber attacks at bay with different degrees of success. On the other hand, cyber criminals and miscreants produced more advanced techniques to compromise Internet infrastructure. Targeted attack or advanced persistent threat (APT) attack is a new challenge and aims to accomplish a specific goal, most often espionage. APTs are presently the biggest threat to governments and organizations. This paper states research questions and propose a novel approach to intrusion detection system processes network traffic and able to detect potential APT attack. This detection of APT attack is based on the correlation between the events which we get as outputs of our detection methods. Each detection method aims to detect one technique used in one of APT attack steps.