Efficient Dynamic Malware Analysis for Collecting HTTP Requests using Deep Learning

One of the numerous ways of addressing the Android malware threat is to run malicious applications in a sandbox environment while monitoring metrics. However, dynamic malware analysis is usually concerned with a one-time execution of an application, and information about behaviour in different environments is lacking in the literature. We fill this gap with a fuzzy-like approach to the problem: by running the same malware multiple times in different environments, we gain insight on the malware behaviour and his peculiarities. To implement this approach, we leverage a client-server sandbox to run experiments, based on a common suit of actions. Scenarios are executed multiple times on a malware sample, each time with a different parameter, and results are compared to determine variation in observed behaviour. In our current experiment, variation was introduced by different levels of simulation, allowing us to compare metrics such as failure rate, data leakages, sending of SMS, and the number of HTTP and DNS requests. We find the behaviour is different for data leakages, which require no simulation to leak information, while all results for other metrics were higher when simulation was used in experiments. We expect that a fuzzing approach with others parameters will further our understanding of malware behaviour, particularly for malware bound to such parameters.

Malware authors or attackers always try to evade detection methods to accomplish their mission. Such detection methods are broadly divided into three types: static feature, host-behavior, and network-behavior based. Static feature-based methods are evaded using packing techniques. Host- behavior-based methods also can be evaded using some code injection methods, such as API hook and dynamic link library hook. This arms race regarding static feature-based and host-behavior- based methods increases the importance of network-behavior-based methods. The necessity of communication between infected hosts and attackers makes it difficult to evade network-behavior- based methods. The effectiveness of such methods depends on how we collect a variety of communications by using malware samples. However, analyzing all new malware samples for a long period is infeasible. Therefore, we propose a method for determining whether dynamic analysis should be suspended based on network behavior to collect malware communications efficiently and exhaustively. The key idea behind our proposed method is focused on two characteristics of malware communication: the change in the communication purpose and the common latent function. These characteristics of malware communications resemble those of natural language from the viewpoint of data structure, and sophisticated analysis methods have been proposed in the field of natural language processing. For this reason, we applied the recursive neural network, which has recently exhibited high classification performance, to our proposed method. In the evaluation with 29,562 malware samples, our proposed method reduced 67.1% of analysis time while keeping the coverage of collected URLs to 97.9% of the method that continues full analyses.

Malware is one of the biggest security threats on the Internet today and deploying effective defensive solutions requires the rapid analysis of a continuously increasing number of malware samples. With the proliferation of metamorphic malware the analysis is further complicated as the efficacy of signature-based static analysis systems is greatly reduced. While dynamic malware analysis is an effective alternative, the approach faces significant challenges as the ever increasing number of samples requiring analysis places a burden on hardware resources. At the same time modern malware can both detect the monitoring environment and hide in unmonitored corners of the system.In this paper we present DRAKVUF, a novel dynamic malware analysis system designed to address these challenges by building on the latest hardware virtualization extensions and the Xen hypervisor. We present a technique for improving stealth by initiating the execution of malware samples without leaving any trace in the analysis machine. We also present novel techniques to eliminate blind-spots created by kernel-mode rootkits by extending the scope of monitoring to include kernel internal functions, and to monitor file-system accesses through the kernel's heap allocations. With extensive tests performed on recent malware samples we show that DRAKVUF achieves significant improvements in conserving hardware resources while providing a stealthy, in-depth view into the behavior of modern malware.

The rapid development of the Internet of Things (IoT) has prompted a recent interest into realistic IoT network traffic generation. Security practitioners need IoT network traffic data to develop and assess network-based intrusion detection systems (NIDS). Emulating realistic network traffic will avoid the costly physical deployment of thousands of smart devices. From an attacker's perspective, generating network traffic that mimics the legitimate behavior of a device can be useful to evade NIDS. As network traffic data consist of sequences of packets, the problem is similar to the generation of sequences of categorical data, like word by word text generation. Many solutions in the field of natural language processing have been proposed to adapt a Generative Adversarial Network (GAN) to generate sequences of categorical data. In this paper, we propose to combine an autoencoder with a GAN to generate sequences of packet sizes that correspond to bidirectional flows. First, the autoencoder is trained to learn a latent representation of the real sequences of packet sizes. A GAN is then trained on the latent space, to learn to generate latent vectors that can be decoded into realistic sequences. For experimental purposes, bidirectional flows produced by a Google Home Mini are used, and the autoencoder is combined with a Wassertein GAN. Comparison of different network characteristics shows that our proposed approach is able to generate sequences of packet sizes that behave closely to real bidirectional flows. We also show that the synthetic bidirectional flows are close enough to the real ones that they can fool anomaly detectors into labeling them as legitimate.

Many research projects studying security threats require realistic network scenarios while dealing with millions of cyber threats (e.g., exploit programs and malware). For instance, studying the execution of malware may require to take into account different network configurations in which malware can propagate, as well as dealing with thousands (or millions) of different malware samples. The same challenge occurs if one wants to evaluate IDSs, study exploit programs or conduct vulnerability assessment using realistic network scenarios.Moreover, cyber threats are highly dynamic. Every day, new vulnerabilities are identified and documented in software commonly used by computers connected to the Internet, and new malware instances and exploit programs are also identified. Consequently, it is not viable to develop (deploy) an environment every time cyber threats or security products (e.g., IDSs and anti-virus) have to be studied from a different perspective.New research methodologies and tools are needed to systematically conduct cyber security research. Automation is required to deal with the increasingly large number of cyber threats. In this paper, we draw the foundations of an experimental approach to cyber security by providing the blueprints of a lightweight Automated Experimentation System (AES). The AES automatically executes experiments based on a specification file. We describe its usage in different network security research projects, from IDS evaluation to static and dynamic malware analysis. The results we derived from these different research projects show that our experimental approach to cyber security, enabled by the AES, enhances the scope (and scale) of research in this field. Consequently, the AES improves our understanding of cyber threats and our assessment of the current state of security products.

With the advent of mobile era, mobile terminals are going through a trend of surpassing PC to become the most popular computing device. Meanwhile, the hackers and viruswriters are paying close attention to the mobile terminals, especially the Android platform. The growing of malwares on the Android system has drawn attentions from both the academia and security industry. Recently, mobile network traffic analysis has been used to identify the malware. But due to the lack of a large-scale malware repository and a systematic analysis of network traffic features, the existing research mostly remain in theory. In this paper, we design an Android malware traffic behavior monitoring scheme to capture traffic data generated by malware samples in a real Internet environment. We capture the network traffic from 5560 malware samples in the first 5 minutes, and analyze the major compositions of the traffic data. We discover that HTTP and DNS traffic are accounted for more than 99% on the application layer traffic. We then present an analysis of related network features: DNS query, HTTP packet length, ratio of downlink to uplink traffic amount, HTTP request and Ad traffic feature. Our statistical results illustrate that: (1) more than 70% malwares generate malicious traffic in the first 5 minutes, (2) DNS query and HTTP request can be used to identify the malware, and the detection rate reaches 69.55% and 40.89% respectively, (3) Ad traffic can greatly affect the malware detection. We believe our research provides an in-depth analysis into mobile malwares' network behaviors.

With the increase in the popularity of mobile devices, malicious applications targeting Android platform have greatly increased. Malware is coded so prudently that it has become very complicated to identify. The increase in the large amount of malware every day has made the manual approaches inadequate for detecting the malware. Nowadays, a new malware is characterized by sophisticated and complex obfuscation techniques. Thus, the static malware analysis alone is not enough for detecting it. However, dynamic malware analysis is appropriate to tackle evasion techniques but incapable to investigate all the execution paths and also it is very time consuming. So, for better detection and classification of Android malware, we propose a hybrid approach which integrates the features obtained after performing static and dynamic malware analysis. This approach tackles the problem of analyzing, detecting and classifying the Android malware in a more efficient manner. In this paper, we have used a robust set of features from static and dynamic malware analysis for creating two datasets i.e. binary and multiclass (family) classification datasets. These are made publically available on GitHub and Kaggle with the aim to help researchers and anti-malware tool creators for enhancing or developing new techniques and tools for detecting and classifying Android malware. Various machine learning algorithms are employed to detect and classify malware using the features extracted after performing static and dynamic malware analysis. The experimental outcomes indicate that hybrid approach enhances the accuracy of detection and classification of Android malware as compared to the case when static and dynamic features are considered alone.

Dynamic malware analysis is essential to develop optimal intrusion detection systems. Performing this process from a virtual environment provides the advantage of containing the malware damage, so the real machine is not affected. However, it is suspected that the behavior of malware may vary when it is being executed in a virtual environment thus invalidating the results obtained when analyzing malware in these environments. In this work we dynamically analyze the behavior of nine different malware samples based on the network traffic generated by each of them. Then, this behavior was compared with ones previously registered from a physical environment. The results obtained show that malware behavior differs significantly in these two environments.

To handle the growing flood of malware, security vendors and analysts rely on tools that automatically identify and analyze malicious code. Current systems for automated malware analysis typically follow a dynamic approach, executing an unknown program in a controlled environment (sandbox) and recording its runtime behavior. Since dynamic analysis platforms directly run malicious code, they are resilient to popular malware defense techniques such as packing and code obfuscation. Unfortunately, in many cases, only a small subset of all possible malicious behaviors is observed within the short time frame that a malware sample is executed. To mitigate this issue, previous work introduced techniques such as multipath or forced execution to increase the coverage of dynamic malware analysis. Unfortunately, using these techniques is potentially expensive, as the number of paths that require analysis can grow exponentially. In this paper, we propose REANIMATOR, a novel solution to determine the capabilities (malicious functionality) of malware programs. Our solution is based on the insight that we can leverage behavior observed while dynamically executing a specific malware sample to identify similar functionality in other programs. More precisely, when we observe malicious actions during dynamic analysis, we automatically extract and model the parts of the malware binary that are responsible for this behavior. We then leverage these models to check whether similar code is present in other samples. This allows us to statically identify dormant functionality (functionality that is not observed during dynamic analysis) in malicious programs. We evaluate our approach on thousands of realworld malware samples, and we show that our system is successful in identifying additional, malicious functionality. As a result, our approach can significantly improve the coverage of malware analysis results.

LIO-IDS: Handling class imbalance using LSTM and improved one-vs-one technique in intrusion detection system

Abstractive single document summarization is considered as a challenging problem in the field of artificial intelligence and natural language processing. Meanwhile and specifically in the last two years, several deep learning summarization approaches were proposed that once again attracted the attention of researchers to this field.It is a well-known issue that deep learning approaches do not work well with small amounts of data. With some exceptions, this is, unfortunately, the case for most of the datasets available for the summarization task. Besides this problem, it should be considered that phonetic, morphological, semantic and syntactic features of the language are constantly changing over the time and unfortunately most of the summarization corpora are constructed from old resources. Another problem is the language of the corpora. Not only in the summarization field, but also in other fields of natural language processing, most of the corpora are only available in English. In addition to the above problems, license terms, and fees of the corpora are obstacles that prevent many academics and specifically non-academics from accessing these data.This work describes an open source framework to create an extendable multilingual corpus for abstractive single document summarization that addresses the above-mentioned problems. We describe a tool consisted of a scalable crawler and a centralized key-value store database to construct a corpus of an arbitrary size using a news aggregator service.

: Network-based intrusion detection systems (NIDSs) are one component of a comprehensive network security solution. The use of IPsec, which encrypts network traffic, renders network intrusion detection virtually useless unless traffic is decrypted at network gateways. One alternative to NIDSs, host-based intrusion detection systems (HIDSs), provides some of the functionality of NIDSs but with limitations. HIDSs cannot perform a network-wide analysis and can be subverted if a host is compromised. We propose an approach to intrusion detection that combines HIDS, NIDS, and a version of IPsec that encrypts the header and the body of IP packets separately. We refer to the latter generically as TwoKey IPsec. We show that all of the network events currently detectable by the Snort NIDS on un- encrypted network traffic are also detectable on encrypted network traffic using this approach. The NIDS detects network-level events that HIDSs have trouble detecting and HIDSs detect application-level events that can't be detected by the NIDS.

The effectiveness and precision of network-based intrusion detection signatures can be evaluated either by direct analysis of the signatures (if they are available) or by using black-box testing (if the system is closed-source). Recently, several techniques have been proposed to generate test cases by automatically deriving variations (or mutations) of attacks. Even though these techniques have been useful in identifying "blindspots" in the signatures of closed-source, network-based intrusion detection systems, the generation of test cases is performed in a random, un- guided fashion. The reason is that there is no information available about the signatures to be tested. As a result, identifying a test case that is able to evade detection is difficult. In this paper, we propose a novel approach to drive the generation of test cases by using the information gathered by analyzing the dynamic behavior of the intrusion detection system. Our approach applies dynamic dataflow analysis techniques to the intrusion detection system to identify which parts of a network stream are used to detect an attack and how these parts are matched by a signature. The result of our analysis is a set of constraints that is used to guide the black-box testing process, so that the mutations are applied to only those parts of the attack that are relevant for detection. By doing this, we are able to perform a more focused generation of the test cases and improve the process of identifying an attack variation that evades detection.

The Intrusion Detection System (IDS) is a useful infiltration detection tool. With IDS, it may mechanically categorize intrusions, attacks, or breaches of security stratagems across host-level within the network and within the organization. The Krill Herd Optimized Deep Neural Network (KH-DNN) is proposed in this document as a multiclass cyberattack methodology. This model is suggested by uniting the network-based intrusion detection system (NIDS) and the host-based intrusion detection system (HIDS) to advance flexible and actual IDS to categorize and detect unexpected and unpredictable cyber attacks. In this method, two types of IDS are classified, that is, NIDS and HIDS. Classification of the system call is exploited to discriminate behavior into normal and attack categories. In this manuscript accuracy, the parameter of the deep neural network classifier is optimized with the Krill Herd optimization algorithm. Finally, the DNN model is performed in the NIDS dataset and the HIDS dataset. For the HIDS dataset, the KDD Cup 99 dataset was used, which employed two datasets: ADFA-LD and ADFA-WD, and several types of attacks were detected and classified, including Adduser, Hydra-FTP, Java-Meterpreter, Hydra-SSH, and Web-Shell. The suggested IDS-KH-DNN algorithm attains higher accuracy 42.56% and 23.4%, high precision 84.74% and 52.43%, high F-score 53.5% 64.455, high sensitivity 33.4% and 45.23%, and high specificity 31.45% and 44.23% for the NIDS dataset. The proposed system is compared to two current processes: intrusion detection using Gray Wolf optimization-based support vector machine (IDS-GWO-SVM) and intrusion detection using artificial bee colony optimization algorithm based support vector machine (IDS-ABCO-SVM). Finally, the simulation results show that the suggested approach is capable of quickly and accurately locating optimal solutions.

Network-based Intrusion Detection Systems (NIDSs) are an important mechanism to identify malicious behaviour or policy violations within a network. Such detection systems typically face several challenges, among which are the base-rate fallacy and the resilience against adaptive adversaries. These challenges are often countered in modern NIDSs by combining multiple detection systems to diversify the used feature levels or utilize the advantages of multiple detection methods. However, currently there exists no suitable framework for a detailed analysis of such composed systems. Therefore, the contribution of this work is an evaluation framework for composed systems, which builds on previous information-theoretic approaches and highlights the utility of information-theoretic redundancies for robustness evaluations. This framework enables an attribution of the overall system performance to its individual components, to fine-tune parameters and to study the dynamics between classifiers. The versatility of the framework is demonstrated by designing and evaluating a composed NIDS example based on systems described in the literature and using an open data set. Studying the impact of an evasion attempt with adversarial examples on this system highlighted the importance of robustness against false-alarms as well as detection evasion. Moreover, the framework enables general insights on how to improve the design of composed NIDSs: based on the dynamics between classifiers, it can be shown that optimizing the operation point of each component individually does not necessarily maximize the overall system performance from an information-theoretic perspective. Additionally, it can be shown that existing classification redundancies might not be fully utilized during an attack on the NIDS components, due to a static system design.