Joint network-host based malware detection using information-theoretic tools

Malware is a global problem. Malware's impact according to studies is escalating. In the fight against malware, Malware detection and analysis techniques are the most important defense tools. That which makes a detector good, or terrible, is largely determined by its techniques of operation. Thorough understanding of the many types of malware detection procedures is essential. This chapter investigates malware detection and analysis techniques, first and second-generation malware (i.e. encrypted, metamorphic, polymorphic, oligomorphic), and the study of machine learning algorithms for malware detection techniques.

ABSTRACTWith the development of information technology, malicious software attack methods are becoming increasingly sophisticated, posing a serious threat to network security. Aiming at the problems of existing malware detection technologies, such as low detection rate, high false alarm rate, and poor real‐time performance, this paper proposes a malware detection and defense algorithm based on a hybrid convolutional graph neural network. This algorithm combines the advantages of CNN and GNN to effectively identify malicious behavior by extracting deep‐seated malware features. The experimental results show our algorithm has high accuracy and real‐time performance in malware detection. We employed publicly available malware datasets during the experiment for training and testing. After comparative experiments, the detection rate reaches 95.2%, which is 8.6% higher than the CNN algorithm and 5.3% higher than that of the GNN algorithm. In terms of false alarm rate, the algorithm in this paper is only 1.2%, far lower than the traditional CNN algorithm's 5.8% and the GNN algorithm's 3.5%. At the same time, our algorithm also shows excellent performance in real‐time, and the average detection time is only 0.12 s, which is 20% lower than the traditional algorithm. In addition, to verify the defense capability of the algorithm, we simulate various malware attack scenarios in our experiments. The results show that our algorithm can effectively identify and defend against 90.6% of malware attacks, which strongly supports network security protection.

The increasing incidence of malware presents significant obstacles to cybersecurity, necessitating sophisticated and effective techniques for detection and categorization. This paper presents a novel method that improves the precision and efficacy of malware classification by utilizing multi-processing and Bag-of-Words (BoW) vectorization. The suggested technique utilizes feature vectors to depict malware samples, leveraging the parallel processing capabilities of contemporary computer systems to expedite classification operations. This solution centers on a bespoke HexVectorizer that transforms hexadecimal strings obtained from malware binaries into feature vectors. The approach employs a balanced subset of the Microsoft Malware Classification dataset, meticulously preprocessed for dependable evaluation to assure thorough analysis. Python's multiprocessing module is utilized to meet the computing requirements of extensive datasets, facilitating the parallelization of vectorization processes and markedly enhancing processing performance. The classification system is centered on XGBoost's XGBClassifier, recognized for its superior performance and accuracy in addressing malware detection and classification issues. The experimental findings validate the efficacy of the suggested technique in real-time malware detection, confirming its relevance to diverse cybersecurity contexts. This paper offers an exhaustive elucidation of the implementation procedure, accompanied by thorough performance assessments. The results highlight the method's promise as a scalable and efficient approach for tackling the increasing issues of malware classification in cybersecurity. This research greatly advances the creation of resilient and efficient malware detection systems by integrating sophisticated vectorization techniques with cutting-edge machine learning algorithms, therefore fulfilling essential requirements in a dynamic threat environment.

SummaryIntrusion detection system (IDS) represents an unavoidable tool to secure our network. It is considered as a second defense line against the different form of attacks. The principal limits of the current IDSs are their inability to combine the detection of the new form of attacks with high detection rate and low false alarm rate. In this paper, we propose an intrusion detection system based on the combination of the probability predictions of a tree of classifiers. Specifically, our model is composed of 2 layers. The first one is a tree of classifiers. The second layer is a classifier that combines the probability predictions of the tree. The built tree contains 4 levels where each node of this tree represents a classifier. The first node classifies the connections in 2 clusters: Denial of Service attacks and Cluster 2. Then, the second node classifies the connections of the Cluster 2 in Probing attacks and Cluster 3. The third node classifies the connections of the Cluster 3 in Remote‐to‐Local attacks and Cluster 4. Finally, the last node classifies the connections of the Cluster 4 in User‐to‐Root attacks and Normal connections. The second layer contains the last classifier that combines the probability predictions of the first layer and take the final decision. The experiments on KDD'99 and NSL‐KDD show that our model gives a low false alarm rate and the highest detection rate. Furthermore, our model is more precise than the recent intrusion detection system models with accuracy equal to 96.27% for KDD'99 and 89.75% for NSL‐KDD.

: With the increased basis of applications and services on Cloud, the dependence on cloud technologies has expanded rapidly. This transition eases some of the hardware management troubles, but it also exposes broad security threats, especially in regard to malware. Malware is a major threat to cloud computing systems mainly in IaaS dominated environments. In this paper we explore the usage of Recurrent Neural Networks, RNNs in malware detection in cloud VMs. In particular, we examine two of the most popular RNN models; Long short term Memory, LSTM and Bidirectional RNNs, BIDI, that try to understand malware patterns by watching the activity of systems and the use of CPU, Memory and Disk. On the dataset of 40,680 samples 40,680 malware and benign samples, the models were trained on process level features from real malware in a free and open cloud environment. Both models achieved detection rates of over 99% in average key performance metrics. Moreover, wealso investigate the impact of different representations of input data on the performance of the models. The results of the study clearly support the viability of RNNs for online malware detection in cloud infrastructure, and allow for high accuracy and real time detection.

The attackers do not want their Malicious software (or malwares) to be reviled by anti-virus analyzer. In order to conceal their malware, malware programmers are getting utilize the anti reverse engineering techniques and code changing techniques such as the packing, encoding and encryption techniques. Malware writers have learned that signature based detectors can be easily evaded by “packing” the malicious payload in layers of compression or encryption. State-of-the-art malware detectors have adopted both static and dynamic techniques to recover the payload of packed malware, but unfortunately such techniques are highly ineffective. If the malware is packed or encrypted, then it is very difficult to analyze. Therefore, to prevent the harmful effects of malware and to generate signatures for malware detection, the packed and encrypted executable codes must initially be unpacked. The first step of unpacking is to detect the packed executable files. The objective is to efficiently and accurately distinguish between packed and non-packed executables, so that only executables detected as packed will be sent to an general unpacker, thus saving a significant amount of processing time. The generic method of this paper show that it achieves very high detection accuracy of packed executables with a low average processing time. In this paper, a packed file detection technique based on complexity measured by several algorithms, and it has tested using a packed and unpacked dataset of file type .exe. The preliminary results are very promising where achieved high accuracy with enough performance. Where it achieved about 96% detection rate on packed files and 93% detection rate on unpacked files. The experiments also demonstrate that this generic technique can effectively prepared to detect unknown, obfuscated malware and cannot be evaded by known evade techniques.

Fuzzy hashing has been utilised in digital forensics and malware analysis for malware detection, malware variant classification, file clustering, document similarity detection, embedded object detection and fragment detection. Previous research considered the efficacy of fuzzy hashing at a point in time for malware classification and did not specifically address the problem of malware evolution. Android malware presents a significant cybersecurity threat, and since malware is constantly mutating, a temporal analysis of the effectiveness of fuzzy hashing techniques for Android malware detection and classification contributes to understanding the value of fuzzy hashes in the evolution of malware. Through experimental examination, this study sought to determine whether or not fuzzy hashes are always effective, how quickly malware is evolving, and how malware evolution affects fuzzy hashing. Comparisons are made between the performance of different fuzzy hashing algorithms and the distinction between hashes at the file and class levels. Experiments with known malware family and analysis with over 4500 APK files, including 100 benign samples collected from 2012 - 2022 were conducted using various fuzzy hashing algorithms, file-level and section-level similarity hashing, symbolic and raw opcode hashing, and optimisations for improving fuzzy hashing comparisons. The performance of the methods was evaluated using detection and false positive rates. The results show that fuzzy hashing algorithms remain a valuable technique that demonstrates robustness to malware evolution with 10-year detection rates of over 80%.

Nowadays, along with network development, due to the threats of unknown sources, information communication is more vulnerable, and thus, more secured information is required. Intrusion Detection System (IDS) is very important for cybersecurity with the presence in particular of various networked computers' foundation. An efficient IDS apply machine learning method as computational technics to increase rates of detection to gain the high accuracy and low false alarms rate within the huge amounts of data. To increase the rate of detection, researcher usually implements the optimizer. Thus, in this research, a comprehensive experimental study is presented based on various binary to optimize the rate of detection and decrease the error. Moreover, Numerous researches have been conducted about intrusion detection systems with the old dataset such as Kddcup'99 dataset, and due to this reason, most of them did not get the potential accuracy to detect intrusion with the current intrusion as the old dataset is not covering the current attacks. Therefore, this research aims to A hybrid Anomaly Classification of IDS with Deep Learning (DL) and Binary Algorithms (BA) as Optimizer with the most updated dataset named "CICIDS2017" which can be used for the intrusion detection evaluation. Moreover, this research conducts the anomaly classification of IDS based on the deep neural network (DNN) as the Deep Learning (DL) platform and Binary Algorithms (BA) in terms of Binary Bat Algorithm (BBA), Binary Genetic Algorithm (BGA), Binary Gravitational Search Algorithm (BGSA) as optimizer to enhance the rates of detection. Some of the results which had been considered and achieved for DNN and the hybrid version (DNN and Binary Algorithms) are in terms of: Accuracy, Recall, Precision, Confusion Matrix, Sensitivity, Specificity, and Cost Error.

The proliferation of the internet of things (IoT) has significantly enhanced the convenience and functionality of various applications ranging from personal devices to industrial systems. However, this expansion has also escalated the vulnerability of these networks to sophisticated malware attacks, posing a critical threat to the security and reliability of IoT systems. This chapter introduces an innovative defense framework based on the local outlier factor (LOF) technique for effective malware detection in IoT networks. The framework employs a systematic approach, starting from data collection and preprocessing to the application of LOF for anomaly detection. The research demonstrates through empirical analysis that the LOF-based method outperforms traditional malware detection techniques, offering higher precision, recall, and lower false positive rates. The comparative analysis with existing IoT malware detection methods such as Mobile-net IoT and Image-net IoT further validates the superiority of the LOF approach.

Nowadays, ransomware is considered one of the most critical cyber-malware categories. In recent years various malware detection and classification approaches have been proposed to analyze and explore malicious software precisely. Malware originators implement innovative techniques to bypass existing security solutions. This paper introduces an efficient End-to-End Ransomware Detection System (E2E-RDS) that comprehensively utilizes existing Ransomware Detection (RD) approaches. E2E-RDS considers reverse engineering the ransomware code to parse its features and extract the important ones for prediction purposes, as in the case of static-based RD. Moreover, E2E-RDS can keep the ransomware in its executable format, convert it to an image, and then analyze it, as in the case of vision-based RD. In the static-based RD approach, the extracted features are forwarded to eight various ML models to test their detection efficiency. In the vision-based RD approach, the binary executable files of the benign and ransomware apps are converted into a 2D visual (color and gray) images. Then, these images are forwarded to 19 different Convolutional Neural Network (CNN) models while exploiting the substantial advantages of Fine-Tuning (FT) and Transfer Learning (TL) processes to differentiate ransomware apps from benign apps. The main benefit of the vision-based approach is that it can efficiently detect and identify ransomware with high accuracy without using data augmentation or complicated feature extraction processes. Extensive simulations and performance analyses using various evaluation metrics for the proposed E2E-RDS were investigated using a newly collected balanced dataset that composes 500 benign and 500 ransomware apps. The obtained outcomes demonstrate that the static-based RD approach using the AB (Ada Boost) model achieved high classification accuracy compared to other examined ML models, which reached 97%. While the vision-based RD approach achieved high classification accuracy, reaching 99.5% for the FT ResNet50 CNN model. It is declared that the vision-based RD approach is more cost-effective, powerful, and efficient in detecting ransomware than the static-based RD approach by avoiding feature engineering processes. Overall, E2E-RDS is a versatile solution for end-to-end ransomware detection that has proven its high efficiency from computational and accuracy perspectives, making it a promising solution for real-time ransomware detection in various systems.

Malicious software (ransom ware) cyber attacks in frequency and severity, posing an increasingly serious threat to computer systems everywhere. Malware detection is a hot study area as several multiple computers, organisations, and governments have been affected by an exponential rise in malware attacks. Dynamic and static assessment of malicious characteristics and behaviour patterns is time expensive and useless in real-time malware detection, according to current technologies. It is becoming increasingly common for malicious apps to use polymorphic and adaptive techniques to rapidly modify their behaviour and develop a number of new malicious apps. In order to undertake an effective malware analysis, machine learning techniques (MLAs) are increasingly being used to create new malware varieties. This approach is time-consuming since it requires considerable feature engineering, learning and representation of features. Moreover the feature extraction process could be effectively eliminated by using advanced MLAs like deep learning. These methods have been shown to perform better with a biased training dataset, which restricts their practical application in real-time scenarios. A new improved approach for successful zero-day malware detection must be developed in order to eliminate biases and analyze these approaches autonomously.

For the last few years, Android is known to be the most widely used operating system and this rapidly increasing popularity has attracted the malware developer’s attention. Android allows downloading and installation of apps from other unofficial market places. This gives malware developers an opportunity to put repackaged malicious applications in third-party app-stores and attack the Android devices. A large number of malware analysis and detection systems have been developed which uses static analysis, dynamic analysis, or hybrid analysis to keep Android devices secure from malware. However, the existing research clearly lags in detecting malware efficiently and accurately. For accurate malware detection, multilayer analysis is required which consumes large amount of hardware resources of resource constrained mobile devices. This research proposes an efficient and accurate solution to this problem, named SAMADroid, which is a novel 3-level hybrid malware detection model for Android operating systems. The research contribution includes multiple folds. First, many of the existing Android malware detection techniques are thoroughly investigated and categorized on the basis of their detection methods. Also, their benefits along with limitations are deduced. A novel 3-level hybrid malware detection model for Android operating systems is developed, that can provide high detection accuracy by combining the benefits of the three different levels: 1) Static and Dynamic Analysis; 2) Local and Remote Host; and 3) Machine Learning Intelligence. Experimental results show that SAMADroid achieves high malware detection accuracy by ensuring the efficiency in terms of power and storage consumption.

Malware detection engines are required a high True Positive Rate (TPR) and low False Positive rate (FPR). For example, to obtain the VB100 certification conducted by Virus Bulletin in the U.K., the detector's TPR must be 99.5% or higher, and FPR less than 0.01%. However, it is difficult for signature-based malware detection engines to detect zero-day or new variants of malware. For such malware, an approach based on machine learning is considered effective because the features of specimens are analyzed in a complex manner. Therefore, we created a malware detection model from surface analysis logs, and PE header dumps by machine learning using the FFRI Dataset 2018. Furthermore, we verified the accuracy under the constraint that the FPR is less than a certain value. As a result, we succeeded in creating a new model with high accuracy: when FPR is set to be less than 1%, TPR is 99.7%, when FPR is set to be less than 0.1%, TPR is 98.7%, and when FPR is set to be less than 0.01% and TPR is 94.5%. In addition, we revealed features with high contribution to malware detection in this model.

Mobile devices are everywhere nowadays, such as mobile phone, mobile tablets. Meanwhile, various malwares on mobile terminals are emerging one after another, especially on the open-source Android system. Traditional detection schemes are based on static method or dynamic method. In recent years, industry and academia have paid close attention to the detection mechanisms using network behaviors to identify the malware. In this paper, we design a real-time Android malware detection system based on network traffic analysis, which includes a training model and a real-time detection model. By training over the malware traffic using the training model, we find that 76.33 % DNS queries and 45.39 % HTTP requests are all malicious. We set up a real-time scanning service based on the malicious URLs that are captured in the training model, which is the core of the real-time detection model. By performing malware detection using the established real-time detection model, we show that the detection rate using the real-time scanning service is much higher than the integrated service. Meanwhile, the detection rate will further improve by integrating more third-party scanning services into our system.

The amount of data in the field of computer networking is growing rapidly and this urges new challenges in the field of an intrusion detection system (IDS). To handle such increasing volume of data, a new hybrid approach has to be developed to overcome the problems such as high detection rate and low false alarm rate. An intrusion detection system plays a vital role in the detection of malicious attacks. Data mining and machine learning techniques are important and play a vital role in the detection of attacks. This paper mainly focuses on detection rate and false alarm rate and so to resolve these problems a hybrid method, ensemble clustering, has been proposed. This method tries to increase detection rate with lowering false alarm rate. The method has been tested on KDDCup'99 network intrusion dataset and performs well as compared with other algorithms in terms of detection rate and false alarm rate.