On the Design of Supervised Binary Classifiers for Malware Detection Using Portable Executable Files

As malicious programs, Trojan horses have become a huge threat to computer networks security. Trojan horses can easily cause loss, damage or even theft of data because they are usually disguised as something useful or desirable, and are always mistakenly activated by computer users, corporations and other organizations. Thus, it is important to adopt an effective and efficient method to detect the Trojan horses, and the exploration of a new method of detection is of greater significance. Scientists and experts have tried many approaches to detecting Trojan horses since they realized the harms of the programs. Up to now, these methods fall mainly into two categories [2]. The first category is to detect Trojan horses through checking the port of computers since the Trojan horses send out message through computer ports [2]. However, these methods can only detect the Trojan horses that are just working when detected. The second class is to detect Trojan horses by examining the signatures of files [2] [19], in the same way as people deal with computer virus. As new Trojan horses may contain unknown signatures, methods in this category may not be effective enough when new and unknown Trojan horses appear continuously, sending out unknown signatures that escape detection. For the above-mentioned reasons, without exception, there are limitations in the existing methods if the un-awakened and unknown Trojan horses are to be detected. This thesis proposes a new method that can detect un-awakened and unknown Trojan horses- the detection by using of a file's static characteristics. This thesis takes PE file format as the object of the research, because approximately 75% of personal computers worldwide are installed the Microsoft Windows [4], and that Trojan horses usually exist as a Portable Executable (PE) file in the Windows platform. Based on the (PE) file format, my research gets all the static information of each part of PE file which is characteristic of a file. Then, this static information is analyzed by the intelligent information processing techniques. Next, a detection model is established to estimate whether a PE file is a Trojan horse. This model can detect the unknown Trojan horses by analyzing static characteristics of a file. The information that is used to verify detecting model is new and unknown to the detecting model; in other words, the information is not used during the training of the model. The thesis is organized as follows. First, this thesis discusses the limitations of traditional detection techniques, related works of research, and a new method to detect Trojan horse based on file's static information. Second, the thesis focuses on the research of the Trojan horse detecting models, covering the extracting of the static information from PE file, choice of intelligent information processing techniques, and setting up the Trojan horse detecting model. Lastly, the thesis discusses the direction of future research in this field.

PE (Portable executable) file is a standard format for executable file and is applied extensively. PE file has diversity, uncertainty of file size, complexity of file structure and singleness of file format, which make PE file easy to be a carrier of information hiding, especially for that of large hiding capacity. A novel software information hiding algorithm is proposed, which makes full use of display characteristics of palette icon of portable executable file. In this algorithm, the information is embedded into the transparent area of the icon by taking advantage of the redundant color items in the palette. The experimental results show that after embedding the information, the size of the icon remains unchanged, that is, the size of the resource section will not change, and the size of the PE file will not change. On the other hand, the icon with embedded data can be correctly analyzed and displayed without any distortion. PE file can run normally, and does not affect the performance of the program, so the algorithm has good concealment. The algorithm selects an index value whose color is black according to rules in the XOR bitmap, these index values are encoded for information hiding, its complexity is low. At the same time, we can further improve the hiding capacity by adding one or more icons to PE file.

This study investigates whether convolutional neural networks (CNNs) trained on visual representations of Portable Executable (PE) files can rival traditional machine learning classifiers trained on engineered features. A dataset of over 200,000 PE files [1] was used to derive two feature sets (Basic and Ember-Lite) [2] and to generate 256x256 greyscale and entropy images [3],[4]. Three CNNs (SimpleCNN, ResNet-18 [5], EfficientNet-B0 [6]) were trained and evaluated against five baselines (Random Forest, XGBoost [7], CatBoost [8], LightGBM, Logistic Regression). Tree-based models with enriched features achieved the highest scores, with CatBoost reaching a ROC-AUC of 0.990. The best CNN, EfficientNet-B0 on entropy images, obtained a ROC-AUC of 0.954. Although CNNs did not surpass feature-based models, they showed competitive results when feature engineering was constrained. These findings indicate that visual approaches offer a promising alternative for static malware detection, particularly when combined with entropy-based representations [9].

The Portable executable file format is the format of executables, object code and DLL’s (Dynamic Link Library) in Microsoft Windows Operating system. It is the standard of how executable files are organized within file system in Microsoft Windows. The Portable executable file format was designed for Windows NT 3.1 that released in 1993. Many of its features are inherited from COFF (Common object file format) used in Unix Operating systems. It consists of header and sections; headers are rules that tell windows loader how the section should be mapped and loaded into memory. The section are nothing but the data or content. Different sections hold different kind of data for example code section contains executable code while bss (Block Started by Symbol) sections contains uninitialized data. Portable executable file could be used in a way to dent the security of computer, therefore great care should be taken while downloading and running Portable executable files. In our work we develop a static malware analysis tool called ‘Portable Executable File Analysis Framework (PEFAF)’ using data mining techniques. A collection of 8 thousand benign and 7 thousand malicious files were used in this work. We extracted 60 features, analyzed them and found that 34 of them are significant for the detection of malware threats. Based on these 34 indicators our tool classifies input file into malicious or non-malicious.

Traditional malware detection methods cannot keep up with the massive amount of newly created malware quickly and effectively. Machine learning is a promising method for the detection and classification of large-scale newly created malware according to the features of samples. The current research trend is to use machine learning technology, such as the Gradient Boosting Decision Tree (GBDT) and deep neural network technology, to learn newly created malware rapidly and accurately. We propose Control-Flow Graph (CFG)- and Graph Isomorphism Network (GIN)-based malware classification, where we first extract the CFG from portable executable (PE) files and use the large-scale pre-training language model MiniLM to generate the node features of CFG. The extracted CFG is compressed to a feature vector with GIN and classified with Multi-Layer Perceptron. To evaluate our approach, we made a CFG-based malware detection dataset from PE files of the Dike Dataset, which we call the Malware Geometric Dataset (MGD), and collected the results. The evaluation results show that our proposal demonstrated 0.9977 in the Area Under Curve metric and achieved a 97.44 % detection rate when the False Positive Rate was 0.1 %.

Classification and detection of zero-day attacks remain a significant challenge within the domain of cybersecurity. Due to the vast types of malware families and the presence of an imbalanced dataset, real-time detection and classification become increasingly complex and inaccurate. Thus, there’s an urgent need to develop an intelligent and adaptive defense mechanism capable of identifying and classifying such attacks with improved precision and robustness. This paper proposed a stacked ensemble federated learning model with an accuracy-aware node weighting scheme to address the challenges posed by inter- and intra-class similarities among different types of malwares. In the initial phase, malware Portable Executable (PE) files are collected from multiple online repositories and validated by three different antivirus programs through VirusTotal to ensure reliability. These validated files are then converted into image form and categorized into 28 families to facilitate feature extraction. In the second phase, deep feature representations are extracted through a transfer learning-based fine-tuned ResNet-50 model, which captures both low-level and high-level patterns that are relevant to malware classification. After feature extraction from multiple distributed nodes, architecture is fed into the novel proposed Ensemble Stacked Federated Model for enhanced generalization and robust classification. The model is tested on both private and publicly available datasets. The experimental results demonstrate that the proposed method outperforms existing baseline approaches in terms of accuracy and computational efficiency. This improvement is achieved because it performs independent training at each federated node separately and then stacks their outputs with a central ensemble model, which enhances the learning rate and reduces overfitting. The code used for the experiments is available here.

Wavelet decomposition of software entropy reveals symptoms of malicious code

One of the main challenges in security today is defending against unknown malware attacks which have the potential to harm a computer or network. Hence, detecting malware has become one of the most important challenges for the security of computer systems. The known malware detection methods based on the appearance of opcode sequences has to construct a matrix from programs of different architectures to extract high-level features. In order to resolve high dimensional inputs vector and differences assembly instruction, this paper proposes a novel method for detecting static characteristics of 32-bit and 64-bit malicious Portable Executable (PE) Windows files by opcode sequences analysis. By compute the frequency of occurrence of each opcode sequence and distinguishing different types of 32-bit and 64-bit PE files, the proposed method shows promising results with less complexity in comparison with previous studies, which is beneficial to train machine learning model such as k-nearest neighbor (KNN) and back-propagation neural network (BP). Our method is evaluated on more than 20,000 samples, and experimental results show that our system can effectively detect and classify unknown malware.

It is a difficult issue in the anti-virus field about how to detect unknown and packed PE (Portable Executable) viruses effectively, and existing schemes for anti-virus detection are dissatisfactory. Based on an analysis of the logical structures of Windows PE file and PE virus, a new method of PE virus detection is presented here, which exploits fragile software watermarking technique for virus detection. It describes how the new scheme embeds a software watermark into the PE files for virus detection. The experiment shows that the new method has a higher rate in detecting unknown PE virus than some existing ones, and can detect directly the packed PE virus.

A novel deep learning-based approach for malware detection

Detecting malware is crucial for ensuring the security of computer systems. Traditional machine learning models face challenges in effectively detecting malware, mainly due to the class imbalance problem, where the number of malware samples is significantly smaller than that of non-malware samples. Additionally, malware’s dynamic and evolving nature, continuously altering its structure and tactics, presents a substantial challenge for conventional artificial intelligence algorithms, further complicating the detection task. In pursuing an optimized malware detection technique, researchers initially explored traditional machine learning algorithms, focusing on the features of Portable Executable (PE) file headers. However, the inherent issues, such as imbalanced datasets and the deceptive nature of malware, have raised concerns about the credibility of the attained results. This can result in misclassifying malware as non-malware, leading to security vulnerabilities. One-Class Classification (OCC) methods have emerged as a promising approach to improve the detection of unknown malware. However, traditional OCC approaches face the challenge of the curse of dimensionality. This research proposes adapting subspace learning-based OCC methods to overcome the curse of dimensionality and effectively handle the class imbalance problem. We propose a pipeline for detecting malware using methods that jointly optimize a subspace and data description for OCC. We evaluate the performance of various one-class classifiers on three different datasets. We observed that the subspace-learning-based OCC is a promising approach. Evaluating various classifiers on three datasets reveals promising results, with a True Positive Rate (TPR) of 100 % for subspace-learning-based OCC. The proposed pipeline can serve as a valuable tool for improving the security of computer systems by accurately detecting malware and protecting against potential attacks.

Distinguishing malicious programs based on visualization and hybrid learning algorithms

The internet’s rapid growth has resulted in an increase in the number of malicious files. Recently, powershell scripts and Windows portable executable (PE) files have been used in malicious behaviors. To solve these problems, artificial intelligence (AI) based malware detection methods have been widely studied. Among AI techniques, the graph convolution network (GCN) was recently introduced. Here, we propose a malicious powershell detection method using a GCN. To use the GCN, we needed an adjacency matrix. Therefore, we proposed an adjacency matrix generation method using the Jaccard similarity. In addition, we show that the malicious powershell detection rate is increased by approximately 8.2% using GCN.

Recent research indicates that effective malware detection can be implemented based on analyzing portable executable (PE) file headers. Such research typically relies on prior knowledge of the header to extract relevant features. However, it is also possible to consider the entire header as a whole, and use this directly to determine whether the file is malware. In this research, we collect a large and diverse malware data set. We then analyze the effectiveness of various machine learning techniques based on PE headers to classify the malware samples. We compare the accuracy and efficiency of each technique considered.

Memory forensics is a young but fast-growing area of research and a promising one for the field of computer forensics. The learned model is proposed to reside in an isolated core with strict communication restrictions to achieve incorruptibility as well as efficiency, therefore providing a probabilistic memory-level view of the system that is consistent with the user-level view. The lower level memory blocks are constructed using primary block sequences of varying sizes that are fed as input into Long-Short Term Memory (LSTM) models. Four configurations of the LSTM model are explored by adding bi- directionality as well as attention. Assembly level data from 50 Windows portable executable (PE) files are extracted, and basic blocks are constructed using the IDA Disassembler toolkit. The results show that longer primary block sequences result in richer LSTM hidden layer representations. The hidden states are fed as features into Max pooling layers or Attention layers, depending on the configuration being tested, and the final classification is performed using Logistic Regression with a single hidden layer. The bidirectional LSTM with Attention proved to be the best model, used on basic block sequences of size 29. The differences between the model’s ROC curves indicate a strong reliance on the lower level, instructional features, as opposed to metadata or string features.