Adversary Resistant Deep Neural Networks with an Application to Malware Detection

In recent years, a significant amount of research has focused on analyzing the effectiveness of machine learning (ML) models for malware detection. These approaches have ranged from methods such as decision trees and clustering to more complex approaches like support vector machine (SVM) and deep neural networks. In particular, neural networks have proven to be very effective in detecting complex and advanced malware. This, however, comes with a caveat. Neural networks are notoriously complex. Therefore, the decisions that they make are often just accepted without questioning why the model made that specific decision. The black box characteristic of neural networks has challenged researchers to explore methods to explain black-box models such as SVM and neural networks and their decision-making process. Transparency and explainability give the experts and malware analysts assurance and trustworthiness about the ML models’ decisions. In addition, it helps in generating comprehensive reports that can be used to enhance cyber threat intelligence sharing. As such, this much needed analysis drives our work in this paper to explore the explainability and interpretability of ML models in the field of online malware detection. In this paper, we used the Shapley Additive exPlanations (SHAP) explainability technique to achieve efficient performance in interpreting the outcome of different ML models such as SVM Linear, SVM-RBF (Radial Basis Function), Random Forest (RF), Feed-Forward Neural Net (FFNN), and Convolutional Neural Network (CNN) models trained on an online malware dataset. To explain the output of these models, explainability techniques such as KernalSHAP, TreeSHAP, and DeepSHAP are applied to the obtained results.

Summary A meticulous interpretation of steady-state or unsteady-state relative permeability (Kr) experimental data is required to determine a complete set of Kr curves. In this work, different machine learning (ML) models were developed to assist in a faster estimation of these curves from steady-state drainage coreflooding experimental runs. These ML algorithms include gradient boosting (GB), random forest (RF), extreme gradient boosting (XGB), and deep neural network (DNN) with a main focus on and comparison of the two latter algorithms (XGB and DNN). Based on existing mathematical models, a leading-edge framework was developed where a large database of Kr and capillary pressure (Pc) curves were generated. This database was used to perform thousands of coreflood simulation runs representing oil-water drainage steady-state experiments. The results obtained from these simulation runs, mainly pressure drop along with other conventional core analysis data, were used to estimate analytical Kr curves based on Darcy’s law. These analytically estimated Kr curves along with the previously generated Pc curves were fed as features into the ML model. The entire data set was split into 80% for training and 20% for testing. The k-fold cross-validation technique was applied to increase the model’s accuracy by splitting 80% of the training data into 10 folds. In this manner, for each of the 10 experiments, nine folds were used for training and the remaining fold was used for model validation. Once the model was trained and validated, it was subjected to blind testing on the remaining 20% of the data set. The ML model learns to capture fluid flow behavior inside the core from the training data set. In terms of applicability of these ML models, two sets of experimental data were needed as input; the first was the analytically estimated Kr curves from the steady-state drainage coreflooding experiments, while the other was the Pc curves estimated from centrifuge or mercury injection capillary pressure (MICP) measurements. The trained/tested model was then able to estimate Kr curves based on the experimental results fed as input. Furthermore, to test the performance of the ML model when only one set of experimental data is available to an end user, a recurrent neural network (RNN) algorithm was trained/tested to predict Kr curves in the absence of Pc curves as an input. The performance of the three developed models (XGB, DNN, and RNN) was assessed using the values of the coefficient of determination (R2) along with the loss calculated during training/validation of the model. The respective crossplots along with comparisons of ground truth vs. artificial intelligence (AI)-predicted curves indicated that the model is capable of making accurate predictions with an error percentage between 0.2% and 0.6% on history-matching experimental data for all three tested ML techniques. This implies that the AI-based model exhibits better efficiency and reliability in determining Kr curves when compared to conventional methods. The developed ML models by no means replace the need to conduct drainage coreflooding or centrifuge experiments but act as an alternative to existing commercial platforms that are used to interpret experimental data to predict Kr curves. The two main advantages of the developed ML models are their capability of predicting Kr curves within a matter of a few minutes as well as with limited intervention from the end user. The results also include a comparison between classical ML approaches, shallow neural networks, and DNNs in terms of accuracy in predicting the final Kr curves. The research presented here is an extension of the state-of-the-art framework proposed by Mathew et al. (2021). However, the two main aspects of the current study are the application of deep learning for the prediction of Kr curves and the application of feature engineering. The latter not only reduces the training/testing time for the ML models but also enables the end user to obtain the final predictions with the least set of experimental data. The various models discussed in this research work currently focus on the prediction of Kr curves for drainage steady-state experiments; however, the work can be extended to capture the imbibition cycle as well.

Optimisation and interpretation of machine and deep learning models for improved water quality management in Lake Loktak

Adversarial attacks are a major security risk for deep neural networks (DNN) and other machine learning models, which can substantially reduce the performance of the model and make security issues on the application of the model. Hence, in order to use DNNs as a reliable method, this paper proposes a DNN framework with the capability to defensive adversarial attacks. In the framework every single DNN model is designed to be differentiate. And decisions made by the framework rely on a part of DNNs in the framework which are chosen random. This makes the framework heterogeneous, dynamic, randomly and general. In the paper, we design a verification instance. In the instance, every DNN has different data transform and the structure is unique to each other. The results of the experiment demonstrate that the framework can giving predictions accurately and reliably in different kinds of adversarial and clean samples. This indicates that the framework is an acceptable method to defense adversarial attacks.

The game of Go has long been viewed as the most challenging of classic games for artificial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves. Here we introduce a new approach to computer Go that uses 'value networks' to evaluate board positions and 'policy networks' to select moves. These deep neural networks are trained by a novel combination of supervised learning from human expert games, and reinforcement learning from games of self-play. Without any lookahead search, the neural networks play Go at the level of state-of-the-art Monte Carlo tree search programs that simulate thousands of random games of self-play. We also introduce a new search algorithm that combines Monte Carlo simulation with value and policy networks. Using this search algorithm, our program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

Malware is a huge category of software, generally designed with malicious intent. It has gained much attention in the media over last decade with a number of high-profile ransomware cases coming to light. Malware is often used as part of a kill chain in Advanced Persistent Threats (APTs) attacks. It can easily be developed using tools such as “Veil Framework” by armature malware writers, or created from scratch by highly skilled individuals or teams as part in organised crime, or even by foreign agencies as part of an advanced cyber intelligence programme. As malware continue to evolve, at an ever-increasing rate the security community has turned to machine learning in a quest to find an effective method for detect and identify malware quickly. In response modern malware, is using more sophisticated detection-avoidance methodologies, against both binary analysis, and dynamic analysis. Thus, to preform, traditional static analysis requires advanced tool and an ever-increasing level of skill. Exploration of these various detection avoidance techniques as been included for understand of the issues encountered by security professionals, along with how improvements in software development methodologies and tooling has helped malware authors avoid detection, for little or no effort. This chapter explores the development of Deep Neural Networks in the defence against malware, looking specifically at how recent advances in Image Classification can be used detect and classify malware. As the Machine Learning industry seeks to make artificial intelligence and machine learning more accessible, through zero-code deep learning model training, this chapter will explore whether a zero-code approach to machine learning is feasible for developing deep neural network models for detecting and classifying malware. In particular the use of image classification models shall be explored, using off-the-self free or low-cost software, or cloud services.

Recent advancements in computer technology have precipitated a shift towards virtual environments, accelerated by the COVID-19 pandemic. Cybercriminals have capitalized on this trend, transitioning their activities to exploit vulnerabilities in cyberspace. Malicious software (malware) has emerged as a preferred tool for launching cyber-attacks, continually evolving with sophisticated obfuscation and packing techniques to evade detection. Traditional machine learning (ML) algorithms, once effective in identifying malware, are now struggling to keep pace with these advancements. In response, deep learning (DL) algorithms offer a promising solution, leveraging their ability to discern intricate patterns and correlations within data. This study proposes a novel hybrid deep-learning-based architecture, integrating two pre-trained network models to enhance classification accuracy. Through extensive evaluation on datasets including Malimg, Microsoft BIG 2015, and Malevis, the proposed method demonstrates significant improvements in accuracy, outperforming existing ML-based malware detection methods in the literature. Specifically, the proposed method achieves an impressive accuracy of 97.78% on the Malimg dataset, underscoring its effectiveness in combating sophisticated malware variants. Keywords — Malware, malware classification, malware detection, malware variants, deep neural networks, transfer learning, deep learning.

Traditional Image Processing Techniques (IPT), used for automating the detection and classification of weld defects from radiography images, have their own limitations, which can be overcome by Deep Neural Networks (DNN). DNN produces considerably good results in fields which offer big dataset for it to train. DNN trained with small datasets by conventional methods produces less accurate results. This limits the use of DNN in many fields. This study focuses to overcome this limitation, by adopting transfer learning using Pre-trained deep convolutional neural networks. By this method, a weld defect radiographic image classifier, which can classify 14 types of weld defects, was constructed. 940 Image patches of weld defects were manually collected and labelled from GDXray database. Subsequently the features of this weld defect dataset were extracted using VGG16 and ResNet50 CNNs, both pre-trained on the ImageNet database. Then machine learning models such as Logistic Regression, Support Vector Machine (SVM) and Random Forest were trained on these extracted features. The Classifier based on SVM trained on features extracted by ResNet50 outperforms the other counter parts with an accuracy of 98%. In all these cases, transfer learning improves performance and reduces the training time and computational system requirements.

Deep neural networks have shown outstanding performance in various areas, but adversarial examples can easily fool them. Although strong adversarial attacks have defeated diverse adversarial defense methods, adversarial training, which augments training data with adversarial examples, remains an effective defense strategy. To further improve adversarial robustness, this paper exploits adversarial examples of adversarial examples. We observe that these doubly adversarial examples tend to return to the original prediction on the clean images but sometimes drift toward other classes. From this finding, we propose a regularization loss that prevents these drifts, which mitigates the vulnerability against multi-targeted attacks. Experimental results on the CIFAR-10 and CIFAR-100 datasets empirically show that the proposed loss improves adversarial robustness.

The increasing prevalence of malware presents a critical challenge to cybersecurity, emphasizing the need for robust detection methods. This study uses a binary tabular classification dataset to evaluate the impact of feature selection, feature scaling, and machine learning (ML) models on malware detection. The methodology involves experimenting with three feature scaling techniques (no scaling, normalization, and min-max scaling), three feature selection methods (no selection, Linear Discriminant Analysis (LDA), and Principal Component Analysis (PCA)), and twelve ML models, including traditional algorithms and ensemble methods. A publicly available dataset with 11,598 samples and 139 features is utilized, and model performance is assessed using metrics such as accuracy, precision, recall, F1-score, and AUC-ROC. Results reveal that the Light Gradient Boosting Machine (LGBM) achieves the highest accuracy of 97.16% when PCA and either min-max scaling or normalization are applied. Additionally, ensemble models consistently outperform traditional ML models, demonstrating their effectiveness in enhancing malware detection. These findings offer valuable insights into optimizing preprocessing and model selection strategies for developing reliable and efficient malware detection systems.

Highlights Estimating the moisture content (MC) of cereal grain and oilseed in real time is a challenge. We explored the accuracy of MC estimation with machine learning models when cereal grain and oilseed were dried with a microwave device. The frequency, attenuation, phase shift, and dielectric properties were utilized for model training. Out of five machine learning models, a custom DNN model performed the best, and the RF model was the next best-performing model. ABSTRACT. Estimating moisture content (MC) in cereal grains and oilseeds plays a major role in their quality control, ensuring safe storage and determining their market value. Conventional MC measurement methods, such as oven drying, are time-consuming and not suitable for continuous monitoring. This study focuses on evaluating various machine learning algorithms to estimate MC in cereal grains and oilseeds by analyzing their dielectric properties (dielectric constant, e', and dielectric loss factor, e"). Machine learning (ML) models, including a modified Deep Neural Network (DNN), Autoencoder, LeNet5, Support Vector Machine (SVM), and Random Forest (RF), were evaluated for MC determination. The dataset consisted of multiple cereal grain and oilseed types of oats, soybeans, sorghum, corn, barley, and wheat (hereafter referred to as “grains” for simplicity). The models were trained on three test sets: (I) only dielectric properties from all grains, (II) all measured physical properties from all grains, and (III) all measured physical properties from wheat. Results showed that the DNN outperformed other models on test sets I and III with an R 2 value of 0.9084 and 0.9995, respectively, whereas RF showed the best performance in test set II with an R 2 value of 0.9784. These results highlight the effectiveness of ML algorithms in estimating the MC level of grains accurately, providing an alternative to traditional methods and enabling real-time and non-destructive monitoring in agricultural processes. Keywords: Cereal grains, Dielectric properties, Machine learning, Microwave sensing, Moisture content, Oilseeds.

Many studies have shown that Deep Neural Networks (DNNs) can be fooled by deceptive inputs, known as adversarial samples. Adversarial samples are produced by putting a small amount of perturbations into input images. These samples might mislead the object detection/recognition phases, thus it leads to the decreasing in the overall reliability of the object recognition process. Therefore, understanding the characteristic of adversarial samples is a very important step to propose the efficient attacking algorithms. In the paper, we present a new approach to particularly attack the face recognition process based on Fast Gradient Sign Attack (FGSA) and Jacobian-based Saliency Map Attack (JSMA) algorithms for the white-box attacks. By this way, we can evaluate the reliability of the face recognition process which are using MobileFaceNets and Caffe models.KeywordsAdversarial samplesFace recognitionFGSAJSMA

Recent work concerning adversarial attacks on ReLU neural networks has shown that unbounded regions and regions with a sufficiently large volume can be prone to containing adversarial samples. Finding the representation of linear regions and identifying their properties are challenging tasks. In practice, one works with deep neural networks and high-dimensional input data that leads to polytopes represented by an extensive number of inequalities, and hence demanding high computational resources. The approach should be scalable, feasible and numerically stable. We discuss an algorithm that finds the H-representation of each region of a neural network and identifies if the region is bounded or not.KeywordsNeural networkUnbounded polytopeLinear programmingReLU activation function

Network traffic classification technologies could be used by attackers to implement network monitoring and then launch traffic analysis attacks or website fingerprint attacks. In order to prevent such attacks, a novel way to generate adversarial samples of network traffic from the perspective of the defender is proposed. By adding perturbation to the normal network traffic, a kind of adversarial network traffic is formed, which will cause misclassification when the attackers are implementing network traffic classification with deep convolutional neural networks (CNN) as a classification model. The paper uses the concept of adversarial samples in image recognition for reference to the field of network traffic classification and chooses several different methods to generate adversarial samples of network traffic. The experiment, in which the LeNet‐5 CNN is selected as a classification model used by attackers and Vgg16 CNN is selected as the model to test the transferability of the adversarial network traffic generated, shows the effect of the adversarial network traffic samples.

Batteries are the most expensive component of battery electric vehicles (BEVs), but they degrade over time and battery operation. State of health (SOH) forecasting models learn how battery operation over long-time periods of weeks or months influences battery aging. Currently, existing methods for SOH forecasting of lithium-ion batteries based on deep neural network (DNN) models lack explainability of their forecasts due to their inherent black box character. However, the explainability of forecasts is essential to build user trust into the forecasting models. In this work, we address this problem from two perspectives: First, we compared four machine learning (ML) models like decision tree and random forest, which are inherently transparent, to two new DNN architectures with a more inherent black box character. Second, we proposed a new method using Gaussian-filtered saliency maps to visualize battery operational states that are relevant to DNN models. This method is applied to the best DNN models previously trained. We used an extensive data corpus consisting of five public data sets with different operational conditions, battery types, and aging trajectories. Furthermore, we show that the Gaussian-filtered saliency maps meaningfully visualize battery operational states that are consistent with findings from controlled laboratory aging experiments. Thus, this work was able to add transparency and interpretability to the SOH forecasting results of two state-of-the-art DNNs, while maintaining their superior performance compared to transparent ML models, while mitigating their inherent black box character.