Deep Learning Models to Identify and Classify Malware Variants

Designing of VehiNet Using Convolutional Neural Networks and Deep Learning Techniques

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Due to the major operating restrictions, ensuring security is the fundamental problem of Wireless Sensor Networks (WSNs). Because of their inadequate security mechanisms, WSNs are indeed a simple point for malware (worms, viruses, malicious code, etc.). According to the epidemic nature of worm propagation, it is critical to develop a worm defense mechanism in the network. This concept aims to establish novel malware detection in WSN that consists of several phases: “(i) Preprocessing, (ii) feature extraction, as well as (iii) detection”. At first, the input data is subjected for preprocessing phase. Then, the feature extraction takes place, in which principal component analysis (PCA), improved linear discriminant analysis (LDA), and autoencoder-based characteristics are retrieved. Moreover, the retrieved characteristics are subjected to the detection phase. The detection is performed employing combined shallow learning and DL. Further, the shallow learning includes decision tree (DT), logistic regression (LR), and Naive Bayes (NB); the deep learning (DL) includes deep neural network (DNN), convolutional neural network (CNN), and recurrent neural network (RNN). Here, the DT output is given to the DNN, LR output is subjected to CNN, and the NB output is given to the RNN, respectively. Eventually, the DNN, CNN, and RNN outputs are averaged to generate a successful outcome. The combination can be thought of as an Ensemble classifier. The weight of the RNN is optimally tuned through the Self Improved Shark Smell Optimization with Opposition Learning (SISSOOL) model to improve detection precision and accuracy. Lastly, the outcomes of the suggested approach are computed in terms of different measures.

Neural network and deep learning techniques are essential tools for data scientists when analyzing big data for forecasting and classification. In supervised learning, data sets are divided into training sets and test sets, and neural network repeatedly adjusts the weight of data to better represent the actual data. This book offers a practical guide to performing a neural network experiment with RapidMiner, which readers can follow step-by-step. For big data, especially non-linear data, deep learning can be employed. This chapter introduces two types of deep learning: convolutional neural networks (CNN) for picture analysis and recurrent neural networks (RNN) for sequential or time series data. The book provides a demonstration of both techniques using RapidMiner, making it accessible to readers who wish to deepen their understanding of these powerful tools.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Deep learning techniques have achieved significant improvements in malware analysis application, such as malware detection, malware classification and vulnerability mining. Convolutional neural network (CNN) is applied to extract local features of samples, while recurrent neural network (RNN) mainly extracts global features from sequence context. However, the pooling layer in CNN is a nonlinear dimensionality reduction operation, which can cause a loss of correlation feature. The raw RNN or LSTM model cannot perform parallel computing when calculating the feature vectors, limiting the effect of model application. Hence, we propose a novel malware classification model named BGSA (Bidirectional GRU with Self- Attention), which applies bi-direction gated recurrent unit (BiGRU) and self-attention mechanism. To demonstrate that the model can improve the feature extraction capability, a series of comparison experiments have been conducted on the VxHeavon dataset. The results show that the classification of BGSA reaches 92.54% in term of precision, which surpass the performance of method based on LSTM+CNN and BiGRU.

In this digital era, there is an exponential growth of text-based content in the electronic world. Data as texts exist in the form of documents, social media posts on Facebook, Twitter, etc., logs, sensor data, and emails. Twitter is a social platform where users express their views on various aspects in a day to day life. Twitter produces over 500 million tweets daily that is 6000 tweets per second. Twitter data is, by definition, very noisy and unstructured in nature. Text classifications based on the machine learning techniques have problems like poor generalization ability and sparsity dimension explosion. Classifiers based on deep learning techniques are implemented to improve accuracy to overcome shortcomings of machine learning techniques and to avoid feature extraction processes and have high prediction accuracy and strong learning ability. In this work, the classification of tweets is performed on Covid-19 dataset by implementing deep learning techniques namely Convolution Neural Network (CNN), Recurrent Neural Network (RNN), Recurrent Convolution Neural Network (RCNN), Recurrent Neural Network with Long Short Term Memory (RNN+LSTM), and Bidirectional Long Short Term Memory with Attention (BI-LSTM + Attention). The algorithms are implemented using two-word embedding techniques namely Global Vectors for Word Representation (GloVe) and Word2Vec. RNN with Bidirectional LSTM model has performed better than all the classifiers considered. It has classified the text with an accuracy of 93% and above when used with GloVe and Word2Vec.

In recent years, Deep Learning (DL), such as the algorithms of Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Generative Adversarial Networks (GAN), has been widely studied and applied in various fields including agriculture. Researchers in the fields of agriculture often use software frameworks without sufficiently examining the ideas and mechanisms of a technique. This article provides a concise summary of major DL algorithms, including concepts, limitations, implementation, training processes, and example codes, to help researchers in agriculture to gain a holistic picture of major DL techniques quickly. Research on DL applications in agriculture is summarized and analyzed, and future opportunities are discussed in this paper, which is expected to help researchers in agriculture to better understand DL algorithms and learn major DL techniques quickly, and further to facilitate data analysis, enhance related research in agriculture, and thus promote DL applications effectively. Keywords: deep learning, smart agriculture, neural network, convolutional neural networks, recurrent neural networks, generative adversarial networks, artificial intelligence, image processing, pattern recognition DOI: 10.25165/j.ijabe.20181104.4475 Citation: Zhu N Y, Liu X, Liu Z Q, Hu K, Wang Y K, Tan J L, et al. Deep learning for smart agriculture: Concepts, tools, applications, and opportunities. Int J Agric & Biol Eng, 2018; 11(4): 32-44.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

We develop novel approaches to enhance the functionality and efficiency of digital twins through deep learning techniques. Digital twins, sophisticated virtual models of physical systems, serve as dynamic counterparts, mirroring real-world entities' behavior and performance. Their primary role includes real-time monitoring, flaw detection, automated control, and proactive maintenance. To perform such tasks it is necessary to use various methods from scientific computing to combine data with physics models. Furthermore, to assess the reliability of a digital twin, it is important to not only make predictions, but also quantify the uncertainty of estimations and predictions. In this regard, Bayesian methods serve as a natural framework to pose the problems. However, the complexity and real-time operation requirements pose significant computational challenges. The thesis explores the integration of deep learning in scientific computing for enabling digital twins. The focus is on overcoming the computational hurdles in real-time data assimilation and inverse problem-solving through surrogate modeling. Deep learning, with its capability to handle high-dimensional functions in both deterministic and stochastic settings is shown to be a viable solution to these challenges. This work investigates several key areas, beginning with a brief introduction to digital twins, data assimilation, and inverse problems. It proceeds to elaborate on the role of Bayesian inversion problems in static and dynamic contexts, highlighting the importance of uncertainty quantification. The focus is on the challenges of real-time computation and the potential of Bayesian methods, despite their computational intensity. The thesis investigates deep learning architectures like dense and residual neural networks, convolutional neural networks, recurrent neural networks and transformers. The core contributions are presented in the four main chapters of the thesis: 1) Reduced Order Modeling for Parameterized Time-Dependent Partial Differential Equations: Introducing a deep learning framework for solving parameterized PDEs using spatially and memory-aware neural networks, demonstrated on linear advection and incompressible Navier-Stokes equations. 2) Markov Chain Generative Adversarial Neural Networks: Enhancing Bayesian inversion for static problems with sparse observations using a unique algorithm combining MCMC and GANs, demonstrated on a Darcy flow problem and pipe flow leakage detection. 3) Probabilistic Digital Twin for Leak Localization: Developing a probabilistic framework using supervised Wasserstein Autoencoders for leak localization in water distribution networks, tested on several network models. 4) The Deep Latent Space Particle Filter: Proposing a real-time nonlinear data assimilation method for dynamic PDEs with uncertainty quantification, using transformer-based dimensionality reduction and time-stepping in particle filters, demonstrated on various dynamic systems. This thesis contributes significantly to digital twins, addressing computational limitations and opening new avenues for practical applications in various industries.

Deep learning methodologies have had a major impact on performance across a wide variety of machine learning tasks, and speech recognition is no exception. We describe a set of deep learning techniques that proved to be particularly successful in achieving performance gains in word error rate on a popular large vocabulary conversational speech recognition benchmark task (“Switchboard”). We found that the best performance is achieved by combining features from both recurrent and convolutional neural networks. We compare two recurrent architectures: partially unfolded nets with max-out activations and bidirectional long short-term memory nets. In addition, inspired by the success of convolutional networks for image classification, we designed a convolutional net with many convolutional layers and small kernels that create a receptive field with more nonlinearity and fewer parameters than standard configurations. When combined, these neural networks achieve a word error rate of 6.2% on this difficult task; this was the best reported rate at the time of this writing and is even more remarkable given that human performance itself is estimated to be 4% on this data.

In recent decades, deep learning (DL) has become a rapidly growing research direction, redefining the state-of-the-art performances in a wide range of techniques, such as object detection and speech recognition. In the aircraft design, dynamics, and control field, many works hinge on the information-rich data-driven approach, which includes the fusion-based prognostic and health management, the airliner's flight safety monitoring, intelligent sensing, and flight control systems development. While DL provides great potentials to solve these data-driven problems, a systematic review and discussion as to how the DL has been/can be used for these problems are still missing in relation to the rapidly developing and widely used DL techniques. In this article, we aim to address this urgent issue to provide a timely overview of the state-of-the-art for applying DL to the aircraft design, dynamics, and control field. In particular, we briefly introduce five representative DL methods, i.e., deep neural network, deep autoencoder, deep belief network, convolutional neural network, and recurrent neural network. Mathematical definitions for each method are presented, and illustrative applications are also discussed. We then review the existing DL-based works that have appeared in the aircraft design, dynamics, and control field. The review efforts are divided into two major groups, i.e., the own-ship aircraft modeling, wherein the works have been/can be implemented online for the aircraft design/dynamics/control, and other airplanes research works, wherein DL-based schemes provide offline monitoring of the aircraft operation. We then summarize the data sources and DL architectures. Referring to the experiences of DL research works/techniques development in other related fields, future opportunities, challenges, and potential solutions for implementing DL in the aircraft design, dynamics, and control field are also discussed.

Personalized medicine represents a paradigm shift in healthcare, aiming to tailor treatment strategies to the unique genetic, environmental, and lifestyle characteristics of individual patients. This approach holds immense potential for improving therapeutic efficacy and minimizing adverse drug reactions. With the rapid advancement of artificial intelligence, deep learning has emerged as a transformative tool in pharmacology, enabling precise modeling of complex biological data and uncovering hidden patterns in patient-specific information. This study investigates the application of deep learning techniques - such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Transformer architectures, and Generative Adversarial Networks (GANs) - in optimizing personalized treatment strategies. Using a diverse dataset comprising electronic health records (EHRs), genomic sequences, and clinical indicators, we developed and trained deep learning models for tasks including drug response prediction, biomarker identification, and adverse drug reaction (ADR) forecasting. Among the models evaluated, Transformer-based architectures demonstrated superior performance, achieving an accuracy of 91.2% and an AUC-ROC of 0.92 in drug response prediction tasks. Moreover, the integration of deep learning models into the treatment pipeline resulted in a 20-30% improvement in drug-patient matching efficiency compared to traditional statistical methods. The findings underscore the potential of AI-powered systems to enhance clinical decision-making and enable precision pharmacotherapy. However, challenges such as data privacy, model interpretability, and regulatory compliance remain critical barriers to widespread adoption. The study also explores future directions, including the implementation of explainable AI (XAI) and federated learning, to address these limitations and facilitate the integration of deep learning into routine clinical practice.