Network Traffic Classification Based on Deep Learning

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Malicious traffic classification has become a challenge in modern communications. It is a very important task for a trained model to successfully distinguish malicious traffic. With the gradual application of machine learning and deep learning in the field of traffic classification, traffic classification has reached a high accuracy rate. Feature selection can lighten models and improve classification performance by selecting the optimal sub-feature set. Therefore, the selection of effective features is an important issue for malicious traffic classification. In this article, we propose the idea of applying feature selection methods Information Gain and RFE to malicious traffic classification. The essence is to select an effective and optimal sub-feature set from a large number of features to characterize network traffic. Then, we used the deep learning method CNN and the machine learning method RF on the three real network traffic datasets of CICIDS2017, NSL-KDD and UNSW-NB15 respectively to evaluate and verify. The experiment shows that the combination of CNN and Information Gain has the best effect. The results of many experiments show that the performance of traffic classification is greatly improved after feature selection.

Recently, with the advent of various Internet of Things (IoT) applications, a massive amount of network traffic is being generated. A network operator must provide different quality of service, according to the service provided by each application. Toward this end, many studies have investigated how to classify various types of application network traffic accurately. Especially, since many applications use temporary or dynamic IP or Port numbers in the IoT environment, only payload-based network traffic classification technology is more suitable than the classification using the packet header information as well as payload. Furthermore, to automatically respond to various applications, it is necessary to classify traffic using deep learning without the network operator intervention. In this study, we propose a traffic classification scheme using a deep learning model in software defined networks. We generate flow-based payload datasets through our own network traffic pre-processing, and train two deep learning models: 1) the multi-layer long short-term memory (LSTM) model and 2) the combination of convolutional neural network and single-layer LSTM models, to perform network traffic classification. We also execute a model tuning procedure to find the optimal hyper-parameters of the two deep learning models. Lastly, we analyze the network traffic classification performance on the basis of the F1-score for the two deep learning models, and show the superiority of the multi-layer LSTM model for network packet classification.

The origin of tobacco is the most important factor in determining the style characteristics and intrinsic quality of tobacco. There are many applications for the identification of tobacco origin by near-infrared spectroscopy. In order to improve the accuracy of the tobacco origin classification, a near-infrared spectrum (NIRS) identification method based on multimodal convolutional neural networks (CNN) was proposed, taking advantage of the strong feature extraction ability of the CNN. Firstly, the one-dimensional convolutional neural network (1-D CNN) is used to extract and combine the pattern features of one-dimensional NIRS data, and then the extracted features are used for classification. Secondly, the one-dimensional NIRS data are converted into two-dimensional spectral images, and the structure features are extracted from two-dimensional spectral images by the two-dimensional convolutional neural network (2-D CNN) method. The classification is performed by the combination of global and local training features. Finally, the influences of different network structure parameters on model identification performance are studied, and the optimal CNN models are selected and compared. The multimodal NIR-CNN identification models of tobacco origin were established by using NIRS of 5,200 tobacco samples from 10 major tobacco producing provinces in China and 3 foreign countries. The classification accuracy of 1-D CNN and 2-D CNN models was 93.15% and 93.05%, respectively, which was better than the traditional PLS-DA method. The experimental results show that the application of 1-D CNN and 2-D CNN can accurately and reliably distinguish the NIRS data, and it can be developed into a new rapid identification method of tobacco origin, which has an important promotion value.

Parkinson's disease (PD) is the second most prevalent neurodegenerative disorder in the world, and it is characterized by the production of different motor and non-motor symptoms which negatively affect speech and language production. For decades, the research community has been working on methodologies to automatically model these biomarkers to detect and monitor the disease; however, although speech impairments have been widely explored, language remains underexplored despite being a valuable source of information, especially to assess cognitive impairments associated with non-motor symptoms. This study proposes the automatic assessment of PD patients using different methodologies to model speech and language biomarkers. One-dimensional and two-dimensional convolutional neural networks (CNNs), along with pre-trained models such as Wav2Vec 2.0, BERT, and BETO, were considered to classify PD patients vs. Healthy Control (HC) subjects. The first approach consisted of modeling speech and language independently. Then, the best representations from each modality were combined following early, joint, and late fusion strategies. The results show that the speech modality yielded an accuracy of up to 88%, thus outperforming all language representations, including the multi-modal approach. These results suggest that speech representations better discriminate PD patients and HC subjects than language representations. When analyzing the fusion strategies, we observed that changes in the time span of the multi-modal representation could produce a significant loss of information in the speech modality, which was likely linked to a decrease in accuracy in the multi-modal experiments. Further experiments are necessary to validate this claim with other fusion methods using different time spans.

Virtual reality (VR) can potentially enhance student engagement and memory retention in the classroom. However, distraction among participants in a VR-based classroom is a significant concern. Several factors, including mind wandering, external noise, stress, etc., can cause students to become internally and/or externally distracted while learning. To detect distractions, single or multi-modal features can be used. A single modality is found to be insufficient to detect both internal and external distractions, mainly because of individual variability. In this work, we investigated multi-modal features: eye tracking and EEG data, to classify the internal and external distractions in an educational VR environment. We set up our educational VR environment and equipped it for multi-modal data collection. We implemented different machine learning (ML) methods, including k-nearest-neighbors (kNN), Random Forest (RF), one-dimensional convolutional neural network - long short-term memory (1 D-CNN-LSTM), and two-dimensional convolutional neural networks (2D-CNN) to classify participants' internal and external distraction states using the multi-modal features. We performed cross-subject, cross-session, and gender-based grouping tests to evaluate our models. We found that the RF classifier achieves the highest accuracy over 83% in the cross-subject test, around 68% to 78% in the cross-session test, and around 90% in the gender-based grouping test compared to other models. SHAP analysis of the extracted features illustrated greater contributions from the occipital and prefrontal regions of the brain, as well as gaze angle, gaze origin, and head rotation features from the eye tracking data.

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

A channel modeling method and deep-learning-based symbol decision method are proposed to improve the performance of a visual MIMO system for communication between a variable-color LED array and camera. Although image processing algorithms using color clustering are available to correct distorted color information in a channel, color-similarity-based approaches are limited by real-world distortions; to overcome such limitations, symbol decision is defined as a multiclass classification problem. Further, to learn a robust classifier against channel distortion, a deep neural network learning technique is applied to adaptively determine symbols from channel distortion. The network designed herein comprises the channel identification and symbol decision modules; the channel identification module extracts a channel identification vector for symbol determination from an input image using a two-dimensional deep convolutional neural network (CNN); the symbol decision module then generates a feature map by combining the channel identification vector and information on adjacent symbols to determine the symbol via learning correlations between adjacent symbols using a one-dimensional CNN. The two modules are connected together and learned simultaneously in an end-to-end manner. We also propose a new channel modeling method that intuitively reflects real-world distortion factors rather than the conventional additive white Gaussian noise channel to efficiently train deep-learning networks. Lastly, in the proposed channel distortion environment, the proposed method shows performance improvement by an average of about 41.8% (up to about 54.8%) compared to the existing Euclidean distance method, and about 6.3% (up to about 9.2%) on average compared to the SVM method.

Breast cancer or ductal carcinoma is prevalent in women and is the leading cause of women's death worldwide. Delaying breast cancer or tumor growth has long-term effects, can also become life-threatening. So the tumor should be identified as early as possible to control the growth and preventing to spread to other tissues. Several types of researches have been done to detect breast cancer early so that the treatment can be started to increase the chance of survival. A mammography procedure for early detection and diagnosis of breast cancer is commonly advised. There are so many techniques that are used for malignancy prediction. Using Artificial Intelligence-powered machine learning is widely recommended. Researches are conducted in machine learning to detect cancerous tumors in the human body. Mainly used algorithms which give high accuracy are SVM, naïve bays, decision trees, KNN. Deep learning, the machine learning sub-branch, can also be used to classify breast cancer. Deep learning is a method that is mostly used to clear, rectify, and detect machine learning errors or disadvantages. Convolutional Neural Networks are the perfect deep learning method for overcoming the drawbacks of machine learning in malignancy detection; however, other strategies such as a recurrent neural network and a deep belief network are being used to overcome the shortcomings of machine learning. As a consequence, using deep learning rather than machine learning yields better results. This review paper's primary motivation is to make budding researchers aware that breast cancer is a serious issue among women and we need to be swift in using different technologies to detect and to improve accuracy as efficiently as possible, it is very important to save our mothers, sisters, loved ones and our society from this dangerous predator. In this review paper, we have discussed all the techniques used by different authors.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Parkinson's disease (PD) is a progressive neurodegenerative disorder affecting millions worldwide. Early detection of PD is crucial for effective treatment and management of the disease. Deep learning (DL) and machine learning (ML) have emerged as promising approaches for detecting PD. In this study, a comparative performance analysis is done for DL and ML applications based on speech signals. DL methods using convolutional neural networks (CNNs), long short-term memory (LSTM) networks, and ML methods employing random forest and the XGBoost model were trained and assessed. Performance of the models are evaluated using a variety of performance metrics, including accuracy, precision, recall, and F1-score. Results showed that the XGBoost model outperformed the DL models in terms of accuracy and F1 score, while the CNN and LSTM models achieved higher precision and recall. These findings suggest that XGBoost can be a useful tool for detecting PD based on speech signals, particularly in scenarios where interpretability and computational efficiency are important.

In this paper, we develop a method based on a deep learning method in financial market prediction, which includes BRICS economies as the test cases. Financial markets are rife with volatility that is affected by a "bed of complexity," coddled by local and distal factors. To leverage these vast datasets both deep learning models such as Convolutional Neural Networks (CNNs), Long Short Term Memory (LSTM) networks as well as hybrid architectures are used in this study. The paper evaluates the predictive accuracy of the models, and by so doing, identifies their strengths in predicting temporal dependencies and intricate market patterns. In particular, deep learning techniques are applied to case studies of individual countries in the BRICS to highlight the application of deep learning to disparate country specific problems, such as liquidity crises and market shocks. These findings show that classical statistical methods are outperformed by deep learning systems in a precise and reliable financial forecasting. This research highlights the ability of AI driven systems to change financial decision making processes, improving investor confidence and improving economic stability in BRICS nations. This study also readers the value of deep learning in financial market analysis, especially in economies in the developing countries. Application of techniques and architectures e.g. Convolutional Neural Networks (CNNs) that excel at identifying spatial patterns, and Long Short-Term Memory (LSTM) networks renowned for their prowess on sequential and time series data, for real world market prediction are explained. In addition, the research discusses hybrid architectures which extend knowledge, fusing strengths of both architectures to improve prediction accuracy and how deep learning develops to solve particular financial challenges. Through reading these notes readers get exposed to data preprocessing techniques such as normalization and feature selection which are important for boosting deep learning performance. The paper also includes an introduction to the evaluation of models using MSE and R-squared values for validating them in terms of reliable outputs. This research combines deep learning theory and practical case study to offer a useful educational resource for students, researchers, and practitioners who want to apply AI in financial forecasting in complex and dynamic global markets.

This study discusses convolutional neural networks (CNNs) for vibration signals analysis, including applications in machining surface roughness estimation, bearing faults diagnosis, and tool wear detection. The one-dimensional CNNs (1DCNN) and two-dimensional CNNs (2DCNN) are applied for regression and classification applications using different types of inputs, e.g., raw signals, and time-frequency spectra images by short time Fourier transform. In the application of regression and the estimation of machining surface roughness, the 1DCNN is utilized and the corresponding CNN structure (hyper parameters) optimization is proposed by using uniform experimental design (UED), neural network, multiple regression, and particle swarm optimization. It demonstrates the effectiveness of the proposed approach to obtain a structure with better performance. In applications of classification, bearing faults and tool wear classification are carried out by vibration signals analysis and CNN. Finally, the experimental results are shown to demonstrate the effectiveness and performance of our approach.

One of the major challenges in using photoplethysmography (PPG) sensors for heart rate monitoring in real-world settings is ensuring signal quality. This work evaluates and compares machine learning (ML) and deep learning (DL) methods for PPG quality assessment on a unique and comprehensive framework that includes different sensors, wavelengths, body measurement locations, and recording environments. The PPG signals from one proprietary and five publicly available datasets were labeled in terms of quality by comparing the PPG-derived heart rate to a reference heart rate estimated from simultaneous electrocardiograms. Diverse techniques based on common ML classifiers and one- and two-dimensional convolutional neural networks (CNN) were trained on a dataset and tested on the remaining ones. The results showed that several generated models performed comparably to previous studies when they were tested on datasets with similar measurement positions and sensors to the training database. Specifically, reductions in sensitivity, specificity, and F<sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub>-score of less than 3% from training to testing were observed on some methods. Contrarily, they reported a notably poorer performance when tested on datasets presenting conditions different from the training. Even the best-performing model, based on the well-known, pre-trained CNN AlexNet, experienced a performance drop of over 20% in that situation. These findings show that the analyzed ML and DL methods lack the ability to generalize across PPG signals captured from diverse environments, sensors, wavelengths, and measurement locations.

Wearable electroencephalography (EEG) devices for seizure detection accuracy and reliability are deep learning (DL) applications in the field of epilepsy diagnosis. In this study, we sought to increase the accuracy of seizure detection using advanced DL algorithms on the Children’s Hospital Boston - Massachusetts Institute of Technology (CHB-MIT) EEG database. First, a fully convolutional network (FCN) was trained and assessed using accuracy and recall/precision metrics, and the early stopping technique was used to avoid overfitting. To assess the performance, the FCN was evaluated in terms of various metrics, including accuracy, precision, recall, F1-score, and receiver operating characteristic (ROC)-area under the curve (AUC). In addition, two-dimensional (2D) convolutional neural networks (CNNs) and long short-term memory (LSTM) models were used to model the database, and their performance was thoroughly measured using different metrics, graphs, and confusion matrices. Using LSTM variants, such as standard LSTM, bidirectional LSTM, stacked LSTM, and LSTM attention mechanisms, hybrid convolutional LSTM (ConvLSTM) models were trained and compared. The comparison was conducted based on the training and validation accuracy and loss, as well as the graphs resulting from the precision–recall curves. Apart from DL approaches, EEG signal analysis using time–frequency techniques, such as wavelet transform and short-time Fourier transform, has also been investigated. These methods assisted in the analysis of the time–frequency features of EEG signals in combination with DL models. This study demonstrates that the performance of wearable EEG devices can be augmented using a combination of DL and seizure signal processing techniques. The FCN achieved an accuracy of 92%, with a recall for seizures of 33%, an F1-score of 0.03, and strong ROC-AUC results. The 2D CNN achieved 96% accuracy, a seizure recall of 70%, an F1-score of 0.12, and an ROC-AUC score of 78%. The baseline LSTM struggled with effectiveness at 53% accuracy with a seizure recall of 18%. In contrast, the LSTM model, which incorporated synthetic minority oversampling technique (SMOTE) balancing, was able to reach improvements of up to 89% accuracy, with a precision of 91%, a recall of 86%, an F1-score of 0.89, and a strong ROC curve performance. Among the models, the LSTM with SMOTE was the best performer, with 89% accuracy, 91% precision, 86% recall, and an F1-score of 0.89. These results provide evidence that applying techniques for data balancing in combination with certain DL network architectures significantly improves the detection of seizures using wearable EEG devices worn on the body. We believe that real-time monitoring and high-performance systems are feasible using optimized DL frameworks. The analysis of the performance of different models allows for the understanding of the possibilities of optimizing the architectures of DL algorithms for the modern diagnosis of epilepsy in real time. The source code used to carry out the experiments is publicly available at CHB-MIT EEG Dataset Python Scripts (https://www.kaggle.com/code/adnankust/adnaneeg1).