Autonomous Malicious Video Content Categorization Using Convolutional Neural Networks

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Deep neural networks for texture classification—A theoretical analysis

We investigate the use of Deep Neural Networks for the classification of image datasets where texture features are important for generating class-conditional discriminative representations. To this end, we first derive the size of the feature space for some standard textural features extracted from the input dataset and then use the theory of Vapnik-Chervonenkis dimension to show that hand-crafted feature extraction creates low-dimensional representations which help in reducing the overall excess error rate. As a corollary to this analysis, we derive for the first time upper bounds on the VC dimension of Convolutional Neural Network as well as Dropout and Dropconnect networks and the relation between excess error rate of Dropout and Dropconnect networks. The concept of intrinsic dimension is used to validate the intuition that texture-based datasets are inherently higher dimensional as compared to handwritten digits or other object recognition datasets and hence more difficult to be shattered by neural networks. We then derive the mean distance from the centroid to the nearest and farthest sampling points in an n-dimensional manifold and show that the Relative Contrast of the sample data vanishes as dimensionality of the underlying vector space tends to infinity.

Methodologies to assess stem cell differentiation in the culturing state are needed for regenerative medicine and tissue engineering techniques. In recent years, convolutional neural networks (CNNs), a class of deep neural networks, have made impressive advancements in image-based classification, recognition and detection tasks. CNNs have been introduced as a non-invasive cell characterization method by learning features directly from image data of unlabeled cells. Furthermore, this approach serves as a rapid and inexpensive methodology with high performance compared to traditional techniques that require complex laboratory procedures including antibody staining and gene expression analysis. Here, we studied the potential of the CNNs approach to recognize stem cell differentiation based on cell morphology utilizing phasecontrast microscopy images.We have examined the differentiation potential of bone marrow mesenchymal stem cells (BMSCs) into tenocytes, with the treatment of bone morphogenetic protein-12 (BMP-12). After treatment, the phase-contrast images of cells were obtained directly from cell culture flasks to train CNN and the differentiated phenotype of stem cells was characterized by immunostaining. CNN was able to classify the cells into three groups including non-stem cells (chondrocytes), stem cells (BMSCs) and differentiated stem cells (tenocytes) based on their morphology with 92.2 % accuracy. The presented study revealed that CNN performed faster and non-invasive cell classification task compared to traditional methodologies.

Deep learning algorithms can be used to design systems that report data on patients and deliver warnings to medical applications or electronic health information if there are changes in the patient's health. These systems could be created using deep Learning. This helps verify that patients get the proper effective care at the proper time for each specific patient. A healthcare decision support system was presented using the Internet of Things and deep learning methods. In the proposed system, we examined the capability of integrating deep learning technology into automatic diagnosis and IoT capabilities for faster message exchange over the Internet. We have selected the suitable Neural Network structure (number of best-hidden layers and activation function classes) to construct the e-health system. In addition, the e-health system relied on data from doctors to understand the Neural Network. In the validation method, the total evaluation of the proposed healthcare system for diagnostics provides dependability under various patient conditions. Based on evaluation and simulation findings, a dual hidden layer of feed-forward NN and its neurons store the tanh function more effectively than other NN. To overcome challenges, this study will integrate artificial intelligence with IoT. This study aims to determine the NN's optimal layer counts and activation function variations.

The use of machine learning (ML) has been increasing rapidly in the medical imaging field, including computer-aided diagnosis (CAD), radiomics, and medical image analysis. Recently, an ML area called deep learning emerged in the computer vision field and became very popular in many fields. It started from an event in late 2012, when a deep-learning approach based on a convolutional neural network (CNN) won an overwhelming victory in the best-known worldwide computer vision competition, ImageNet Classification. Since then, researchers in virtually all fields, including medical imaging, have started actively participating in the explosively growing field of deep learning. In this paper, the area of deep learning in medical imaging is overviewed, including (1) what was changed in machine learning before and after the introduction of deep learning, (2) what is the source of the power of deep learning, (3) two major deep-learning models: a massive-training artificial neural network (MTANN) and a convolutional neural network (CNN), (4) similarities and differences between the two models, and (5) their applications to medical imaging. This review shows that ML with feature input (or feature-based ML) was dominant before the introduction of deep learning, and that the major and essential difference between ML before and after deep learning is the learning of image data directly without object segmentation or feature extraction; thus, it is the source of the power of deep learning, although the depth of the model is an important attribute. The class of ML with image input (or image-based ML) including deep learning has a long history, but recently gained popularity due to the use of the new terminology, deep learning. There are two major models in this class of ML in medical imaging, MTANN and CNN, which have similarities as well as several differences. In our experience, MTANNs were substantially more efficient in their development, had a higher performance, and required a lesser number of training cases than did CNNs. "Deep learning", or ML with image input, in medical imaging is an explosively growing, promising field. It is expected that ML with image input will be the mainstream area in the field of medical imaging in the next few decades.

Due to the advent of modern embedded systems and mobile devices with constrained resources, there is a great demand for incredibly efficient deep neural networks for machine learning purposes. There is also a growing concern of privacy and confidentiality of user data within the general public when their data is processed and stored in an external server which has further fueled the need for developing such efficient neural networks for real-time inference on local embedded systems. The scope of our work presented in this paper is limited to image classification using a convolutional neural network. A Convolutional Neural Network (CNN) is a class of Deep Neural Network (DNN) widely used in the analysis of visual images captured by an image sensor, designed to extract information and convert it into meaningful representations for real-time inference of the input data. In this paper, we propose a neoteric variant of deep convolutional neural network architecture to ameliorate the performance of existing CNN architectures for real-time inference on embedded systems. We show that this architecture, dubbed CondenseNeXt, is remarkably efficient in comparison to the baseline neural network architecture, CondenseNet, by reducing trainable parameters and FLOPs required to train the network whilst maintaining a balance between the trained model size of less than 3.0 MB and accuracy trade-off resulting in an unprecedented computational efficiency.

Security researchers have used Natural Language Processing (NLP) and Deep Learning techniques for programming code analysis tasks such as automated bug detection and vulnerability prediction or classification. These studies mainly generate the input vectors for the deep learning models based on the NLP embedding methods. Nevertheless, while there are many existing embedding methods, the structures of neural networks are diverse and usually heuristic. This makes it difficult to select effective combinations of neural models and the embedding techniques for training the code vulnerability detectors. To address this challenge, we extended a benchmark system to analyze the compatibility of four popular word embedding techniques with four different neural networks, including the standard Bidirectional Long Short-Term Memory (Bi-LSTM), the Bi-LSTM applied attention mechanism, the Convolutional Neural Network (CNN), and the classic Deep Neural Network (DNN). We trained and tested the models by using two types of vulnerable function datasets written in C code. Our results revealed that the Bi-LSTM model combined with the FastText embedding technique showed the most efficient detection rate on a real-world but not on an artificially constructed dataset. Further comparisons with the other combinations are also discussed in detail in our result.

Fluctuating wave force on a bluff body is of great significance in many offshore and marine engineering applications. We present a Convolutional Neural Network (CNN) based data-driven computing to predict the unsteady wave forces on bluff bodies due to the free-surface wave motion. For the full-order modeling and high-fidelity data generation, the air-water interface for such wave-body problems must be captured accurately for a broad range of physical and geometric parameters. Originated from the thermodynamically consistent theories, the physically motivated Allen-Cahn phase-field method has many advantages over other interface capturing techniques such as level-set and volume-of-fluid methods. The Allen-Cahn equation is solved in the mass-conservative form by imposing a Lagrange multiplier technique. While a tremendous amount of wave-body interaction data is generated in offshore engineering via both CFD simulations and experiments, the results are generally underutilized. Design space exploration and flow control of such practical scenarios are still time-consuming and expensive. An alternative to semi-analytical modeling, CNN is a class of deep neural network for solving inverse problems which is efficient in parametric data-driven computation and can use the domain knowledge. It establishes a model with arbitrarily generated model parameters, makes predictions using the model and existing input parametric settings, and adjusts the model parameters according to the error between the predictions and existing results. The computational cost of this prediction process, compared with high-fidelity CFD simulation, is significantly reduced, which makes CNN an accessible tool in design and optimization problems. In this study, CNN-based data-driven computing is utilized to predict the wave forces on bluff bodies with different geometries and distances to the free surface. The discrete convolution process with a non-linear rectification is employed to approximate the mapping between the bluff-body shape, the distance to the free-surface and the fluid forces. The wave-induced fluid forces on bluff bodies of different shapes and submergences are predicted by the trained CNN. Finally, a convergence study is performed to identify the effective hyper-parameters of the CNN such as the convolution kernel size, the number of kernels and the learning rate. Overall, the proposed CNN-based approximation procedure has a profound impact on the parametric design of bluff bodies experiencing wave loads.

As artificial intelligence continues to develop new applications in ophthalmic image recognition, we provide here an introduction for ophthalmologists and a primer on the mechanisms of deep learning systems. Deep learning has lent itself to the automated interpretation of various retinal imaging modalities, including fundus photography and optical coherence tomography. Convolutional neural networks (CNN) represent the primary class of deep neural networks applied to these image analyses. These have been configured to aid in the detection of diabetes retinopathy, AMD, retinal detachment, glaucoma, and ROP, among other ocular disorders. Predictive models for retinal disease prognosis and treatment are also being validated. Deep learning systems have begun to demonstrate a reliable level of diagnostic accuracy equal or better to human graders for narrow image recognition tasks. However, challenges regarding the use of deep learning systems in ophthalmology remain. These include trust of unsupervised learning systems and the limited ability to recognize broad ranges of disorders.

Deep learning is a fast flattering highest development in the machine learning technique leading to augment performance in diverse medical applications. The tributary artificial intelligence is rapidly becoming metamorphic in the domain of Healthcare Technology rendering facility to analyze the medical data with high rapidity and exactness. Due to the overwhelming development of artificial intelligence (AI), the contemporary encroachments of deep learning employing advanced and futuristic deep learning methods has become a vivacious research domain for processing medical data in the health care industries. This paper presents a complete analysis about the deep structured learning based methods used in medical image analysis and also elucidate about the advancement of convolutional neural network which is a major class of deep neural networks used in medical image processing. It also gives insight about the fundamentals of deep learning methods and reviews in image interpretation. It also explains about the implementation of an integrated framework for medical image synthesis combining Speckle GAN and Di-Conv-AE-Net.

In the field of big data analytics, deep learning has garnered a lot of interest due to the fact that it is capable of doing feature extraction and classification. The categorization of big data has traditionally been carried out by academics via the use of Machine Learning methods; nonetheless, the process of feature extraction has traditionally been carried out manually. Deep learning is a method developed by researchers that allows for the extraction of features via the use of various algorithms. This research made use of an algorithm known as "Convolutional Neural Network," or CNN for short, in order to get a deeper comprehension of the dependability of big data analytics. Primary research has been conducted in order to get an understanding of how hidden nodes and layers influence the accuracy of this neural network. The "Recurrent Neural Network" (also known as RNN) and the "Artificial Neural Network" (also known as ANN) are the two other neural network algorithms that have been selected for comparison with CNN in order to evaluate whether or not CNN is more effective than the other algorithms. In the first research, the reliability of CNN as well as other methods was analyzed in relation to the number of nodes, the number of hidden layers, the amount of training time, and the amount of validating time. Control variables were employed in both correlation and regression investigations. These variables included Hidden Nodes, Training time, Valid time, and Secret Layers. CNN, ANN, and RNN were significant variables. According to the findings, CNN is more dependable than other neural networks for analyzing huge amounts of data, scoring 92%, whereas other neural networks scored less than 90%. The inclusion of hidden nodes results in a significant increase in CNN's reliability.

Simple SummaryBreast cancer is one of the leading causes of cancer death among women. Developing machine learning-based diagnosis models receives great attention from researchers and scientists using histopathology images. Deep learning (DL) algorithms automatically extract features from raw data through convolutional operations. The generalization of the DL models’ results relies on large datasets, although they eliminate the expert knowledge in the feature extraction stage. This work aimed to compare the performance of the features extracted via deep learning and a knowledge-based approach in breast cancer detection from histopathology images.Cancer develops when a single or a group of cells grows and spreads uncontrollably. Histopathology images are used in cancer diagnosis since they show tissue and cell structures under a microscope. Knowledge-based and deep learning-based computer-aided detection is an ongoing research field in cancer diagnosis using histopathology images. Feature extraction is vital in both approaches since the feature set is fed to a classifier and determines the performance. This paper evaluates three feature extraction methods and their performance in breast cancer diagnosis. Features are extracted by (1) a Convolutional Neural Network, (2) a transfer learning architecture VGG16, and (3) a knowledge-based system. The feature sets are tested by seven classifiers, including Neural Network (64 units), Random Forest, Multilayer Perceptron, Decision Tree, Support Vector Machines, K-Nearest Neighbors, and Narrow Neural Network (10 units) on the BreakHis 400× image dataset. The CNN achieved up to 85% for the Neural Network and Random Forest, the VGG16 method achieved up to 86% for the Neural Network, and the knowledge-based features achieved up to 98% for Neural Network, Random Forest, Multilayer Perceptron classifiers.

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.