An Improvement of Data Classification Using Random Multimodel Deep Learning (RMDL)

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Breast cancer is a significant health concern worldwide, and early detection plays a crucial role in improving patient outcomes.In recent years, deep learning approaches have shown promise in accurately predicting breast cancer, aiding in early diagnosis and treatment planning.This research paper presents a comprehensive study on breast cancer prediction using deep learning techniques.The study utilizes a large dataset of mammographic images collected from various medical institutions.Preprocessing techniques are employed to enhance the quality of the data, including image normalization and feature extraction.Several deep learning architectures, including convolutional neural networks (CNNs) and recurrent neural networks (RNNs), are explored to develop predictive models for breast cancer.

The automated detection of abusive content on social media websites faces a variety of challenges including imbalanced training sets, the identification of an appropriate feature representation and the selection of optimal classifiers. Classifiers such as support vector machines (SVM), combined with bag of words or ngram feature representation, have traditionally dominated in text classification for decades. With the recent emergence of deep learning and word embeddings, an increasing number of researchers have started to focus on deep neural networks. In this paper, our aim is to explore cutting-edge techniques in automated abusive content detection. We use two deep learning approaches: convolutional neural networks (CNNs) and recurrent neural networks (RNNs). We apply these to 9 public datasets derived from various social media websites. Firstly, we show that word embeddings pre-trained on the same data source as the subsequent classification task improves the prediction accuracy of deep learning models. Secondly, we investigate the impact of different levels of training set imbalances on classifier types. In comparison to the traditional SVM classifier, we identify that although deep learning models can outperform the classification results of the traditional SVM classifier when the associated training dataset is seriously imbalanced, the performance of the SVM classifier can be dramatically improved through the use of oversampling, surpassing the deep learning models. Our work can inform researchers in selecting appropriate text classification strategies in the detection of abusive content, including scenarios where the training datasets suffer from class imbalance.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Agriculture is considered to be a field of great importance and with a serious economic impact in all successful countries. Due to the substantial increase in world population, it has become a relevant concern to be able to meet people's daily dietary needs. Henceforth, it has become inevitable to make a transition to smart agricultural techniques to achieve the set food security goals. In recent times, several deep learning methods, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), have been vigorously studied, applied, and researched in different fields, including farming and agriculture. In this project, we aim at analyzing existing research on deep learning techniques in smart farming and agriculture and propose solutions for different aspects of farming using various deep learning architectures. Furthermore, we studied the farming parameters such as weather reports, plant irrigation information, pests that affect common crops, germination periods of the flowers/seeds, disease/anomaly detection in their leaves, etc., and proposed modular solutions for each of the respective areas of smart farming. Additionally, we also compared relevant studies regarding farming and focused agricultural methods, problems being faced, the method for collecting data being used, and the deep learning model suggested.

Convolutional neural network (CNN) and recurrent neural network (RNN) are two popular architectures used in text classification. Traditional methods to combine the strengths of the two networks rely on streamlining them or concatenating features extracted from them. In this paper, we propose a novel method to keep the strengths of the two networks to a great extent. In the proposed model, a convolutional neural network is applied to learn a 2D weight matrix where each row reflects the importance of each word from different aspects. Meanwhile, we use a bidirectional RNN to process each word and employ a neural tensor layer that fuses forward and backward hidden states to get word representations. In the end, the weight matrix and word representations are combined to obtain the representation in a 2D matrix form for the text. We carry out experiments on a number of datasets for text classification. The experimental results confirm the effectiveness of the proposed method.

Algorithms based on deep learning are taking off as solutions to several problems in numerous fields. Numerous deep learning architectures like convolutional neural networks, recurrent neural networks, deep belief networks, artificial neural networks and deep neural networks have been used in fields such as audio recognition, machine vision, processing of natural language, recognition of speech, social network filtering, computer vision, translating languages and bioinformatics. One such application of deep learning architectures is object detection. Object detection in satellite and aerial imagery undertakes an important role in a long list of applications and has allured a lot of attention which resulted in exceptional progress in recent years. It has abundant uses in various fields such as surveillance, military, national security, urban planning, environmental applications, traffic control and forestry. Several issues like varying size, background and orientation of the target object make the automatic detection of target in satellite imagery a challenging issue. Recent research in this field has given better speed and accuracy to the detection of the target object. With the rise in the data being generated by the Earth Observation programs, it is becoming much more imperative to find better and efficient ways to exploit this data by improving upon the existing Deep Learning research. In this paper, we bring forth a wide-ranging analysis of the recent advancements in methods of object detection based on deep learning to recognize the objects in satellite imagery.KeywordsDeep learningObject detectionSatellite imageryRemote sensingYou Only Look Once (YOLO)You Only Look Twice (YOLT)Single Shot Detector (SSD)Convolutional Neural Networks (CNN)Faster Region-based Convolutional Neural Networks (Faster RCNN)Hybrid Deep Convolutional Neural Network (HDNN)Satellite Imagery Multiscale Rapid Detection with Windowed Networks (SIMRDWN)

Deep learning (DL) is now generally acknowledged as the benchmark and evolution in machine learning (ML) fields. Further more, it has steadily suited the most extensively in computational techniques for ML, delivering excellent outcomes in several challenging intellectual works that have equal or still outperformed human ability. DL has the advantage of learning from enormous amounts of data. Here, the DL approach has been applied to classify thunderstorms and non-thunderstorms with a meteorological dataset. Daily observational hourly data sets from 2016 to 2021 have been used to classify the incidence of thunderstorms and non-thunderstorms. Three DL approaches are used: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) to classify thunderstorm incidences. These DL techniques are compared with conventional ML techniques, showing that DL approaches outperformed ML. The CNN method provides the highest performance with 92.01% accuracy among all DL and ML approaches.

This paper proposes a novel deep learning architecture involving combinations of Convolutional Neural Networks (CNN) layers and Recurrent neural networks (RNN) layers that can be used to perform segmentation and classification of 5 cardiac rhythms based on ECG recordings. The algorithm is developed in a sequence to sequence setting where the input is a sequence of five second ECG signal sliding windows and the output is a sequence of cardiac rhythm labels. The novel architecture processes as input both the spectrograms of the ECG signal as well as the heartbeats' signal waveform. Additionally, we are able to train the model in the presence of label noise. The model's performance and generalizability is verified on an external database different from the one we used to train. Experimental result shows this approach can achieve an average F1 scores of 0.89 (averaged across 5 classes). The proposed model also achieves comparable classification performance to existing state-of-the-art approach with considerably less number of training parameters.

IoT ecosystems generates vast amount of data from interconnected devices, sensors, and networks. This vast amount of data poses significant challenges for traditional way of analysis. Deep learning comes into picture, with its ability to model complex pattern and unstructured data, addressing these challenges and elevating the IoT systems to new levels of intelligence. The integration of deep learning with Internet of Things (IoT) ecosystems provides an advancement in creating the intelligent network of connected devices. Key deep learning architectures like convolutional neural network (CNN) and Recurrent Neural Network (RNN) excel at processing sensor data for visual inspection and environmental monitoring. Looking forward, advancement in improvised learning, explainable AI, and next generation networks like 5G/6G will further enhance the capabilities of intelligent IoT ecosystems. This chapter explains how deep learning architecture facilitates the applications of IoT.

In the era of big data, transformation of biomedical big data into valuable knowledge has been one of the most important challenges in bioinformatics. Deep learning has advanced rapidly since the early 2000s and now demonstrates state-of-the-art performance in various fields. Accordingly, application of deep learning in bioinformatics to gain insight from data has been emphasized in both academia and industry. Here, we review deep learning in bioinformatics, presenting examples of current research. To provide a useful and comprehensive perspective, we categorize research both by the bioinformatics domain (i.e. omics, biomedical imaging, biomedical signal processing) and deep learning architecture (i.e. deep neural networks, convolutional neural networks, recurrent neural networks, emergent architectures) and present brief descriptions of each study. Additionally, we discuss theoretical and practical issues of deep learning in bioinformatics and suggest future research directions. We believe that this review will provide valuable insights and serve as a starting point for researchers to apply deep learning approaches in their bioinformatics studies.

In recent years, successful applications of deep learning (DL) have inspired scientists to explore the possibilities of applying DL approaches to modeling scientific problems. Existing studies have revealed that to bake the physics into the DL makes a good supplement to the traditional numerical methods (e.g., finite element, finite volume method) which primarily rely on partial differential equations (PDEs). While DL models are ordinarily trained in a purely data-driven manner, integrating physics into them for simulating scientific problems has several benefits such as (i) physics constraints could regularize the over-parameterized model and hence mitigate the overfitting issue commonly seen in DL; (ii) physics information could also effectively reduce the amount of data needed for training the model; (iii) the resultant physics-informed DL models feature better interpretability and generalizability compared with the conventional black-box model. Furthermore, the powerful expressiveness of the deep network, guaranteed by the universal approximation theorem, makes it a suitable approximator for the solution to a physical system. In this dissertation, we develop two different DL architectures (or approaches), one being continuous scheme-based while the other discrete scheme-based, that leverage physics knowledge for modeling scientific problems. Through comprehensive numerical experiments, we demonstrate the proposed models can be used in solving general PDEs, establishing predictive data-driven models for dynamical systems, identifying the parameters in governing PDEs or even discovering the entire governing PDEs of dynamical systems from scarce and noisy measurements. The continuous model roots on the physics-informed neural network (PINN) which uses a fully connected neural network (FCNN) to approximate the physical fields of a system globally. This model is mesh-free as the residual of the physics (e.g., PDEs, initial/boundary values) is evaluated on a set of collocation points within the physical domain. Several applications including the forward simulations, data-driven simulations and solving inverse problems are presented to exemplify the advantages of PINN over traditional numerical methods. However, the original PINN suffers from inaccurate initial/boundary values due to the weak enforcement of the initial/boundary conditions (I/BCs). To overcome this issue, we propose an improved PINN model by utilizing multiple deep neural networks (DNNs) to construct the solution. Through a DNN pre-trained to represent the initial/boundary values, the approximated solution would obey the given I/BCs forcibly. With several numerical examples, we show that the improved PINN is characterized with much better accuracy on the I/BCs. Though the PINN shows great promise in data-driven modeling and solving inverse problems, some inherent limitations of PINN still exist, such as (i) the solution might lacks fine-scale details due to the global approximation of FCNN; (ii) high computational expense caused by the FCNN it roots on; (iii) incapability to incorporate existing PDE terms (e.g., $\Delta u$) into the network architecture. To overcome these drawbacks, this dissertation also proposes a discrete model - Physics-encoded Recurrent Convolutional Neural Network (PeRCNN) which recurrently updates the solution (or state variable) for time marching. Specifically, it utilizes convolutional neural network (CNN) to capture the spatial patterns of the solution while the recurrent network mimics the forward Euler scheme (or Runge-Kutta scheme) in numerical methods. PeRCNN is a mesh-based and discrete model due to the discretization in time and spatial dimension. The local connectivity of CNN makes PeRCNN more computationally efficient. In addition, the coercive encoding mechanism of physics in PeRCNN, fundamentally different from the PINN relying on soft penalty, ensures the network to rigorously obey given physics. The proposed PeRCNN is successfully applied to solving general PDEs, the data-driven modeling of dynamical systems and the data-driven discovery of governing PDEs from scarce and noisy measurements. Comparisons with the state-of-the-art DL models demonstrate that the proposed PeRCNN possesses excellent computational efficiency, accuracy and generalizability. --Author's abstract

Deep learning (DL) has shown explosive growth in its application to bioinformatics and has demonstrated thrillingly promising power to mine the complex relationship hidden in large-scale biological and biomedical data. A number of comprehensive reviews have been published on such applications, ranging from high-level reviews with future perspectives to those mainly serving as tutorials. These reviews have provided an excellent introduction to and guideline for applications of DL in bioinformatics, covering multiple types of machine learning (ML) problems, different DL architectures, and ranges of biological/biomedical problems. However, most of these reviews have focused on previous research, whereas current trends in the principled DL field and perspectives on their future developments and potential new applications to biology and biomedicine are still scarce. We will focus on modern DL, the ongoing trends and future directions of the principled DL field, and postulate new and major applications in bioinformatics.