A survey of deep learning-driven architecture for predictive maintenance

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep structured learning or hierarchical learning or deep learning in short is part of the family of machine learning methods which are themselves a subset of the broader field of Artificial Intelligence. Each successive layer uses the output from the previous layer as input. Deep neural networks, deep belief networks and recurrent neural networks have been applied to fields such as computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, and bioinformatics where they produced results comparable to and in some cases better than human experts have. CNN are applied for deep feature extraction and LSTM is used to detect the class based on extracted features. Deep learning technique that would sort the fruit into abnormal and normal based on the feature such as fruit color, number of fruit spots , and shapes of the fruit, The proposed system achieved an accuracy of 98.17%, The susceptibility of disease detection was less with lesser availability of enhanced detection methods for detecting disease in earlier stages. The issue with various existing algorithms is that the accuracy was reduced so some problems not eliminated. Deep learning delivers methodologies, approaches and functionalities that can help to resolve analytic and predictive analysis accurately.

In recent decades, deep learning (DL) has become a rapidly growing research direction, redefining the state-of-the-art performances in a wide range of techniques, such as object detection and speech recognition. In the aircraft design, dynamics, and control field, many works hinge on the information-rich data-driven approach, which includes the fusion-based prognostic and health management, the airliner's flight safety monitoring, intelligent sensing, and flight control systems development. While DL provides great potentials to solve these data-driven problems, a systematic review and discussion as to how the DL has been/can be used for these problems are still missing in relation to the rapidly developing and widely used DL techniques. In this article, we aim to address this urgent issue to provide a timely overview of the state-of-the-art for applying DL to the aircraft design, dynamics, and control field. In particular, we briefly introduce five representative DL methods, i.e., deep neural network, deep autoencoder, deep belief network, convolutional neural network, and recurrent neural network. Mathematical definitions for each method are presented, and illustrative applications are also discussed. We then review the existing DL-based works that have appeared in the aircraft design, dynamics, and control field. The review efforts are divided into two major groups, i.e., the own-ship aircraft modeling, wherein the works have been/can be implemented online for the aircraft design/dynamics/control, and other airplanes research works, wherein DL-based schemes provide offline monitoring of the aircraft operation. We then summarize the data sources and DL architectures. Referring to the experiences of DL research works/techniques development in other related fields, future opportunities, challenges, and potential solutions for implementing DL in the aircraft design, dynamics, and control field are also discussed.

Alzheimer's Disease (AD) is a serious neurological illness that causes memory loss gradually by destroying brain cells. This deadly brain illness primarily strikes the elderly, impairing their cognitive and bodily abilities until brain shrinkage occurs. Modern techniques are required for an accurate diagnosis of AD. Machine learning has gained attraction in the medical field as a means of determining a person's risk of developing AD in its early stages. One of the most advanced soft computing neural network-based Deep Learning (DL) methodologies has garnered significant interest among researchers in automating early-stage AD diagnosis. Hence, a comprehensive review is necessary to gain insights into DL techniques for the advancement of more effective methods for diagnosing AD. This review explores multiple biomarkers associated with Alzheimer's Disease (AD) and various DL methodologies, including Deep Neural Networks (DNN), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), The k-nearest-neighbor (k-NN), Deep Boltzmann Machines (DBM), and Deep Belief Networks (DBN), which have been employed for automating the early diagnosis of AD. Moreover, the unique contributions of this review include the classification of ATN biomarkers for Alzheimer's Disease (AD), systemic description of diverse DL algorithms for early AD assessment, along with a discussion of widely utilized online datasets such as ADNI, OASIS, etc. Additionally, this review provides perspectives on future trends derived from critical evaluation of each variant of DL techniques across different modalities, dataset sources, AUC values, and accuracies.

Algorithms based on deep learning are taking off as solutions to several problems in numerous fields. Numerous deep learning architectures like convolutional neural networks, recurrent neural networks, deep belief networks, artificial neural networks and deep neural networks have been used in fields such as audio recognition, machine vision, processing of natural language, recognition of speech, social network filtering, computer vision, translating languages and bioinformatics. One such application of deep learning architectures is object detection. Object detection in satellite and aerial imagery undertakes an important role in a long list of applications and has allured a lot of attention which resulted in exceptional progress in recent years. It has abundant uses in various fields such as surveillance, military, national security, urban planning, environmental applications, traffic control and forestry. Several issues like varying size, background and orientation of the target object make the automatic detection of target in satellite imagery a challenging issue. Recent research in this field has given better speed and accuracy to the detection of the target object. With the rise in the data being generated by the Earth Observation programs, it is becoming much more imperative to find better and efficient ways to exploit this data by improving upon the existing Deep Learning research. In this paper, we bring forth a wide-ranging analysis of the recent advancements in methods of object detection based on deep learning to recognize the objects in satellite imagery.KeywordsDeep learningObject detectionSatellite imageryRemote sensingYou Only Look Once (YOLO)You Only Look Twice (YOLT)Single Shot Detector (SSD)Convolutional Neural Networks (CNN)Faster Region-based Convolutional Neural Networks (Faster RCNN)Hybrid Deep Convolutional Neural Network (HDNN)Satellite Imagery Multiscale Rapid Detection with Windowed Networks (SIMRDWN)

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .

Because of the two sequenced methods stated above, SG and AMP, are being used in different ways, present a deep learning methodology for taxonomic categorization of the metagenomic information which could be utilized for either. To place the suggested pipeline to a trial, 1000 16 S full-length genomes were used to generate either SG or AMP short-reads. Then, to map sequencing as matrices into such a number space, used a k-mer model. Our analysis of the existing approaches revealed several drawbacks, including limited ability to handle complex hierarchical representations of data and suboptimal feature extraction from grid-like structures. To overcome these limitations, we introduce DBNs for feature learning and dimensionality reduction, and CNNs for efficient processing of grid-like metagenomic data. Finally, a training set for every taxon was obtained by training two distinct deep learning constructions, specifically deep belief network (DBN) and convolutional neural network (CNN). This examined the proposed methodology to determine the best factor that determines and compared findings to the classification abilities offered by the RDP classifier, a standard classifier for bacterium identification. These designs outperform using RDP classifiers at every taxonomic level. So, at the genetic level, for example, both CNN and DBN achieved 91.4% accuracy using AMP short-reads, but the RDP classifier achieved 83.9% with the same information. This paper, suggested a classification method for 16 S short-read sequences created on k-mer representations and a deep learning structure, that every taxon creates a classification method. The experimental findings validate the suggested pipelines as a realistic strategy for classifying bacterium samples; as a result, the technique might be included in the most commonly used tools for the metagenomic research. According to the outcomes, it could be utilized to effectively classify either SG or AMP information.

A Novel Predictive Selective Maintenance Strategy Using Deep Learning and Mathematical Programming

Handwritten digit recognition is the intelligence of computers to recognize digits written by humans. But it becomes one of the most challenging tasks for machines as handwritten digits are not perfect and can be made with many different: flavors, size, thickness. Thus, as a solution to this problem, Handwriting digit recognition model comes into picture. Many machine learning techniques have been employed to solve the handwritten digit recognition problem. This paper focuses on Neural Network (NN) approaches. Among the three famous NN approaches: deep neural network (DNN), deep belief network (DBN) and convolutional neural network (CNN), the specialization of CNN as compared to other NN of being able to detect pattern is what makes it so useful for recognizing handwritten digits. Humans can very easily see, read &amp; write any handwritten digits, when written in proper format. Even if the digits are not written in proper format we can use our logic and predict what digit it could be. But It is a hard task for the machine to recognize handwritten digits as these are not perfect and can be made with many different flavors. Thus handwritten digit recognition is the solution to this problem which uses the image of a digit and recognizes the digit present in the image. The handwritten digit recognition is the ability of computers to recognize human handwritten digits. The goal is to build a model that can efficiently and reliably recognize the digits and output the proper result. Amongst all the other neural networks, working and implementing a model using Convolution Neural Network gives out the most precise results. It is most popularly used for analyzing images as well as for other data analysis or classification problems. CNN has hidden layers called convolutional layers. These layers work the same way as other layers do but here we need to specify the no of filters each layer should have These filters are actually what detects the pattern. Patterns could be edges, corners, circles or any complex other objects like eyes, ears or even deeper full dogs, cats, etc. Thus, the specialization of CNN as compared to other NN of being able to detect patterns is what makes it so useful for recognizing handwritten digits Key Word: Neural Networks; Convolutional Neural Networks (CNN); Image Processing; Optical Character Recognition (OCR); MNIST Dataset

Recently, there have been significant advances in deep neural networks (DNNs) and they have shown distinctive performance in speech recognition, natural language processing, and image recognition. In this paper, we explore DNNs to push the limit for branch prediction. We treat branch prediction as a classification problem and employ both deep convolutional neural networks (CNNs), ranging from LeNet to ResNet-50, and deep belief network (DBN) for branch prediction. We compare the effectiveness of DNNs with the state-of-the-art branch predictors, including the perceptron, our prior work, Multi-poTAGE+SC, and MTAGE+SC branch predictors. The last two are the most recent winners of championship branch prediction (CBP) contests. Several interesting observations emerged from our study. First, for branch prediction, the DNNs outperform the perceptron model as high as 60-80%. Second, we analyze the impact of the depth of CNNs (i.e., number of convolutional layers and pooling layers) on the misprediction rates. The results confirm that deeper CNN structures can lead to lower misprediction rates. Third, the DBN could outperform our prior work, but not outperform the state-of-the-art TAGE-like branch predictor; the ResNet-50 could not only outperform our prior work, but also the Multi-poTAGE+SC and MTAGE+SC.

ABSTRACTIn recent years, there has been a significant surge in the utilisation of deep learning and machine learning techniques for addressing complex and time‐intensive problems. The significance of employing deep learning becomes increasingly evident as the complexity of the problem increases. In the field of electromagnetics, the utilisation of deep learning techniques has exhibited exceptional efficacy across many applications, especially in wireless communications. In wireless communications, providing a structure that can simultaneously generate multiple beams and beamform them involves complexity and specific constraints. In this article, the time modulation technique is utilised to generate harmonics in the sidebands alongside the fundamental beam in various planar antenna arrays. By demonstrating the nonlinear shaping of the harmonic beams relative to each other, a novel approach is proposed that leverages deep learning techniques for the beamforming of both the fundamental beam and harmonic beams. In this regard, two models are proposed: a deep neural network (DNN) and a convolutional neural network (CNN). The input of CNN is comprised of two‐dimensional patterns of the main beam and harmonics. The input to DNN, on the other hand, includes useful details about the main beam and harmonics, such as their scanning angles, side lobe levels and directivities. The output of the models consists of the time modulation parameters of the array elements, including the pulse width and the pulse delay. The results demonstrate that DNN has achieved better accuracy and a shorter processing time in comprehending the relationship between the time modulation of array elements with different array dimensions and the radiation pattern of the fundamental beam and harmonic beams. Additionally, several samples are presented to evaluate the proposed model. The results demonstrate a high level of accuracy in fundamental beamforming, as well as in harmonic beamforming and beam steering.

In recent years, machine learning and Deep Learning have increased and gathered epic success in traditional application domains and new areas of Artificial Intelligence. The performance using Deep Learning has dominated experimental results compared to conventional machine learning algorithms. This paper presents an overview of the progress that has occurred in Deep Learning (DL) concerning some application domains like Autonomous Driving, Healthcare, Voice Recognition, Image Recognition, Advertising, Predicting Natural Calamities, National Stock Exchange and many more. Additionally, deeper insights into several Deep Learning techniques, their working principles, and experimental results are scrutinized. The survey covers Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL).

&lt;div class="section abstract"&gt;&lt;div class="htmlview paragraph"&gt;Predictive maintenance is crucial for Industry 4.0, and deep neural networks are a promising approach for predicting the capacity of electric batteries. However, few applications effectively utilize neural networks for this purpose with lithium-ion batteries. In this work, different deep learning models are developed, starting with simple neural networks, dense neural networks, convolutional networks, and recurrent networks. Using a public domain dataset, training, testing, and validation datasets were generated to predict battery capacity as a function of the number of cycles. Despite the limited number of samples in the dataset, deep learning techniques are employed to ensure robust prediction performance. The work presents the loss functions for each iteration of the algorithms and the average absolute error. The models made good generalizations over the test dataset within a short prediction time window. Finally, the work presents an average absolute error below 0.3, ensuring good convergence, avoiding data overfitting, and ensuring good generalization of the model to the test set. This work is a first estimate of the life cycle of electric vehicle batteries using small samples as inputs for estimating the remaining useful life of this electric vehicle component.&lt;/div&gt;&lt;/div&gt;

At present, deep learning is becoming one of the most popular approach to creation of the artificial intelligences systems such as speech recognition, natural language processing, computer vision and so on. The paper presents a historical overview of deep learning in neural networks. The model of the artificial neural network is described as well as the learning algorithms for neural networks including the error backpropagation algorithm, which is used to train deep neural networks. The development of neural networks architectures is presented including neocognitron, autoencoders, convolutional neural networks, restricted Boltzmann machine, deep belief networks, long short-term memory, gated recurrent neural networks, and residual networks. Training deep neural networks with many hidden layers is impeded by the vanishing gradient problem. The paper describes the approaches to solve this problem that provide the ability to train neural networks with more than hundred layers. An overview of popular deep learning libraries is presented. Nowadays, for computer vision tasks convolutional neural networks are utilized, while for sequence processing, including natural language processing, recurrent networks are preferred solution, primarily long short-term memory networks and gated recurrent neural networks.

Computer-aided diagnosis of ocular disorders like glaucoma can be effectively performed with retinal fundus images. Since the advent of machine learning and later deep learning techniques in medical image analysis, research focusing on automated and early detection of glaucoma has gained prominence. In this paper, we show how a deep Convolutional Neural Network model can be used in visualising the features related to Glaucoma that are consistent with (a subset of) those used by ophthalmologists for diagnosis. However, since the doctors do not necessarily depend entirely on these features but back them up with other important measurements such as visual field testing (HVF), any deep neural network such as CNN may not necessarily be expected to yield 100% accuracy. Further, other important parameters such as Intraocular Pressure (IOP), which may be built into a deep learning model, may not necessarily correlate well with the presence of Glaucoma. In this work, a deep CNN model has been trained and tested on a large number of high-quality fundus images, both normal and glaucomatous. We compare the results with transfer learning models such as VGG16, ResNet50, and MobileNetV2. On an average, we obtained an accuracy of 93.75% in identifying glaucoma, focusing only on the features of the fundus image.