Hybrid Deep Neural Network based on SDAE and GRUNN

Owing to high transmission ratio and compact structure, planetary gearboxes are widely used in industrial applications. However, due to running under noisy environment and time-varying rotational speed conditions, the acquired vibration signals are nonstationary, which seriously degrade the performance of fault diagnosis methods of planetary gears. To address this problem, a fault diagnosis method of planetary gears using a stacked denoising autoencoder (SDAE) and a gated recurrent unit neural network (GRUNN) is proposed in this paper. First, a hybrid model based on SDAE and GRUNN is developed to remove noise components from input data and process pre- and post-correlation time-series data. The training samples for planetary gear fault diagnosis are regarded as the input data of the developed hybrid model to automatically extract robust fault features. Then, the training process of the developed hybrid model is presented. The Adam optimization algorithm is utilized to optimize the parameters, and the dropout technique is employed to prevent from overfitting. Finally, a softmax classifier is employed to identify planetary gear states in test samples. The effectiveness of the proposed method is validated through a fault diagnosis experiment of planetary gears. The experimental results show that the proposed method possesses strong anti-noise ability and adaptability to time-varying rotational speed.

The constitutive modelling of granular soils has been a long-standing research subject in geotechnical engineering, and machine learning (ML) has recently emerged as a promising tool for achieving this goal. This paper proposes two recurrent neural networks, namely, the Gated Recurrent Unit Neural Network (GRU-NN) and the Long Short-Term Memory Neural Network (LSTM-NN), which utilize input parameters such as the initial void ratio, initial fabric anisotropy, uniformity coefficient, mean particle size, and confining pressure to establish the high-dimensional relationships of granular soils from micro to macro levels subjected to triaxial shearing. The research methodology consists of several steps. Firstly, 200 numerical triaxial tests on idealized granular soils comprising polydisperse spherical particles are performed using the discrete element method (DEM) simulation to generate datasets and to train and test the proposed neural networks. Secondly, LSTM-NN and GRU-NN are constructed and trained, and their prediction performance is evaluated by the mean absolute percentage error (MAPE) and R-square against the DEM-based datasets. The extremely low error values obtained by both LSTM-NN and GRU-NN indicate their outstanding capability in predicting the constitutive behaviour of idealized granular soils. Finally, the trained ML-based models are applied to predict the constitutive behaviour of a miniature glass bead sample subjected to triaxial shearing with in situ micro-CT, as well as to two extrapolated test sets with different initial parameters. The results show that both methods perform well in capturing the mechanical responses of the idealized granular soils.

Heart disease is a prevalent and life-threatening condition that affects a significant portion of the global population. Preventing negative consequences and better health outcomes may be greatly helped by early identification and precise prognosis of cardiac disease. Electrocardiogram (ECG) is the important apparatus which is used for diagnosticians of assessment of cardiogram in the clinical. This will Owing to its ability to learn complicated patterns from massive volumes of data, deep neural networks (DNNs) have recently emerged as strong tools for medical diagnostic and prediction tasks. In this part of research, people investigate the use of deep neural networks for cardiac illness forecasting which is easy way for analysis of cardio problem. We analyse various studies and approaches that have utilized Deep Neural Networks for heart disease prediction. This study gives us whole sheds light on the evolution of methods that use in deep neural networks to forecast cardiac issues. The findings from this survey can guide researchers and practitioners in designing and implementing effective DNN models for heart disease prediction, ultimately contributing to improved clinical decision-making and patient care.In this study, we will examine the various DNN techniques, including CNNs (Convolutional Neural Networks) and Stack Denoising Auto encoder, Deep Belief Network, Recurrent Neural Network (RNN), Long Short term Memory (LSTM) and Gated Recurrent Unit (GRU).

Prediction concept needs human-like thinking and ideas. ANN helps make a program for describing the solutions to the prediction problems. To gain accurate prediction, there is a need for deep thinking in the knowledge system discovered. Deep learning helps form such next-level type of ANN thinking. Deep learning models do not require humans for programming the problems. It can determine the prediction by itself by learning the dataset. This paper describes the prediction by considering sequential model for continuous accessing of data with the help of multi-layers. Deep learning is very helpful for predicting larger dataset. This paper proposes an algorithmic model known as a Comprehensive Deep Recurrent Artificial Neural Networks (CDRANN) by hybridizing LSTM (long short-term memory) with RNN (recurrent neural network) backtrack solver for better future stock prediction. This paper also explains the need for hybridizing evolutionary model CDRANN to create a novel change in future prediction.KeywordsCDRANN (Comprehensive Deep Recurrent Artificial Neural Networks)LSTM (long short-term memory)RNN (recurrent neural network)Stock marketPrediction

In recent years, growing interest in deep learning neural networks has raised a question on how they can be used for effective processing of high-dimensional datasets produced by hyperspectral imaging (HSI). HSI, traditionally viewed as being within the scope of remote sensing, is used in non-invasive substance classification. One of the areas of potential application is forensic science, where substance classification on the scenes is important. An example problem from that area—blood stain classification—is a case study for the evaluation of methods that process hyperspectral data. To investigate the deep learning classification performance for this problem we have performed experiments on a dataset which has not been previously tested using this kind of model. This dataset consists of several images with blood and blood-like substances like ketchup, tomato concentrate, artificial blood, etc. To test both the classic approach to hyperspectral classification and a more realistic application-oriented scenario, we have prepared two different sets of experiments. In the first one, Hyperspectral Transductive Classification (HTC), both a training and a test set come from the same image. In the second one, Hyperspectral Inductive Classification (HIC), a test set is derived from a different image, which is more challenging for classifiers but more useful from the point of view of forensic investigators. We conducted the study using several architectures like 1D, 2D and 3D convolutional neural networks (CNN), a recurrent neural network (RNN) and a multilayer perceptron (MLP). The performance of the models was compared with baseline results of Support Vector Machine (SVM). We have also presented a model evaluation method based on t-SNE and confusion matrix analysis that allows us to detect and eliminate some cases of model undertraining. Our results show that in the transductive case, all models, including the MLP and the SVM, have comparative performance, with no clear advantage of deep learning models. The Overall Accuracy range across all models is 98–100% for the easier image set, and 74–94% for the more difficult one. However, in a more challenging inductive case, selected deep learning architectures offer a significant advantage; their best Overall Accuracy is in the range of 57–71%, improving the baseline set by the non-deep models by up to 9 percentage points. We have presented a detailed analysis of results and a discussion, including a summary of conclusions for each tested architecture. An analysis of per-class errors shows that the score for each class is highly model-dependent. Considering this and the fact that the best performing models come from two different architecture families (3D CNN and RNN), our results suggest that tailoring the deep neural network architecture to hyperspectral data is still an open problem.

Due to the major operating restrictions, ensuring security is the fundamental problem of Wireless Sensor Networks (WSNs). Because of their inadequate security mechanisms, WSNs are indeed a simple point for malware (worms, viruses, malicious code, etc.). According to the epidemic nature of worm propagation, it is critical to develop a worm defense mechanism in the network. This concept aims to establish novel malware detection in WSN that consists of several phases: “(i) Preprocessing, (ii) feature extraction, as well as (iii) detection”. At first, the input data is subjected for preprocessing phase. Then, the feature extraction takes place, in which principal component analysis (PCA), improved linear discriminant analysis (LDA), and autoencoder-based characteristics are retrieved. Moreover, the retrieved characteristics are subjected to the detection phase. The detection is performed employing combined shallow learning and DL. Further, the shallow learning includes decision tree (DT), logistic regression (LR), and Naive Bayes (NB); the deep learning (DL) includes deep neural network (DNN), convolutional neural network (CNN), and recurrent neural network (RNN). Here, the DT output is given to the DNN, LR output is subjected to CNN, and the NB output is given to the RNN, respectively. Eventually, the DNN, CNN, and RNN outputs are averaged to generate a successful outcome. The combination can be thought of as an Ensemble classifier. The weight of the RNN is optimally tuned through the Self Improved Shark Smell Optimization with Opposition Learning (SISSOOL) model to improve detection precision and accuracy. Lastly, the outcomes of the suggested approach are computed in terms of different measures.

MOOCs (Massive Open Online Courses) are definitely one of the best approach to support the international agenda about inclusive and equitable education and lifelong learning opportunities for all (SDG4) [1]. A great deal universities and institutions offer valuable free courses to their numerous students and to people around the word through MOOC platforms. However, because of huge number of learners and data generated, learner’s behaviour in those platforms remain a kind of black box for learners themselves and for courses instructors who are supposed to tutor or monitor learners in the learning process. Therefore, learner do not receive sufficient support from instructors and from their peers, during the learning process [2]. This is one the main reasons that lead to high dropout, low completion and success rate observed in the MOOCs. Many research work have suggested descriptive, predictive and prescriptive models to address this issue, but most of these models focus on predicting dropout, completion and/or success, and do not generally pay enough attention to one of the key step (learner behaviour), that comes before, and can explain dropping out and failure. Our research aims to develop a deep learning model to predict learner behaviour (learner interactions) in the learning process, in order to equip learners and course instructors with insight understanding of the learner behaviour in the learning process. This specific paper will focus on analysing relevant algorithms to develop such model. For this analysis, we used data from UNESCO-IICBA (International Institute for Capacity Building in Africa) MOOC platform, designed for teacher training in Africa, and then we examine many types of features: geographical, social behavioural and learning behavioural features. Learner’s behaviour being a time series Big data, we built the predictive model using Deep Learning algorithms for better performance and accuracy (Thanks to the power of deep learning) compared to baseline Machine learning algorithms. Time series data is best handled by recurrent neural networks (RNN) [3], so, we choose RNN and implemented/tested the three main architectures of RNN: Simple RNNs, GRU (Gated Recurrent Unit) RNNs and LSTM (Long short-term memory) RNNs. The models were trained using L2 Regularization, based on the predictions results, we unexpectedly found model with simple RNNs produced the best performance and accuracy on the dataset used than the other RNN architectures. We had couple of observations, example: we saw a correlation between video viewing and quiz behaviour and the participation of the learner to the learning process. This observation could allow teachers to provide meaningful support and guidance to at risk or less active students. We also observed that, the shorter the video or the quiz, the more the viewer. We conclude that we could use learner video or quiz viewing behaviour to predict his behaviour concerning other MOOC contents. These results suggest the need of deeper research on educational video and educational quiz designing for MOOCs.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Determining the structure of a protein given its sequence is a challenging problem. Deep learning is a rapidly evolving field which excels at problems where there are complex relationships between input features and desired outputs. Deep Neural Networks have become popular for solving problems in protein science. Various deep neural network architectures have been proposed including deep feed-forward neural networks, recurrent neural networks and more recently neural Turing machines and memory networks. This article provides a short review of deep learning applied to protein prediction problems.

Sensitive information such as credit card information, username, password and social security number etc, can be stolen using a fake page that imitates trusted website is called phishing. The attacker designs a similar webpage either by copying or making small manipulation to the legitimate page so that the online user cannot distinguish the legitimate and fake websites. A Deep Neural Network (DNN) was introduced to detect the phishing Uniform Resource Locator (URL). Initially, a 30-dimension feature vector was constructed based on URL-based features, Hypertext Markup Language (HTML)-based features and domain-based features. These features were processed in DNN to detect the phishing URL. However, the irrelevant, redundant and noisy features in the dataset increase the complexity of DNN classifier. So the feature selection is required for efficient phishing attack detection. But feature selection is a time-consuming process since it is an independent process. So in this paper, a feature vector is generated by DNN itself using Stacked Denoise Auto Encoder (SDAE). Moreover, the noisy data such as missing features affect the efficiency of phishing detection so the SDAE is trained to reconstruct a clean input feature vector. The initial input feature vector is corrupted by setting some feature vectors as zero. Then the corrupted feature vector is then mapped with basic auto encoder, to a hidden representation from which the input feature vector is reconstructed. The reconstructed features are given as input to DNN which selects the most relevant features and predicts the phishing URL. Hence the sparse feature representation of SDAE increases the classification accuracy of DNN. The experiments are conducted in Ham, Phishing Corpus and Phishload datasets in terms of accuracy, precision, recall and F-measure to prove the effectiveness of DNN-SDAE.

This paper proposes an hybrid deep neural network (DNN) for speech recognition and an energy-efficient reconfigurable architecture with approximate computing for accelerating the DNN. The hybrid DNN consists of two network models: a binary weight network (BWN) for twenty key words recognition; a recurrent neural network (RNN) for processing acoustic model of high precision common words recognition. To accelerate the hybrid DNN and reduce the energy cost, we propose a digital-analog mixed reconfigurable architecture with approximate computing units, including: a BWN accelerator with analog multi-chain delay-addition units for bit-wise approximate computing, and a RNN accelerator with approximate multiplication units for different calculation accuracy requirements. Implementation and simulation with TSMC 28nm HPC+ process technology, the energy efficiency of proposed architecture can achieves 163.8TOPS/W for twenty key words recognition and 3.3TOPS/W for common words recognition. Comparing with State-of-the-Art architectures, this work achieves over 1.7X better in energy efficiency with approximate computing.

Feature data processing: Making medical data fit deep neural networks

Internet of Things (IoT) technology has been incorporated into the majority of people’s everyday lives and places of employment due to the quick development in information technology. Modern agricultural techniques increasingly use the well-known and superior approach of managing a farm known as “smart farming”. Utilizing a variety of information and agricultural technologies, crops are observed for their general health and productivity. This requires monitoring the condition of field crops and looking at many other indicators. The goal of smart agriculture is to reduce the amount of money spent on agricultural inputs while keeping the quality of the final product constant. The Internet of Things (IoT) has made smart agriculture possible through data collection and storage techniques. For example, modern irrigation systems use effective sensor networks to collect field data for the best plant irrigation. Smart agriculture will become more susceptible to cyber-attacks as its reliance on the IoT ecosystem grows, because IoT networks have a large number of nodes but limited resources, which makes security a difficult issue. Hence, it is crucial to have an intrusion detection system (IDS) that can address such challenges. In this manuscript, an IoT-based privacy-preserving anomaly detection model for smart agriculture has been proposed. The motivation behind this work is twofold. Firstly, ensuring data privacy in IoT-based agriculture is of the utmost importance due to the large volumes of sensitive information collected by IoT devices, including on environmental conditions, crop health, and resource utilization data. Secondly, the timely detection of anomalies in smart agriculture systems is critical to enable proactive interventions, such as preventing crop damage, optimizing resource allocation, and ensuring sustainable farming practices. In this paper, we propose a privacy-encoding-based enhanced deep learning framework for the difficulty of data encryption and intrusion detection. In terms of data encoding, a novel method of a sparse capsule-auto encoder (SCAE) is proposed along with feature selection, feature mapping, and feature normalization. An SCAE is used to convert information into a new encrypted format in order to prevent deduction attacks. An attention-based gated recurrent unit neural network model is proposed to detect the intrusion. An AGRU is an advanced version of a GRU which is enhanced by an attention mechanism. In the results section, the proposed model is compared with existing deep learning models using two public datasets. Parameters such as recall, precision, accuracy, and F1-score are considered. The proposed model has accuracy, recall, precision, and F1-score of 99.9%, 99.7%, 99.9%, and 99.8%, respectively. The proposed method is compared using a variety of machine learning techniques such as the deep neural network (DNN), convolutional neural network (CNN), recurrent neural network (RNN), and long short-term memory (LSTM).

In this study, we investigate the effectiveness of various deep learning algorithms for customer segmentation in the banking sector, aiming to enhance targeted service delivery and customer experience. We employ a comprehensive pipeline encompassing data collection, preprocessing, feature selection, feature extraction, model development, and rigorous evaluation. Our dataset, derived from real-world banking customer profiles, was processed using normalization, encoding, and dimensionality reduction techniques. We implemented and compared eight models: Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), CNN-LSTM Hybrid, Autoencoder-Based Network, and Deep Neural Network (DNN). Among them, the Autoencoder-Based model achieved the highest accuracy of 91.56%, outperforming others in terms of segmentation clarity and computational efficiency. These findings suggest that deep learning methods, particularly Autoencoder-Based architectures, offer robust solutions for real-time banking customer segmentation, enabling institutions to tailor products and services more effectively.

Trajectory-level fog detection based on in-vehicle video camera with TensorFlow deep learning utilizing SHRP2 naturalistic driving data