Multivariate Time-Series Modeling for Forecasting Sintering Temperature in Rotary Kilns Using DCGNet

Rolling bearing is an essential part of various rotating machines, and its signal is the typical nonlinear signal. Traditional fault diagnosis usually relies on manual experience to extract the features of signals first. Deep convolutional neural networks (DCNN) can make fuller use of time series than traditional convolutional neural networks (CNN). Because of the low accuracy rate, fault diagnosis using gated recurrent unit (GRU) alone is not unsatisfactory. In order to improve the temporality of one-dimensional convolutional neural networks (1D-CNN) and enhance the accuracy of GRU, a novel fault diagnosis method called deep convolutional neural networks - gated recurrent unit (DCNN-GRU) is first put forward, which combines DCNN with GRU. The original signals without preprocessing are input into the DCNN, and the outputs of DCNN are input into the GRU consequently. Then the faults of rolling bearing can be diagnosed effectively. As the post-processing method, the t-distributed stochastic neighbor embedding (t-SNE) method is applied to visualize the fault diagnosis results. Six different network models, including the DCNN-GRU, are used to train the same fault dataset for comparison. The simulation results show that the proposed method can reach more than 99.9% accuracy stably for the given dataset, which can verify the feasibility and effectiveness of proposed method. And the DCNN-GRU can also be verified with good generalization ability using different dataset.

Messenger RNA (mRNA) has emerged as a critical global technology that requires global joint efforts from different entities to develop a COVID-19 vaccine. However, the chemical properties of RNA pose a challenge in utilizing mRNA as a vaccine candidate. For instance, the molecules are prone to degradation, which has a negative impact on the distribution of mRNA among patients. In addition, little is known of the degradation properties of individual RNA bases in a molecule. Therefore, this study aims to investigate whether a hybrid deep learning can predict RNA degradation from RNA sequences. Two deep hybrid neural network models were proposed, namely GCN_GRU and GCN_CNN. The first model is based on graph convolutional neural networks (GCNs) and gated recurrent unit (GRU). The second model is based on GCN and convolutional neural networks (CNNs). Both models were computed over the structural graph of the mRNA molecule. The experimental results showed that GCN_GRU hybrid model outperform GCN_CNN model by a large margin during the test time. Validation of proposed hybrid models is performed by well-known evaluation measures. Among different deep neural networks, GCN_GRU based model achieved best scores on both public and private MCRMSE test scores with 0.22614 and 0.34152, respectively. Finally, GCN_GRU pre-trained model has achieved the highest AuC score of 0.938. Such proven outperformance of GCNs indicates that modeling RNA molecules using graphs is critical in understanding molecule degradation mechanisms, which helps in minimizing the aforementioned issues. To show the importance of the proposed GCN_GRU hybrid model, in silico experiments has been contacted. The in-silico results showed that our model pays local attention when predicting a given position’s reactivity and exhibits interesting behavior on neighboring bases in the sequence.

In recent years, channel state information (CSI) in WiFi 802.11n has been increasingly used to collect data pertaining to human activity. Such raw data are then used to enhance human activity recognition. Activities such as lying down, falling, walking, running, sitting down, and standing up can now be detected with the use of information collected through CSI. Human activity recognition has a multitude of applications, such as home monitoring of patients. Four deep learning models are presented in this paper, namely: a convolution neural network (CNN) with a Gated Recurrent Unit (GRU); a CNN with a GRU and attention; a CNN with a GRU and a second CNN, and a CNN with Long Short-Term Memory (LSTM) and a second CNN. Those models were trained to perform Human Activity Recognition (HAR) using CSI amplitude data collected by a CSI tool. Experiments conducted to test the efficacy of these models showed superior results compared with other recent approaches. This enhanced performance of our models may be attributable the ability of our models to make full use of available data and to extract all data features, including high dimensionality and time sequence. The highest average recognition accuracy reached by the proposed models was achieved by the CNN-GRU, and the CNN-GRU with attention models, standing at 99.31% and 99.16%, respectively. In addition, the performance of the models was evaluated for unseen CSI data by training our models using a random split-of-dataset method (70% training and 30% testing). Our models achieved impressive results with accuracies reaching 100% for nearly all activities. For the lying down activity, accuracy obtained from the CNN-GRU model stood at 99.46%; slightly higher than the 99.05% achieved by the CNN-GRU with attention model. This confirmed the robustness of our models against environmental changes.

Accurate forecast of container throughput is an important basis for reasonably planning port construction, making port operation plan and adjusting port development direction. Aiming at the complex nonlinear characteristics of port container throughput, a hybrid model based on selective deep-ensemble for container throughput forecasting (HMSD) is presented in this paper. Firstly, this model decomposes the original container throughput time series into several intrinsic mode functions and a residual by empirical mode decomposition (EMD). Considering highly nonlinear characteristics of each intrinsic mode function, the proposed model constructs three deep neural networks, namely, long short term memory (LSTM), gated recurrent unit (GRU) and convolutional neural network (CNN), as base learners to predict intrinsic mode functions. Then, this model establishes selective deep-ensemble forecasting model by improved group method of data handling (GMDH) on intrinsic mode functions and obtains their ensemble forecasting results. Furthermore, this model uses an autoregressive integrated moving average model to predict the linear residual. Finally, the forecasting result of the total container throughput is obtained by integrating the forecasting results of all intrinsic mode functions and the residual. The empirical analysis of container throughput data of Xiamen port and Shanghai port in China shows that the HMSD model has better forecasting effect than other models.

One of the major problems, apparent in online social media, is the toxic online content. This has continued unabated, as people from diverse cultural backgrounds access the Internet, concealing their identity under the cloud of anonymity. Deep neural networks have been employed to detect hate speech from online content. This paper describes three different Deep Neural Network (DNN) Architectures for detection of hate words in Twitter - Gated Recurrent Unit (GRU), useful in capturing sequence orders, Convolution Neural Network (CNN), good for feature extraction, and Universal Language Model Fine-tuning (ULMFiT) model, which is based on transfer learning technique. ULMFiT model uses the DNN Architecture called Average-SGD Weight-Dropped Long Short Term Memory (AWD-LSTM). AWD –LSTM model was pre-trained using WikiText103 dataset. This method significantly outperformed the other Architectures.

Self-mixing interferometry (SMI) is superior to other laser interferometry methods due to its simplicity and compactness. However, SMI signals are often complex and difficult to process due to interference in the form of variations in the effective reflectivity of the target, noisy signals, complex signal shapes, and other dependencies. Deep neural networks have been a very popular area of research in computer artificial intelligence in recent years, allowing more implicit features to be uncovered than traditional shallow machine learning. It has been shown that convolutional and back-propagation neural networks can be used for SMI signal processing. There are also studies that have used machine learning genetic algorithms for absolute distance measurement. Based on the above, this study used convolutional neural networks to form a deep neural network for absolute distance measurement based on SMI technology. We first trained the deep convolutional neural network at different feedback strengths. The results of the Convolutional Neural Network (CNN) model showed a coefficient of determination of 0.9987. which is consistent with the required model. The trained network was then used to estimate absolute distances with and without the addition of noise. The comparison proves that the proposed method is noise-proof and has high adaptability for measurements under different conditions.

Due to the continuous growth and advancement of technology, an enormous volume of text data is generated daily across various sources including social media platforms, websites, search engines, healthcare records, and news articles. Extracting meaningful patterns from text data, such as viewpoints, related theories, journal distribution, facts, and the development of online news text, is a challenging task due to the varying lengths of the texts. One issue arises from the length of the text data itself, and another challenge lies in extracting valuable features, especially in news articles. In the deep learning models, the convolutional neural networks (CNNs) are capable of capturing local features in text data, but unable to capture the structural information or semantic relationships between words. Consequently, a sole CNN network often yields poor performance in text classification tasks, whereas the Gated Recurrent Unit (GRU) is adept at effectively extracting semantic information and understanding the global structural relationships present in textual data. This paper presents a solution to the problem by introducing a new text classification that integrates the strengths of CNN and GRU. The proposed hybrid models incorporate word vectorization and word dispersion in parallel. Initially, the model trains word vectors using the Word2vec model and then leverages the GRU model to capture semantic information from text sentences. Subsequently, the CNN method is employed to capture crucial semantic features, leading to classification using the SoftMax layer. Experimental findings demonstrated that the proposed hybrid GRU_CNN model outperformed and achieved accuracy 97.73% as compared to individual CNN, LSTM, and GRU models in terms of classification effectiveness and accuracy.

Prediction Of Raw Material Batches For The Production Of Clinker By Means Of Artificial Neural Networks -Analysis Of Behaviour

Investigation of crowdshipping delivery trip production with real-world data

Text classification is a task to assign text documents according to its content to one or more classes automatically. Recently character-level models using deep neural networks have been developed to do classification text. Moreover, in some cases, character-level models have outperformed word-level models and traditional models, especially on user-generated dataset. The topologies that have been used for the character-level models are convolutional neural networks (CNN) and bidirectional recurrent neural networks (Bi-RNN), with its variants; long short-term memory (LSTM) and gated recurrent units (GRU). In this paper, CNN, Bi-RNN, and the combination of both are tested with character-level features and word-level features for text classification on English and Indonesian social media datasets. On small size datasets, word-level model outperformed character-level models. However, on dataset with millions of data, character-level model outperformed word-level model. Further analysis on the evaluation of word-level and character-level models is also discussed in this paper.

The detection of poultry egg microcracks based on electrical characteristic models is a new and effective method. However, due to the disorder, mutation, nonlinear, time discontinuity, and other factors of the current data, detection algorithms such as support-vector machines (SVM) and random forest (RF) under traditional statistical characteristics cannot identify subtle defects. The detection system voltage is set to 1500 V in the existing method, and higher voltages may cause damage to the hatched eggs; therefore, how to reduce the voltage is also a focus of research. In this paper, to address the problem of the low signal-to-noise ratio of microcracks in current signals, a wavelet scattering transform capable of extracting translation-invariant and small deformation-stable features is proposed to extract multi-scale high-frequency feature vectors. In view of the time series and low feature scale of current signals, various convolutional networks, such as a one-dimensional convolutional neural network (1DCNN), long short-term memory (LSTM), bi-directional long short-term memory (Bi-LSTM), and gated recurrent unit (GRU) are adopted. The detection algorithm of the wavelet scattering convolutional network is implemented for electrical sensing signals. The experimental results show that compared with previous works, the accuracy, precision, recall, F1-score, and Matthews correlation coefficient of the proposed wavelet scattering convolutional network on microcrack datasets smaller than 3 μm at a voltage of 1000 V are 99.4393%, 99.2523%, 99.6226%, 99.4357%, and 98.8819%, respectively, with an average increase of 2.0561%. In addition, the promotability and validity of the proposed detection algorithm were verified on a class-imbalanced dataset and a duck egg dataset. Based on the good results of the above experiments, further experiments were conducted with different voltages. The new feature extraction and detection method reduces the sensing voltage from 1500 V to 500 V, which allows for achieving higher detection accuracy with a lower signal-to-noise ratio, significantly reducing the risk of high voltage damage to hatching eggs and meeting the requirements for crack detection.

In recent years Neural Networks and especially Deep Neural Networks (DNN) have seen a rise in popularity. DNNs have increased the overall performance of algorithms in applications such as image recognition and classification, 2D and 3D pose detection, natural language processing, and time series forecasting. Especially in image classification and recognition, so-called Convolutional Neural Network (CNN) have gained high interest as they can reach a high accuracy, which makes them a viable solution in cancer detection or autonomous driving. As CNNs are widely used in image tasks, different visualizations techniques have bee developed to show how their internals are working. Apart from image tasks, CNNs are applicable to other problems, e.g., time series classification, time series forecasting, or natural language processing. CNNs in those contexts behave similarly, allowing for the same visualization techniques to make them more interpretable. In this chapter, we adapt image visualization algorithms to time series problems, allowing us to build granular, intuitively interpretable feature hierarchies to make a time series forecast as understandable as an image recognition task. We do so by using our previous work on power time series forecasting using CNN Auto Encoders (AEs) and applying typical CNN visualization techniques to it. Thus, we guid computer scientists to provide better interpretable figures for a time series forecasting task to application domain experts.KeywordsVisualizationConvolutional neural networksTime series data

The main neural network architectures applied for product demand forecasting are examined. Particular attention is given to Recurrent Neural Networks (RNN), which demonstrate high efficiency when working with time series and detecting dependencies within data. A detailed analysis is provided for the Long Short-Term Memory (LSTM) architecture, an advanced variant of RNN that effectively addresses the vanishing gradient problem, a common issue in traditional RNN. Due to its capability to retain information over extended periods, LSTM is ideal for tasks requiring the analysis of long-term dependencies, such as seasonal fluctuations in product demand. The application of Convolutional Neural Networks (CNN) is analyzed, which show strong performance when processing structured data, such as images or matrices. With their ability to effectively detect spatial dependencies, CNN can be useful for demand forecasting when complex relationships between various data features need to be considered, particularly for demand evaluations dependent on multiple factors. The architectures Feedforward Neural Networks (FNN), Gated Recurrent Units (GRU), Attention-based models (ABM) and Autoencoders (AE) are considered, which can also be applied to solve demand forecasting problems. A comparative analysis of the architectures is made in terms of forecast accuracy and ability to work with time series. By evaluating the training speed, data processing time, and reliability of these architectures in terms of the stability of results and forecast reliability, it has been determined that LSTM architectures are the most effective for product demand forecasting. These networks provide high accuracy and reliability of results and perform well with time series, which is crucial for forecasting demand under dynamic conditions.

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially deep autoencoders, have proven promising both for crossmodal translation and for early fusion via multimodal embedding. A particular architecture, bidirectional symmetrical deep neural networks, have been proven to yield improved multimodal embeddings over classical autoencoders, while also being able to perform crossmodal translation. In this work, we focus firstly at evaluating good single-modal continuous representations both for textual and for visual information. Word2Vec and paragraph vectors are evaluated for representing collections of words, such as parts of automatic transcripts and multiple visual concepts, while different deep convolutional neural networks are evaluated for directly embedding visual information, avoiding the creation of visual concepts. Secondly, we evaluate methods for multimodal fusion and crossmodal translation, with different single-modal pairs, in the task of video hyperlinking. Bidirectional (symmetrical) deep neural networks were shown to successfully tackle downsides of multimodal autoencoders and yield a superior multimodal representation. In this work, we extensively tests them in different settings, with different single-modal representations, within the context of video-hyperlinking. Our novel bidirectional symmetrical deep neural networks are compared to classical autoencoders and are shown to yield significantly improved multimodal embeddings that significantly (alpha=0.0001) outperform multimodal embeddings obtained by deep autoencoders with an absolute improvement in precision at 10 of 14.1% when embedding visual concepts and automatic transcripts and an absolute improvement of 4.3% when embedding automatic transcripts with features obtained with very deep convolutional neural networks, yielding 80% of precision at 10.

Deep Convolutional Neural Networks for Large-scale Speech Tasks