A Comparison of Quantized Convolutional and LSTM Recurrent Neural Network Models Using MNIST

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

Deep learning models for short-term travel speed prediction on urban expressways, such as the convolutional neural network (CNN), still present several limitations in multiscale spatiotemporal feature extraction. Hence, in this paper, three hybrid CNN models are proposed to improve the basic CNN model with regard to three target aspects for short-term (i.e., 5 min) travel speed prediction on urban expressways. More specifically, long short-term memory (LSTM), AutoEncoder (AE), and Inception module are incorporated into the basic CNN model to capture multiscale spatiotemporal features of travel speed data effectively and improve the accuracy and robustness of the basic CNN model. Based on loop detector data collected on the Yan’an expressway in Shanghai, the proposed hybrid CNN models are trained and tuned. To validate the improvements on the target aspects, a comprehensive comparison is conducted using a classical statistical model (i.e., autoregressive integrated moving average), a typical shallow neural network model (i.e., artificial neural network), and two basic deep learning models (i.e., recurrent neural network and CNN). Results show that the prediction accuracies of all the proposed hybrid CNN models exceed 96% and the mean absolute errors are less than 2.5 km/h, which are superior to other models. In terms of target improving aspects, two new metrics were introduced, and the proposed models, especially the AE–CNN model, showed better robustness under various input data structures and traffic states. The LSTM–CNN model outperformed the other models in learning time-series features, and the Inception–CNN model is superior in reproducing the dynamics of traffic congestion patterns on urban expressways.

ABSTRACTIn this paper, the convolutional recurrent neural network (ConvLSTM) architecture is proposed to predict individual stock prices. The characteristics of stock data are automatically extracted through convolutional neural network (CNN). Afterward, the extracted features are inputted into a long short-term memory (LSTM) model with memory characteristics for prediction. Since CNN has been a representation learning model, it is quite appropriate for automatic feature extraction. Besides, the LSTM architecture of the recurrent neural network could effectively surmount the issues of gradient disappearance and expansion of the time series data. Ten stock historical data had been collected in the experimental data set. Furthermore, many commonly used technical indicators are calculated in advance for the expansion of the dimension of the training samples. The experimental results obtain a 3.449 RMSE (root-mean-square error) in average.

The presented paper proposes a new speech command recognition model for novel engineering applications with limited resources. We built the proposed model with the help of a Convolutional Recurrent Neural Network (CRNN). The use of CRNN instead of Convolutional Neural Network (CNN) helps us to reduce the model parameters and memory requirement as per resource constraints. Furthermore, we insert transmute and curtailment layer between the layers of CRNN. By doing this we further reduce model parameters and float number of operations to half of the CRNN requirement. The proposed model is tested on Google’s speech command dataset. The obtained result shows that the proposed CRNN model requires 1/3 parameters as compared to the CNN model. The number of parameters of the CRNN model is further reduced by 45% and the float numbers of operations between 2% to 12 % in different recognition tasks. The recognition accuracy of the proposed model is 96% on Google’s speech command dataset, and on laboratory recording, its recognition accuracy is 89%.

People counting and pedestrian flow statistics are challenging tasks because of the perspective distortions, appearance changes and occlusion. In this paper, we address the two tasks: people counting in images of highly dense crowds and pedestrian flow statistics in a place over a period of time. Our first contribution is to propose a new convolution neural network (CNN) model which is composed of a deep and shallow fully convolution network to fulfill the task of people counting. We extract different layer features from the deep fully convolution network and the last layer features from the shallow fully convolution network, and concatenate them together. After that we add two deconvolution layers to make the output image have the same resolution with the input image. Our second contribution is to combine pedestrian flow statistics task with people counting task. According to the density maps that CNN model generates, we can calculate the number of people crossing a place based on the recurrent neural network (RNN). Besides, we also have collected two datasets and labelled them. Extensive experiments have been implemented, our people counting method outperforms other existing methods, and our pedestrian flow statistics method combined with CNN model also outperforms the model which only uses long-short term memory (LSTM).

Text classification is a classic task in the field of natural language processing. however, the existing methods of text classification tasks still need to be improved because of the complex abstraction of text semantic information and the strong relecvance of context. In this paper, we combine the advantages of two traditional neural network model, Long Short-Term Memory(LSTM) and Convolutional Neural Network(CNN). LSTM can effectively preserve the characteristics of historical information in long text sequences, and extract local features of text by using the structure of CNN. We proposes a hybrid model of LSTM and CNN, construct CNN model on the top of LSTM, the text feature vector output from LSTM is further extracted by CNN structure. The performance of the hybrid model is compared with that of other models in the experiment. The experimental results show that the hybrid model can effectively improve the accuracy of text classification.

Online leakage current classification using convolutional neural network long short-term memory for high voltage insulators on web-based service

In the realm of water resource management, meticulous monitoring and control methodologies are quintessential to the refinement of wastewater treatment processes. This research elucidates an avant-garde methodology for forecasting the Chemical Oxygen Demand (COD), an instrumental indicator of water quality, by harnessing the capabilities of long short-term memory (LSTM) neural networks in conjunction with Internet of Things (IoT) paradigms. The efficacy of the LSTM model is juxtaposed with that of an advanced Deep Belief Network (DBN), as well as contemporary models like a Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) hybrid model and a Transformer-based model, employing data sourced from a wastewater treatment facility located in Changsha. The empirical findings show that notwithstanding the comparable training durations used, the LSTM model exhibits a preeminent error rate of merely 7%, thus surpassing the DBN model (which has an error rate of 35%), the CNN-LSTM model (registering a 22% error rate), and the Transformer-based model (with a 17% error rate) in its predictive precision. This research underscores the potential of integrating an astute wastewater control system with IoT and LSTM models, thereby hinting at prospective enhancements in the sustainability and operational efficacy of wastewater treatment installations.

An ensemble deep learning model for the detection of Oral Squamous Cell Carcinoma (OSCC) using histopathological images. To improve the early detection and diagnosis of oral cancer through advanced image analysis techniques. The basic methods of the convolutional neural network (CNN) and long short-term memory (LSTM) neural network are combine the convolutional neural network (CNN) and long short-term memory (LSTM) neural network to design an open oral scoring model based on CNN + LSTM. An experimental environment is then built to preprocess the data, and finally the model built in this study is trained and simulated. Evaluation metrics demonstrate the effectiveness of the model in diagnosing OSCC with high precision, preprocess the data, offering a potential tool for clinical decision support. Final outcome an oral cancer data analysis based cancer detection for cancer or non-cancer classification.

This paper proposes a multi-layered anomaly detection scheme to train feature extraction and to test anomaly prediction by using Convolutional Neural Networks (CNNs) layer, Bidirectional and Unidirectional Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNNs), which is one of a novel deep architecture named stacked convolutional bidirectional LSTM network (SCB-LSTM). In the proposed model, the stacked CNNs perform feature extraction of vibration sensor signal patterns, and the result is used to feature learning with the stacked bidirectional LSTMs (SB-LSTMs). After this procedure, the stacked unidirectional LSTMs (SU-LSTMs) enhance the feature learning, and a regression layer finally predicts anomaly detections. The experimental results of bearing data not only show the accuracy of the proposed model in anomaly detection for rotating machinery diagnostics, but also suggest the better performance than other state-of-the-art algorithms such as a plain uni-LSTM or Bi-LSTM.

Forecasting stock prices has always been a complex and challenging task due to its high volatility, noise in data, and abrupt price fluctuations caused by different factors. Traditional approaches, such as quantitative, fundamental, and technical analysis, often struggle to accurately predict stock price movements due to their difficulty in processing non-linear data. The evolution of machine learning, particularly neural network models, opened new opportunities for improving prediction accuracy by handling non-linear and multidimensional data with high effectiveness. This study focuses on evaluating and comparing three neural network architectures, in which the last is a combination of the first two - Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and a hybrid CNN-LSTM model, for forecasting stock prices. Using ten years of historical stock data, taken for Apple Inc. (AAPL) stocks as an example, ranging from 2015 to 2025, the models were trained and tested on key performance metrics such as Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and R². Results show that while both CNN and LSTM models show good performance individually, the hybrid CNN-LSTM model can significantly outperform them, achieving the highest accuracy with the lowest error rates. By combining the strengths of CNN for feature extraction and LSTM for temporal sequence processing, the hybrid model effectively captures complex patterns in stock price movements. By iterating over the types of layers to use, we came to the conclusion, that by combining both models into one hybrid model, some layers, like one CNN layer, and one LSTM layer should be removed. This research demonstrates the potential of hybrid neural network models in financial time-series forecasting and comprehensively analyzes their advantages over non-hybrid models.

Handwritten digit recognition, a task in computer vision, is critical for applications such as postal automation, banking, and digitization of forms. Traditional approaches have leveraged statistical models, but the rise of deep learning, particularly Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM), has revolutionized the field. However, a comprehensive performance comparison of CNN and LSTM architectures in the context of handwritten digit classification remains underexplored. This study aimed to address this gap by evaluating and comparing CNN and LSTM models for the classification of handwritten digits. Two machine learning models – Convolutional Neural Network (CNN) and Long-Short Term Memory (LSTM) were trained with the preprocessed handwritten digit dataset. The CNN model was designed with multiple convolutional and pooling layers, along with dropout for regularization. The LSTM model was designed with LSTM layers to capture sequential patterns in the data, followed by a dense layer for classification. The models were implemented in python, evaluated and compared based on accuracy, precision, recall and F1-score. The evaluation and comparison results indicate that CNN achieved 99.31% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score, while LSTM achieved 98.90% accuracy, 99.0% precision, 99.0% recall, and a 99.0% F1-score. The results demonstrated that CNN outperformed LSTM in terms of accuracy and misclassification errors, making it the optimal choice for image-based handwritten digit recognition. This finding underscores the efficiency of CNN in addressing challenges related to digit recognition, contributing to the advancement of automated digit classification systems and improving the accuracy of image-based classification tasks.

This research aims to implement deep learning techniques to determine fact and fake news in Indonesian language. The methods used are Convolutional Neural Network (CNN) and Long Short Term Memory (LSTM). The stages of the research consisted of collecting data, labeling data, preprocessing data, word embedding, splitting data, forming CNN and LSTM models, evaluating, testing new input data and comparing evaluations of the established CNN and LSTM models. The Data are collected from a fact and fake news provider site that is valid, namely TurnbackHoax.id. There are 1786 news used in this study, with 802 fact and 984 fake news. The results indicate that the CNN and LSTM methods were successfully applied to determine fact and fake news in Indonesian language properly. CNN has an accuracy test, precision and recall value of 0.88, while the LSTM model has an accuracy test and precision value of 0.84 and a recall of 0.83. In testing the new data input, all of the predictions obtained by CNN are correct, while the prediction results obtained by LSTM have 1 wrong prediction. Based on the evaluation results and the results of testing the new data input, the model produced by the CNN method is better than the model produced by the LSTM method.

Joint estimation of State of Charge (SOC) and State of Health (SOH) for lithium ion batteries using Support Vector Machine (SVM), Convolutional Neural Network (CNN) and Long Sort Term Memory Network (LSTM) models