Developments in Capsule Network Architecture: A Review

The presented paper proposes a new speech command recognition model for novel engineering applications with limited resources. We built the proposed model with the help of a Convolutional Recurrent Neural Network (CRNN). The use of CRNN instead of Convolutional Neural Network (CNN) helps us to reduce the model parameters and memory requirement as per resource constraints. Furthermore, we insert transmute and curtailment layer between the layers of CRNN. By doing this we further reduce model parameters and float number of operations to half of the CRNN requirement. The proposed model is tested on Google’s speech command dataset. The obtained result shows that the proposed CRNN model requires 1/3 parameters as compared to the CNN model. The number of parameters of the CRNN model is further reduced by 45% and the float numbers of operations between 2% to 12 % in different recognition tasks. The recognition accuracy of the proposed model is 96% on Google’s speech command dataset, and on laboratory recording, its recognition accuracy is 89%.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

Social media and health-related forums, including the expression of customer reviews, have recently provided data sources for adverse drug reaction (ADR) identification research. However, in the existing methods, the neglect of noise data and the need for manually labeled data reduce the accuracy of the prediction results and greatly increase manual labor. We propose a novel architecture named the weakly supervised mechanism (WSM) convolutional neural network (CNN) long-short-term memory (WSM-CNN-LSTM), which combines the strength of CNN and bi-directional long short-term memory (Bi-LSTM). The WSM applies the weakly labeled data to pre-train the parameters of the model and then uses the labeled data to fine-tune the initialized network parameters. The CNN employs a convolutional layer to study the characteristics of the drug reviews and active features at different scales, and then the feed-forward and feed-back neural networks of the Bi-LSTM utilize these salient features to output the regression results. The experimental results effectively demonstrate that our model marginally outperforms the comparison models in ADR identification and that a small quantity of labeled samples results in an optimal performance, which decreases the influence of noise and reduces the manual data-labeling requirements.

To develop and evaluate deep learning models for cardiac arrest rhythm classification during cardiopulmonary resuscitation (CPR), we analyzed 508 electrocardiogram (ECG) segments (each 4 s in duration, recorded at 250 Hz) from 131 cardiac arrest patients. Compression-affected segments were recorded during chest compressions, while non-compression segments were extracted during compression pauses or immediately after return of spontaneous circulation (ROSC) declaration. One-dimensional convolutional neural network (1D-CNN) and recurrent neural network (RNN) models were employed for four binary classification tasks: (1) shockable rhythms (VF and pVT) versus non-shockable rhythms (asystole and PEA) in all ECG segments; (2) the same classification in compression-affected ECG segments; (3) pulse-generating rhythms (ROSC rhythm) versus non-pulse-generating rhythms (asystole, PEA, VF and pVT) in all ECG segments; and (4) the same classification in compression-affected ECG segments. The 1D-CNN model consistently outperformed the RNN model across all classification tasks. For shockable versus non-shockable rhythm classification, the 1D-CNN achieved accuracies of 91.3% and 89.8% for all ECG segments and compression-affected ECG segments, respectively, compared to 50.6% and 54.5% for the RNN. In detecting pulse-generating rhythms, the 1D-CNN demonstrated accuracies of 90.9% and 85.7% for all ECG segments and compression-affected ECG segments, respectively, while the RNN achieved 92.2% and 84.4%. The 1D-CNN model demonstrated superior performance in cardiac arrest rhythm classification, maintaining high accuracy even with compression-affected ECG data.

In order to improve the accuracy of capsule network in disentangled representation, and further expand its application in computer vision, a novel BDARS_CapsNet (bi-directional attention routing sausage capsule network) architecture is proposed in this paper. Firstly, the bi-directional routing, namely bottom-up and top-down attention is used to achieve information feed-forward and feedback mechanism, which contributes to describing the attributes of object entity more accurately and completely. Secondly, inspired by the concept of covering learning, the sausage measure model is introduced into the network. The sausage model measures both the similarities and differences of the capsules and projects them into a more complex curved surface, which makes it possible to approximate any nonlinear function with arbitrary precision and preserving the local responsiveness of capsule entity to the maximum. Finally, the BDARS_CapsNet combines the CNN (Convolutional Neural Network), bi-directional attention routing, and sausage measure into capsule network modeling, and makes full use of high-level category information and low-level vision information; as a result, the reconstruction and classification accuracy is accordingly improved. Experiments demonstrate the effectiveness of proposed information routing, sausage measure, and new framework. Furthermore, the proposed BDARS_CapsNet provides a foundation for future research on disentangled representation learning.

Multimodal emotion recognition has become a hot topic in human-computer interaction and intelligent healthcare fields. However, combining information from different human different modalities for emotion computation is still challenging. In this paper, we propose a three-dimensional convolutional recurrent neural network model (referred to as 3FACRNN network) based on multimodal fusion and attention mechanism. The 3FACRNN network model consists of a visual network and an EEG network. The visual network is composed of a cascaded convolutional neural network-time convolutional network (CNN-TCN). In the EEG network, the 3D feature building module was added to integrate band information, spatial information and temporal information of the EEG signal, and the band attention and self-attention modules were added to the convolutional recurrent neural network (CRNN). The former explores the effect of different frequency bands on network recognition performance, while the latter is to obtain the intrinsic similarity of different EEG samples. To investigate the effect of different frequency bands on the experiment, we obtained the average attention mask for all subjects in different frequency bands. The distribution of the attention masks across the different frequency bands suggests that signals more relevant to human emotions may be active in the high frequency bands γ (31-50 Hz). Finally, we try to use the multi-task loss function Lc to force the approximation of the intermediate feature vectors of the visual and EEG modalities, with the aim of using the knowledge of the visual modalities to improve the performance of the EEG network model. The mean recognition accuracy and standard deviation of the proposed method on the two multimodal sentiment datasets DEAP and MAHNOB-HCI (arousal, valence) were 96.75 ± 1.75, 96.86 ± 1.33; 97.55 ± 1.51, 98.37 ± 1.07, better than those of the state-of-the-art multimodal recognition approaches. The experimental results show that starting from the multimodal information, the facial video frames and electroencephalogram (EEG) signals of the subjects are used as inputs to the emotion recognition network, which can enhance the stability of the emotion network and improve the recognition accuracy of the emotion network. In addition, in future work, we will try to utilize sparse matrix methods and deep convolutional networks to improve the performance of multimodal emotion networks.

1 - Congruence of deep learning in biomedical engineering: future prospects and challenges

Development in Computer vision applications, most prominently in Artificial Intelligence (AI) over the past decade, has encouraged researchers to use computational algorithms. Machine Learning (ML) and Deep Learning (DL) algorithms are getting attention in the biomedical area to early diagnosis and prognosis of lethiferous diseases with higher accuracy using statistical analysis. Cervical cancer is one of the predominant gynecological infectious forms of cancer in females ecumenically, generally transpiring in less-developed nations. DL algorithms are more optimized and convenient due to the automatic feature extraction in Cervical lesion detection at an early phase. Here we analyze the convolutional neural network (CNN) models' performance using variation in channels depth comprises in the convolution layers to classify multi-class liquid-based cytology (LBC) Whole slide images (WSI). We propose three CNN models having two convolution layers with the number of channels (4,8), (8,16), and (32,64) and two pooling layers. CNN model with the highest number of channels in the convolution layers gives the leading performance with an accuracy of 96.89%, precision of 93.38%, the sensitivity of 93.75%, and F-score of 94.15%.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

Among the many deep learning methods, the convolutional neural network (CNN) model has an excellent performance in image recognition. Research on identifying and classifying image datasets using CNN is ongoing. Animal species recognition and classification with CNN is expected to be helpful for various applications. However, sophisticated feature recognition is essential to classify quasi-species with similar features, such as the quasi-species of parrots that have a high color similarity. The purpose of this study is to develop a vision-based mobile application to classify endangered parrot species using an advanced CNN model based on transfer learning (some parrots have quite similar colors and shapes). We acquired the images in two ways: collecting them directly from the Seoul Grand Park Zoo and crawling them using the Google search. Subsequently, we have built advanced CNN models with transfer learning and trained them using the data. Next, we converted one of the fully trained models into a file for execution on mobile devices and created the Android package files. The accuracy was measured for each of the eight CNN models. The overall accuracy for the camera of the mobile device was 94.125%. For certain species, the accuracy of recognition was 100%, with the required time of only 455 ms. Our approach helps to recognize the species in real time using the camera of the mobile device. Applications will be helpful for the prevention of smuggling of endangered species in the customs clearance area.

Machine Learning can be used to process a lot of data and learn patterns from that data to predict the future. One of the most widely used parts of machine learning is Deep Learning. The Deep Learning method that currently provides the most significant results in image recognition is Convolutional Neural Network (CNN). Convolutional Neural Network (CNN) is one of the deep learning algorithms used for computer vision use cases such as image or video classification and detecting objects within images or even image areas. Some research related to the CNN model states that this model has a very good accuracy of 92% but with a fairly small amount of data and the use of epochs, namely 100, resulting in a higher validation error value than the error value in the training process, so that over fitting will occur. Based on several problems in the related research literature, this article aims to identify the weaknesses and shortcomings of Deep Learning algorithms using CNN models that refer to the state of the art, so that they can be used as a reference for further research. The state of the art related to research in the last five years, the Deep Learning algorithm using the CNN model found that (1) The number of epochs can affect the accuracy of the CNN model, (2) 2. The application of architecture can affect the accuracy of the CNN model, (3) the application of the type of layer can affect the accuracy of the CNN model. Based on several problems in the research literature related to the identification of weaknesses and shortcomings of Deep Learning using the CNN model which refers to Table 1. State of the Art summary of literature review research for the last five years, it can be concluded that to increase the accuracy of the CNN model, it is necessary to increase the number of epochs, apply the right architecture according to the problems in the research conducted, and use the type of layer. The hypothesis of this article can be used as a reference for further research related to Deep Learning using the CNN model.

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

Because it is so difficult to distinguish handwritten digits, digit identification is one of the most critical applications in computer vision. This is one of the reasons why it is so tough. The field of handwritten character recognition is one in which a great deal of application of numerous deep learning models has occurred. The startling parallels that can be drawn between deep learning and the brain are primarily responsible for its meteoric rise in popularity. In this study, the Artificial Neural Network and the Convolutional Neural Network, two of the most used Deep Learning algorithms, were investigated with an eye toward the recognition process's feature extraction and classification phases. With the assistance of the categorical cross-entropy loss and the ADAM optimizer, the models were trained on the MNIST dataset. Backpropagation and gradient descent are the two methods utilized during the training process of neural networks that contain reLU activations and carry out automatic feature extraction. In computer vision, one of the most common and widely used classifiers is the Convolution Neural Network, sometimes referred to as ConvNets or Convolutional neural networks. This network is used for the recognition and categorization of images.

This work aims to construct and optimize the public emotion network communication model under the artificial intelligence (AI) environment and provide a new research direction for exploring public emotion network communication with intelligent human–computer interaction (HCI). The main challenge of this research is the lack of interactivity of intelligent human–computer expression design, and the accuracy of emotion recognition is not high. First, the theoretical analysis of network public opinion is made, and the network public opinion calculation method and emotion recognition method based on deep learning are proposed. On this basis, the public emotion network communication model is established, and the model’s communication mechanism and rules are described in detail. Next, the emotion calculation of human–computer expression interaction and the emotion expression mode of the humanoid robot are proposed, which provides an experimental basis for HCI and public emotion network communication analysis. According to the data of email network and microblog network in the real network, the network degree distribution map is established. The statistical results show that email and microblog networks belong to power law distribution, and the aggregation coefficient is relatively high. Microblog network degree changes slowly, and more points are scattered in the middle area. In a practical sense, when the positive and negative emotions in the public emotion network communication model are input into the bionic robot, the robot can show happy and lost expressions. The results show that compared with the traditional convolutional neural network (CNN) model or recurrent neural network (RNN) model, the RNN–CNN structure model of RNN combined with CNN reduces the waiting time by about 20% and improves the algorithm accuracy by at least 3.1%. The public emotion network communication model based on deep learning and intelligent HCI can accurately reflect the public emotion state in the network, which provides a practical basis for the application of AI technology in network public opinion judgment.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.