A Hybrid Deep Learning Heart Disease Prediction Framework Utilizing Multi-Modal Medical Imaging and Novel Feature Fusion Techniques

Automatically recognizing facial expressions in video sequences is a challenging task because there is little direct correlation between facial features and subjective emotions in video. To overcome the problem, a video facial expression recognition method using spatiotemporal recurrent neural network and feature fusion is proposed. Firstly, the video is preprocessed. Then, the double-layer cascade structure is used to detect a face in a video image. In addition, two deep convolutional neural networks are used to extract the time-domain and airspace facial features in the video. The spatial convolutional neural network is used to extract the spatial information features from each frame of the static expression images in the video. The temporal convolutional neural network is used to extract the dynamic information features from the optical flow information from multiple frames of expression images in the video. A multiplication fusion is performed with the spatiotemporal features learned by the two deep convolutional neural networks. Finally, the fused features are input to the support vector machine to realize the facial expression classification task. The experimental results on cNTERFACE, RML, and AFEW6.0 datasets show that the recognition rates obtained by the proposed method are as high as 88.67%, 70.32%, and 63.84%, respectively. Comparative experiments show that the proposed method obtains higher recognition accuracy than other recently reported methods.

Spoofing speech detection aims to differentiate spoofing speech from natural speech. Frame-based features are usually used in most of previous works. Although multiple frames or dynamic features are used to form a super-vector to represent the temporal information, the time span covered by these features are not sufficient. Most of the systems failed to detect the non-vocoder or unit selection based spoofing attacks. In this work, we propose to use a temporal convolutional neural network (CNN) based classifier for spoofing speech detection. The temporal CNN first convolves the feature trajectories with a set of filters, then extract the maximum responses of these filters within a time window using a max-pooling layer. Due to the use of max-pooling, we can extract useful information from a long temporal span without concatenating a large number of neighbouring frames, as in feedforward deep neural network (DNN). Five types of feature are employed to access the performance of proposed classifier. Experimental results on ASVspoof 2015 corpus show that the temporal CNN based classifier is effective for synthetic speech detection. Specifically, the proposed method brings a significant performance boost for the unit selection based spoofing speech detection.

In addressing the problem of low prediction accuracy in the remaining useful life (RUL) prediction of rolling bearings, caused by noise interference and insufficient extraction of sensitive features by deep learning models, this paper presents a life prediction method based on signal reconstruction and dual-channel network fusion. First, addressing the issue of extracting weak features from rolling bearing vibration signals, an optimized combination of variational mode decomposition (VMD) and Teager–Kaiser energy operator (TKEO) for signal reconstruction is proposed. TKEO is used to track the energy of high-frequency impulses in the original signal. The sparrow search algorithm is employed for optimizing VMD to perform high-frequency filtering, resulting in an optimized impulse energy signal. This signal is then multiplied with the original signal to enhance weak impulse features under noisy background. Next, a dual-channel network model for RUL prediction is constructed based on temporal convolutional network (TCN) and convolutional neural networks (CNNs). The one-dimensional time-series signal obtained after reconstruction and denoising serves as the input for the TCN network channel. Simultaneously, the signal obtained after reconstruction and denoising undergoes synchrosqueezed wavelet transforms to generate a two-dimensional time–frequency representation as input for the CNN network channel. This setup allows TCN and CNN to respectively extract temporal features from the vibration signal and time–frequency features from the spectrogram. By incorporating self-attention mechanisms, internal correlations between different features are fully explored, thereby enhancing prediction accuracy.

The auditory brain circuits are biologically constructed to recognize and localize sounds by encoding a combination of cues that help individuals interpret sounds. The development of computational methods inspired by human capacities has established opportunities for improving machine hearing. Recent studies based on deep learning show that using convolutional recurrent neural networks (CRNNs) is a promising approach for sound event detection and localization in spatial sound. Nevertheless, depending on the sound environment, the performance of these systems is still far from reaching perfect metrics. Therefore, this work intends to boost the performance of state-of-the-art (SOTA) systems by using bio-inspired gammatone auditory filters and intensity vectors (IVs) for the acoustic feature extraction stage, along with the implementation of a temporal convolutional network (TCN) block into a CRNN model, to capture long term dependencies. Three data augmentation techniques are applied to increase the small number of samples in spatial audio datasets. The mentioned stages constitute our proposed Gammatone-based Sound Events Localization and Detection (G-SELD) system, which exceeded the SOTA results on four spatial audio datasets with different levels of acoustical complexity and with up to three sound sources overlapping in time.

Research on smartphone-based human activity recognition (HAR) is prevalent in the field of healthcare, especially for elderly activity monitoring. Researchers usually propose to use of accelerometers, gyroscopes or magnetometers that are equipped in smartphones as an individual sensing modality for human activity recognition. However, any of these alone is limited in capturing comprehensive movement information for accurate human activity analysis. Thus, we propose a smartphone-based HAR approach by leveraging the inertial signals captured by these three sensors to classify human activities. These heterogeneous sensors deliver information on various aspects of nature, motion and orientation, offering a richer set of features for more accurate representations of the activities. Hence, a deep learning approach that amalgamates long short-term memory (LSTM) in temporal convolutional network (TCN) is proposed. We use independent temporal convolutional networks, coined as temporal convolutional streams, to independently analyse the temporal data of each sensing modality. We name this architecture multi-stream TC-LSTM. The performance of multi-stream TC-LSTM is assessed on the self-collected elderly activity database. Empirical results exhibit that multi-stream TC-LSTM outperforms the existing machine learning and deep learning models, with an F1 score of 98.3 %

Data in time series format, such as biological signals from medical sensors or machine signals from sensors in industrial environments are rich sources of information that can give crucial insights on the present and future condition of a person or machine. The task of predicting future values of time series has been initially approached with simple machine learning methods, and lately with deep learning. Two models that have shown good performance in this task are the temporal convolutional network and the attention module. However, despite the promising results of deep learning methods, their black-box nature makes them unsuitable for real-world applications where the predictions need to be explainable in order to be trusted. In this paper we propose an architecture comprised of a temporal convolutional network with an attention mechanism that makes predictions while presenting the timesteps of the input that were most influential for future outputs. We apply it on two datasets and we show that we gain interpretability without degrading the accuracy compared to the original temporal convolutional models. We then go one step further and we combine our configuration with various machine learning methods on top, creating a pipeline that achieves interpretability both across timesteps and input features. We use it to forecast a different variable from one of the above datasets and we study how the accuracy is affected compared to the original black-box approach.

This paper presents an accurate state of health (SOH) estimation algorithm using a temporal convolutional neural network (TCN) for lithium-ion batteries (LIB). With its self-learning ability, this data-driven approach can model the highly non-linear behaviour of LIB due to changes of environment and working conditions all along the battery lifetime. The precise SOH predictions of the TCN are especially needed to ensure a safe and efficient usage of retired electric vehicle batteries within second-life applications. The provided network is trained and tested with data gathered from commercial industry applications in the domain of energy storage. It is shown, that even for dynamic load profiles, the TCN achieves a mean squared error (MSE) of less than 1.0 %. Using this approach, the uncertainty of the heterogeneous performances and characteristics of retired electric vehicle batteries can be drastically reduced.

Locomotion mode recognition has been shown to substantially contribute to the precise control of robotic lower-limb prostheses under different walking conditions. In this study, we proposed a temporal convolutional capsule network (TCCN) which integrates the spatial-temporal-based, dilation-convolution-based, dyna- mic routing and vector-based features for recognizing locomotion mode recognition with small data rather than big-data-based neural networks for robotic prostheses. TCCN proposed in this study has four characteristics, which extracts the (1) spatial-temporal information in the data and then makes (2) dilated convolution to deal with small data, and uses (3) dynamic routing, which produces some similarities to the human brain to process the data as a (4) vector, which is different from other scalar-based networks, such as convolutional neural network (CNN). By comparison with a traditional machine learning, e.g., support vector machine(SVM) and big-data-driven neural networks, e.g., CNN, recurrent neural network(RNN), temporal convolutional network(TCN) and capsule network(CN). The accuracy of TCCN is 4.1% higher than CNN under 5-fold cross-validation of three-locomotion-mode and 5.2% higher under the 5-fold cross-validation of five-locomotion modes. The main confusion we found appears in the transition state. The results indicate that TCCN may handle small data balancing global and local information which is closer to the way how the human brain works, and the capsule layer allows for better processing vector information and retains not only magnitude information, but also direction information.

COVID-DSNet: A novel deep convolutional neural network for detection of coronavirus (SARS-CoV-2) cases from CT and Chest X-Ray images

Non-Intrusive Residential Electricity Load Disaggregation Based on Temporal Convolutional Neural Network

Aims This research paper aims to check the effectiveness of a variety of machine learning models in classifying esophageal cancer through MRI scans. The current study encompasses Convolutional Neural Network (CNN), K-Nearest Neighbor (KNN), Recurrent Neural Network (RNN), and Visual Geometry Group 16 (VGG16), among others which are elaborated in this paper. This paper aims to identify the most accurate model to facilitate increased, improved diagnostic accuracy to revolutionize early detection methods for this dreadful disease. The ultimate goal is, therefore, to improve the clinical practice performance and its results with advanced machine learning techniques in medical diagnosis. Background Esophageal cancer poses a critical problem for medical oncologists since its pathology is quite complex, and the death rate is exceptionally high. Proper early detection is essential for effective treatment and improved survival. The results are positive, but the conventional diagnostic methods are not sensitive and have low specificity. Recent progress in machine learning methods brings a new possibility to high sensitivity and specificity in the diagnosis. This paper explores the potentiality of different machine-learning models in classifying esophageal cancer through MRI scans to complement the constraints of the traditional diagnostics approach. Objective This study is aimed at verifying whether CNN, KNN, RNN, and VGG16, amongst other advanced machine learning models, are effective in correctly classifying esophageal cancer from MRI scans. This review aims at establishing the diagnostic accuracy of all these models, with the best among all. It plays a role in developing early detection mechanisms that increase patient outcome confidence in the clinical setting. Methods This study applies the approach of comparative analysis by using four unique machine learning models to classify esophageal cancer from MRI scans. This was made possible through the intensive training and validation of the model using a standardized set of MRI data. The model’s effectiveness was assessed using performance evaluation metrics, which included accuracy, precision, recall, and F1 score. Results In classifying esophageal cancers from MRI scans, the current study found VGG16 to be an adequate model, with a high accuracy of 96.66%. CNN took the second position, with an accuracy of 94.5%, showing efficient results for spatial pattern recognition. The model of KNN and RNN also showed commendable performance, with accuracies of 91.44% and 88.97%, respectively, portraying their strengths in proximity-based learning and handling sequential data. These findings underline the potential to add significant value to the processes of esophageal cancer diagnosis using machine learning models. Conclusion The study concluded that machine learning techniques, mainly VGG16 and CNN, had a high potential for escalated diagnostic precision in classifying esophageal cancer from MRI imaging. VGG16 showed great accuracy, while CNN displayed advanced spatial detection, followed by KNN and RNN. Thus, the results set new opportunities for introducing advanced computational models to the clinics, which might transform strategies for early detection to improve patient-centered outcomes in oncology.

AC arc faults are one of the most important causes of residential electrical wiring fires, which may produce extremely high temperatures and easily ignite surrounding combustible materials. The global interest in machine learning-based methods for arc fault diagnosis applications is increasing due to continuous challenges in efficiency and accuracy. In this paper, a temporal domain visualization convolutional neural network (TDV-CNN) methodology is proposed. The current transformer and high-speed data acquisition system are used to collect the current of a series of arc faults, then the signal is filtered by a digital filter and converted into a gray image in time sequence before being fed into TDV-CNN. Five different electric loads were selected for experimental validation with various signal characteristics, including vacuum cleaner, fluorescent lamp, dimmer, heater, and desktop computer. The experimental results confirm that the classification accuracy of the five loads’ work states in the ten categories could reach 98.7% or even higher by adjusting parameters perfectly. The methodology is believed to be reliable for series arc detection with relatively high accuracy and also has important potential applications in other fault diagnosis fields.

Bearings play a crucial role in rotating machinery, and their failure significantly impacts performance and safety. Early fault detection is challenging due to weak signals, especially under variable operating conditions. Deep learning techniques, such as Convolutional Neural Networks (CNN) and attention mechanisms, have shown great potential in improving fault detection accuracy in recent years. However, handling complex fault patterns and extracting early fault features remain challenging. To address these issues, this paper proposes a new method that combines Temporal Convolutional Networks (TCN), CNN, and Multi-Head Attention. TCN captures short-term dependencies, CNN extracts long-term features, and Multi-Head Attention enhances focus on key fault-related features. The model was tested on the Case Western Reserve University (CWRU) bearing fault dataset and compared with baseline models, showing a significant improvement in accuracy, achieving 99.3%. This study provides valuable insights for handling complex and multi-fault scenarios, offering essential implications for industrial applications.

Efficient identification of key nodes is crucial to optimizing detector deployment and enhancing traffic monitoring in intelligent transportation systems. However, existing approaches often struggle to adapt to dynamic traffic variations, leading to suboptimal coverage and increased deployment costs. We propose bidirectional temporal convolutional graph attention networks (BTC-GATs) to address these limitations. This novel framework integrates bidirectional attention mechanisms to capture upstream and downstream dependencies, temporal convolutional networks for multiscale feature extraction, and graph attention networks for spatial information aggregation. BTC-GATs incorporates adaptive temporal modeling to capture nonlinear traffic dynamics, gradient-based variation analysis to quantify node influence, and a ranking mechanism that fuses attention coefficients with topological attributes to further enhance robustness and interoperability. In addition, a key node coverage study is conducted to examine the trade-off between accuracy and deployment efficiency. Extensive experiments on the California Highway PeMS04 dataset demonstrate that BTC-GATs outperforms benchmark methods in key node identification, offering superior accuracy and stability. Further analysis confirms its robustness under varying traffic conditions and initialization settings, highlighting its potential as a scalable, adaptive, and cost-effective solution for intelligent traffic monitoring. By facilitating efficient sensor placement, BTC-GATs contributes to improved data collection and congestion management in large-scale transportation networks.

Computer algorithms that simulate lower-doses computed tomography (CT) images from clinical-dose images are widely available. However, most operate in the projection domain and assume access to the reconstruction method. Access to commercial reconstruction methods may often not be available in medical research, making image-domain noise simulation methods useful. However, the introduction of non-linear reconstruction methods, such as iterative and deep learning-based reconstruction, makes noise insertion in the image domain intractable, as it is not possible to determine the noise textures analytically. To develop a deep learning-based image-domain method to generate low-dose CT images from clinical-dose CT (CDCT) images for non-linear reconstruction methods. We propose a fully image domain-based method, utilizing a series of three convolutional neural networks (CNNs), which, respectively, denoise CDCT images, predict the standard deviation map of the low-dose image, and generate the noise power spectra (NPS) of local patches throughout the low-dose image. All three models have U-net-based architectures and are partly or fully three-dimensional. As a use case for this study and with no loss of generality, we use paired low-dose and clinical-dose brain CT scans. A dataset of paired scans was retrospectively obtained. All images were acquired with a wide-area detector clinical system and reconstructed using its standard clinical iterative algorithm. Each pair was registered using rigid registration to correct for motion between acquisitions. The data was randomly partitioned into training ( samples), validation ( samples), and test ( samples) sets. The performance of each of these three CNNs was validated separately. For the denoising CNN, the local standard deviation decrease, and bias were determined. For the standard deviation map CNN, the real and estimated standard deviations were compared locally. Finally, for the NPS CNN, the NPS of the synthetic and real low-dose noise were compared inside and outside the skull. Two proof-of-concept denoising studies were performed to determine if the performance of a CNN- or a gradient-based denoising filter on the synthetic low-dose data versus real data differed. The denoising network had a median decrease in noise in the cerebrospinal fluid by a factor of and introduced a median bias of HU. The network for standard deviation map estimation had a median error of HU. The noise power spectrum estimation network was able to capture the anisotropic and shift-variant nature of the noise structure by showing good agreement between the synthetic and real low-dose noise and their corresponding power spectra. The two proof of concept denoising studies showed only minimal difference in standard deviation improvement ratio between the synthetic and real low-dose CT images with the median difference between the two being 0.0 and +0.05 for the CNN- and gradient-based filter, respectively. The proposed method demonstrated good performance in generating synthetic low-dose brain CT scans without access to the projection data or to the reconstruction method. This method can generate multiple low-dose image realizations from one clinical-dose image, so it is useful for validation, optimization, and repeatability studies of image-processing algorithms.