Binaural source localization using deep learning and head rotation information

Deep sentiments in Roman Urdu text using Recurrent Convolutional Neural Network model

Multimodal emotion recognition has become a hot topic in human-computer interaction and intelligent healthcare fields. However, combining information from different human different modalities for emotion computation is still challenging. In this paper, we propose a three-dimensional convolutional recurrent neural network model (referred to as 3FACRNN network) based on multimodal fusion and attention mechanism. The 3FACRNN network model consists of a visual network and an EEG network. The visual network is composed of a cascaded convolutional neural network-time convolutional network (CNN-TCN). In the EEG network, the 3D feature building module was added to integrate band information, spatial information and temporal information of the EEG signal, and the band attention and self-attention modules were added to the convolutional recurrent neural network (CRNN). The former explores the effect of different frequency bands on network recognition performance, while the latter is to obtain the intrinsic similarity of different EEG samples. To investigate the effect of different frequency bands on the experiment, we obtained the average attention mask for all subjects in different frequency bands. The distribution of the attention masks across the different frequency bands suggests that signals more relevant to human emotions may be active in the high frequency bands γ (31-50 Hz). Finally, we try to use the multi-task loss function Lc to force the approximation of the intermediate feature vectors of the visual and EEG modalities, with the aim of using the knowledge of the visual modalities to improve the performance of the EEG network model. The mean recognition accuracy and standard deviation of the proposed method on the two multimodal sentiment datasets DEAP and MAHNOB-HCI (arousal, valence) were 96.75 ± 1.75, 96.86 ± 1.33; 97.55 ± 1.51, 98.37 ± 1.07, better than those of the state-of-the-art multimodal recognition approaches. The experimental results show that starting from the multimodal information, the facial video frames and electroencephalogram (EEG) signals of the subjects are used as inputs to the emotion recognition network, which can enhance the stability of the emotion network and improve the recognition accuracy of the emotion network. In addition, in future work, we will try to utilize sparse matrix methods and deep convolutional networks to improve the performance of multimodal emotion networks.

The latest wireless sensor network (WSN) developments in critical applications have introduced security risks, like jamming. Intrusion Detection System (IDS) in WSN is the method of recognizing malevolent or unauthorized activities in the network. The intruder's presence to launch different attacks within the network cannot be disregarded. Despite a great deal of effort by the researcher workers, IDS still experienced difficulties enhancing recognition performance while minimizing the false alarm rate and identifying novel intrusions. Recently, Deep Learning (DL) and Machine Learning (ML) based IDS system has been deployed as promising solution to effectively identify intrusion across the network. Therefore, the study presents a Moth Search Optimization with DL-based Intrusion Detection (MSODL-ID) method in the WSN. The MSODL-ID technique aims to effectually identify the occurrence of malicious activities or intrusions in the network. To accomplish this, the MSODL-ID technique undergoes two stages of preprocessing: data conversion and data scaling. In addition, the MSODL-ID technique employs Convolutional Recurrent Neural Network (CRNN) model with a Hopfield layer for intrusion detection purposes. For optimal hyperparameter selection of the CRNN model, the MSO algorithm is used and thereby enhances the classification performance of the CRNN model. The stimulation analysis of the MSODL-ID system is tested by means of Kaggle datasets, and the outcomes exhibit the promising performance of the MSODL-ID system over other current DL approaches.

The presented paper proposes a new speech command recognition model for novel engineering applications with limited resources. We built the proposed model with the help of a Convolutional Recurrent Neural Network (CRNN). The use of CRNN instead of Convolutional Neural Network (CNN) helps us to reduce the model parameters and memory requirement as per resource constraints. Furthermore, we insert transmute and curtailment layer between the layers of CRNN. By doing this we further reduce model parameters and float number of operations to half of the CRNN requirement. The proposed model is tested on Google’s speech command dataset. The obtained result shows that the proposed CRNN model requires 1/3 parameters as compared to the CNN model. The number of parameters of the CRNN model is further reduced by 45% and the float numbers of operations between 2% to 12 % in different recognition tasks. The recognition accuracy of the proposed model is 96% on Google’s speech command dataset, and on laboratory recording, its recognition accuracy is 89%.

Self-attention based recurrent convolutional neural network for disease prediction using healthcare data

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

This paper presents a viewpoint from computer vision to the radar echo extrapolation task in the precipitation nowcasting domain. Inspired by the success of some convolutional recurrent neural network models in this domain, including convolutional LSTM, convolutional GRU and trajectory GRU, we designed a new sequence-to-sequence neural network structure to leverage these models in a realistic data context. In this design, we decreased the numbers of channels in high abstract recurrent layers rather than increasing them. We formulated the task as a problem of encoding five radar images and predicting 10 steps ahead at the pixel level, and found that using only the common mean squared error can misguide the training and mislead the testing. Especially, the image quality of last predictions usually degraded rapidly. As a solution, we employed some visual image quality assessment techniques including Structural Similarity (SSIM) and multi-scale SSIM to train our models. Experimental results show that our structure was more tolerant to increasing uncertainty in the data, and the use of image quality metrics can significantly reduce the blurry image issue. Moreover, we found that using SSIM was very effective and a combination of SSIM with mean squared error and mean absolute error yielded the best prediction quality.

The lack of labeled data is a major obstacle in many music information retrieval tasks such as melody extraction, where labeling is extremely laborious or costly. Semi-supervised learning (SSL) provides a solution to alleviate the issue by leveraging a large amount of unlabeled data. In this paper, we propose an SSL method using teacher-student models for vocal melody extraction. The teacher model is pre-trained with labeled data and guides the student model to make identical predictions given unlabeled input in a self-training setting. We examine three setups of teacher-student models with different data augmentation schemes and loss functions. Also, considering the scarcity of labeled data in the test phase, we artificially generate large-scale testing data with pitch labels from unlabeled data using an analysis-synthesis method. The results show that the SSL method significantly increases the performance against supervised learning only and the improvement depends on the teacher-student models, the size of unlabeled data, the number of self-training iterations, and other training details. We also find that it is essential to ensure that the unlabeled audio has vocal parts. Finally, we show that the proposed SSL method allows a simple convolutional recurrent neural network model to achieve performance comparable to state-of-the-arts.

Introduction: The early automatic diagnosis of the novel coronavirus (COVID-19) disease could be very helpful to reduce its spread around the world. In this study, we revisit the identification of COVID-19 from chest X-ray images using deep learning. Methods: We collected a relatively large COVID-19 dataset—compared with previous studies—containing 309 real COVID-19 chest X-ray images. We also prepared 2,000 chest X-ray images of pneumonia cases and 1,000 images of healthy controls. Deep transfer learning was used to detect abnormalities in our image dataset. We fine-tuned three, pre-trained convolutional neural network (CNN) models on a training dataset: DenseNet 121, NASNetLarge, and NASNet-Mobile. Results: The evaluation of our models on a test dataset showed that these models achieved an average sensitivity rate of approximately 99.45% and an average specificity rate of approximately 99.5%. Conclusion: A larger dataset of COVID-19 X-ray images could lead to more accurate and reliable identification of COVID-19 infections using deep transfer learning. However, the clinical diagnosis of COVID-19 disease is always necessary.

Bone cancer is a significant health issue that leads to severe complications and deaths worldwide. Early detection can significantly improve patient outcomes, often resulting in a complete cure. Traditional approaches to managing bone cancer can be augmented with technology-driven methods, particularly those enabled by artificial intelligence. Convolutional Neural Networks (CNNs) and their variants have proven effective in medical data analysis. However, deep learning techniques require enhancements to improve performance in cancer detection across various medical imaging modalities. In this paper, we propose a deep-learning framework that utilizes advanced CNN models for the automatic screening of bone cancer. We have enhanced both the CNN model and the ResNet-50 model, which are integral components of the proposed framework. Additionally, we introduced an algorithm called Learning-based Bone Cancer Detection (LbBCD), designed to optimize the utilization of these enhanced deep learning models to improve bone cancer detection efficiency. Our research emphasizes a Region of Interest (ROI) based approach to enhance the screening process for bone cancer. Using a benchmark dataset known as the Bone CT Scan dataset, our empirical study demonstrated that the proposed deep learning framework, integrated with enhanced CNN and ResNet-50 models, achieved remarkable performance. Specifically, the enhanced CNN model reached an accuracy of 90.90%, while the enhanced ResNet-50 model achieved an accuracy of 92.20%, outperforming state-of-the-art deep learning models. Therefore, this proposed system can be integrated into healthcare applications for automatically screening bone cancer, contributing to a clinical decision support system for healthcare professionals.

PurposeTo develop and compare deep learning (DL) algorithms to detect keratoconus on the basis of corneal topography and validate with visualization methods.MethodsWe retrospectively collected corneal topographies of the study group with clinically manifested keratoconus and the control group with regular astigmatism. All images were divided into training and test datasets. We adopted three convolutional neural network (CNN) models for learning. The test dataset was applied to analyze the performance of the three models. In addition, for better discrimination and understanding, we displayed the pixel-wise discriminative features and class-discriminative heat map of diopter images for visualization.ResultsOverall, 170 keratoconus, 28 subclinical keratoconus and 156 normal topographic pictures were collected. The convergence of accuracy and loss for the training and test datasets after training revealed no overfitting in all three CNN models. The sensitivity and specificity of all CNN models were over 0.90, and the area under the receiver operating characteristic curve reached 0.995 in the ResNet152 model. The pixel-wise discriminative features and the heat map of the prediction layer in the VGG16 model both revealed it focused on the largest gradient difference of topographic maps, which was corresponding to the diagnostic clues of ophthalmologists. The subclinical keratoconus was positively predicted with our model and also correlated with topographic indexes.ConclusionsThe DL models had fair accuracy for keratoconus screening based on corneal topographic images. The visualization mentioned in the current study revealed that the model focused on the appropriate region for diagnosis and rendered clinical explainability of deep learning more acceptable.Translational RelevanceThese high accuracy CNN models can aid ophthalmologists in keratoconus screening with color-coded corneal topography maps.

Noninvasive diagnosis of internal traits in fruit crops is a high unmet need; however it generally requires time, costs, and special methods or facilities. Recent progress in deep neural network (or deep learning) techniques would allow easy, but highly accurate diagnosis with single RGB images, and the latest applications enable visualization of “the reasons for each diagnosis” by backpropagation of neural networks. Here, we propose an application of deep learning for image diagnosis on the classification of internal fruit traits, in this case seedlessness, in persimmon fruit (Diospyros kaki). We examined the classification of seedlessness in persimmon fruit by using four convolutional neural networks (CNN) models with various layer structures. With only 599 pictures of ‘Fuyu’ persimmon fruit from the fruit apex side, the neural networks successfully made a binary classification of seedless and seeded fruits with up to 85% accuracy. Among the four CNN models, the VGG16 model with the simplest layer structure showed the highest classification accuracy of 89%. Prediction values for the binary classification of seeded fruits were significantly increased in proportion to seed numbers in all four CNN models. Furthermore, explainable AI methods, such as Gradient-weighted Class Activation Mapping (Grad-CAM) and Guided Grad-CAM, allowed visualization of the parts and patterns contributing to the diagnosis. The results indicated that finer positions surrounding the apex, which correspond to hypothetical bulges derived from seeds, are an index for seeded fruits. These results suggest the novel potential of deep learning for noninvasive diagnosis of fruit internal traits using simple RGB images and also provide novel insights into previously unrecognized features of seeded/seedless fruits.

A case study on computer-aided diagnosis of nonerosive reflux disease using deep learning techniques

BACKGROUND AND AIM: Outdoor ultrafine particles (UFP; 100 nm) and black carbon (BC) vary greatly within cities and may have adverse impacts on human health such as cardiovascular mortality and brain tumour incidence. Traditional regression methods for estimating spatial distributions of ambient air pollution depend on extensive, curated geospatial information system databases. This has resulted in a disparity wherein little is known about local air pollution in the data-sparse settings of low- and middle- income countries. We used a hybrid approach to develop new models to estimate within-city spatial variations in outdoor UFP and BC concentrations across Bucaramanga, Colombia. METHODS: We conducted a large-scale mobile monitoring campaign over twenty days in 2019. Land use regression (LUR) models were developed using land use parameters from curated and open-source databases. Convolutional Neural Network (CNN) models were developed using Google Maps satellite view and street view images. Combined models were developed by combining predictions from the LUR and CNN models. After training in the training and validation sets, predictions were generated in the test set and compared to measured values. RESULTS:The combined UFP model (R2=0.54) outperformed the CNN (R2=0.47) and land use regression (LUR) models (R2=0.47) on their own. Similarly, the combined BC model also outperformed the CNN and LUR BC models (R2 = 0.51 vs 0.43 and 0.45 respectively). Spatial variations in model performance were more stable for the CNN and combined models compared to the LUR models suggesting that the combined approach may be less likely to contribute to differential exposure measurement error in epidemiological studies. CONCLUSIONS:Estimates from these models can then be applied to population-based cohorts in order to evaluate population health risks. Additionally, our findings demonstrated that satellite and street-level images can be combined with a traditional LUR modelling approach to improve predictions of within-city spatial variations in outdoor UFP and BC concentrations. KEYWORDS: Ultrafine particles, Black Carbon, Deep learning, Images, Land use regression

Predicting the thermal conductivity of polymer composites with one-dimensional oriented fillers using the combination of deep learning and ensemble learning