Improving Subject-Independent EEG Preference Classification Using Deep Learning Architectures with Dropouts

Human emotions play a key role in numerous decision-making processes. The ability to correctly identify likes and dislikes as well as excitement and boredom would facilitate novel applications in neuromarketing, affective entertainment, virtual rehabilitation and forensic neuroscience that leverage on sub-conscious human affective states. In this neuroinformatics investigation, we seek to recognize human preferences and excitement passively through the use of electroencephalography (EEG) when a subject is presented with some 3D visual stimuli. Our approach employs the use of machine learning in the form of deep neural networks to classify brain signals acquired using a brain-computer interface (BCI). In the first part of our study, we attempt to improve upon our previous work, which has shown that EEG preference classification is possible although accuracy rates remain relatively low at 61%-67% using conventional deep learning neural architectures, where the challenge mainly lies in the accurate classification of unseen data from a cohort-wide sample that introduces inter-subject variability on top of the existing intra-subject variability. Such an approach is significantly more challenging and is known as subject-independent EEG classification as opposed to the more commonly adopted but more time-consuming and less general approach of subject-dependent EEG classification. In this new study, we employ deep networks that allow dropouts to occur in the architecture of the neural network. The results obtained through this simple feature modification achieved a classification accuracy of up to 79%. Therefore, this study has shown that the use of a deep learning classifier was able to achieve an increase in emotion classification accuracy of between 13% and 18% through the simple adoption of the use of dropouts compared to a conventional deep learner for EEG preference classification. In the second part of our study, users are exposed to a roller-coaster experience as the emotional stimuli which are expected to evoke the emotion of excitement, while simultaneously wearing virtual reality goggles, which delivers the virtual reality experience of excitement, and an EEG headset, acquires the raw brain signals detected when exposed to this excitement stimuli. Here, a deep learning approach is used to improve the excitement detection rate to well above the 90% accuracy level. In a prior similar study, the use of conventional machine learning approaches involving k-Nearest Neighbour (kNN) classifiers and Support Vector Machines (SVM) only achieved prediction accuracy rates of between 65% and 89%. Using a deep learning approach here, rates of 78%-96% were achieved. This demonstrates the superiority of adopting a deep learning approach over other machine learning approaches for detecting human excitement when immersed in an immersive virtual reality environment.

Brain-Computer Interfaces (BCIs) leverage brain signals to facilitate communication and control, particularly benefiting individuals with motor impairments. Motor imagery (MI)-based BCIs, utilizing non-invasive electroencephalography (EEG), face challenges due to high signal variability, noise, and class overlap. Deep learning architectures, such as CNNs and LSTMs, have improved EEG classification but still struggle to fully capture discriminative features for overlapping motor imagery classes. This study introduces a hybrid deep neural architecture that integrates Convolutional Neural Networks, Long Short-Term Memory networks, and a novel SVM-enhanced attention mechanism. The proposed method embeds the margin maximization objective of Support Vector Machines directly into the self-attention computation to improve interclass separability during feature learning. We evaluate our model on four benchmark datasets: Physionet, Weibo, BCI Competition IV 2a, and 2b, using a Leave-One-Subject-Out (LOSO) protocol to ensure robustness and generalizability. Results demonstrate consistent improvements in classification accuracy, F1-score, and sensitivity compared to conventional attention mechanisms and baseline CNN-LSTM models. Additionally, the model significantly reduces computational cost, supporting real-time BCI applications. Our findings highlight the potential of SVM-enhanced attention to improve EEG decoding performance by enforcing feature relevance and geometric class separability simultaneously.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep learning has already been proved as a powerful state-of-the-art technique for many image understanding tasks in computer vision and other applications including remote sensing (RS) image analysis. Unmanned aircraft systems (UASs) offer a viable and economical alternative to a conventional sensor and platform for acquiring high spatial and high temporal resolution data with high operational flexibility. Coastal wetlands are among some of the most challenging and complex ecosystems for land cover prediction and mapping tasks because land cover targets often show high intra-class and low inter-class variances. In recent years, several deep convolutional neural network (CNN) architectures have been proposed for pixel-wise image labeling, commonly called semantic image segmentation. In this paper, some of the more recent deep CNN architectures proposed for semantic image segmentation are reviewed, and each model’s training efficiency and classification performance are evaluated by training it on a limited labeled image set. Training samples are provided using the hyper-spatial resolution UAS imagery over a wetland area and the required ground truth images are prepared by manual image labeling. Experimental results demonstrate that deep CNNs have a great potential for accurate land cover prediction task using UAS hyper-spatial resolution images. Some simple deep learning architectures perform comparable or even better than complex and very deep architectures with remarkably fewer training epochs. This performance is especially valuable when limited training samples are available, which is a common case in most RS applications.

Recent advances in deep learning have had a methodological and practical impact on brain-computer interface (BCI) research. Among the various deep network architectures, convolutional neural networks (CNNs) have been well suited for spatio-spectral-temporal electroencephalogram (EEG) signal representation learning. Most of the existing CNN-based methods described in the literature extract features at a sequential level of abstraction with repetitive nonlinear operations and involve densely connected layers for classification. However, studies in neurophysiology have revealed that EEG signals carry information in different ranges of frequency components. To better reflect these multi-frequency properties in EEGs, we propose a novel deep multi-scale neural network that discovers feature representations in multiple frequency/time ranges and extracts relationships among electrodes, i.e., spatial representations, for subject intention/condition identification. Furthermore, by completely representing EEG signals with spatio-spectral-temporal information, the proposed method can be utilized for diverse paradigms in both active and passive BCIs, contrary to existing methods that are primarily focused on single-paradigm BCIs. To demonstrate the validity of our proposed method, we conducted experiments on various paradigms of active/passive BCI datasets. Our experimental results demonstrated that the proposed method achieved performance improvements when judged against comparable state-of-the-art methods. Additionally, we analyzed the proposed method using different techniques, such as PSD curves and relevance score inspection to validate the multi-scale EEG signal information capturing ability, activation pattern maps for investigating the learned spatial filters, and t-SNE plotting for visualizing represented features. Finally, we also demonstrated our method's application to real-world problems. Based on our experimental results and analyses, we believe that the proposed multi-scale neural network can be useful for various BCI paradigms, as a starting model or as a backbone network in any new BCI experiments.

The spine is composed of pieces of bone called vertebrae that lie between the skull and the tailbone. Various medical conditions can affect the spine. In this study, two types of degenerative diseases, scoliosis, and spondylolisthesis, were studied. Deep AI architectures have recently enabled further disease diagnosis innovation using medical images. Various traditional and deep learning studies use medical images for disease diagnosis in the literature. This study aims to classify spine X-ray images according to three possible conditions (Normal, Scoliosis, and Spondylolisthesis) and to exploit the potential of these X-ray images to detect possible diseases occurring in the spine. The performance of deep learning models and optimization algorithms used in this process was evaluated. The study uses a data set created and/or analyzed during an existing study. This data set consists of images that belong to three different classes: scoliosis, spondylolisthesis, or x-ray images of normal (i.e. healthy) individuals. A total of 338 spine X-ray images, 188 scoliosis images, 79 spondylolisthesis images, and 71 normal images. Six different deep-learning architectures have been used in the study. These architectures are Alexnet, GoogLeNet, ResNet-18, ResNet-50, ResNet-101, and EfficientNet-bo. While working on these deep architectures, each model has been evaluated using different optimization algorithms. These optimization algorithms are RmsProp, SGDM, and Adam. According to the classification processes, the deep learning model with the highest accuracy value was Alexnet, and the optimization algorithm used with it, Sgdm (99.01%), and the training time lasted 38 seconds. According to the classification processes, the deep learning model with the fastest completion time (30 seconds) was Alexnet and the optimization algorithm used with it was RmsProp. An accuracy rate of 98.02% has been obtained in the training of this model.

This study proposes a compact deep learning (DL) architecture and a highly parallelized computing hardware platform to reconstruct the blood flow index (BFi) in diffuse correlation spectroscopy (DCS). We leveraged a rigorous analytical model to generate autocorrelation functions (ACFs) to train the DL network. We assessed the accuracy of the proposed DL using simulated and milk phantom data. Compared to convolutional neural networks (CNN), our lightweight DL architecture achieves 66.7% and 18.5% improvement in MSE for BFi and the coherence factor β, using synthetic data evaluation. The accuracy of rBFi over different algorithms was also investigated. We further simplified the DL computing primitives using subtraction for feature extraction, considering further hardware implementation. We extensively explored computing parallelism and fixed-point quantization within the DL architecture. With the DL model's compact size, we employed unrolling and pipelining optimizations for computation-intensive for-loops in the DL model while storing all learned parameters in on-chip BRAMs. We also achieved pixel-wise parallelism, enabling simultaneous, real-time processing of 10 and 15 autocorrelation functions on Zynq-7000 and Zynq-UltraScale+ field programmable gate array (FPGA), respectively. Unlike existing FPGA accelerators that produce BFi and the β from autocorrelation functions on standalone hardware, our approach is an encapsulated, end-to-end on-chip conversion process from intensity photon data to the temporal intensity ACF and subsequently reconstructing BFi and β. This hardware platform achieves an on-chip solution to replace post-processing and miniaturize modern DCS systems that use single-photon cameras. We also comprehensively compared the computational efficiency of our FPGA accelerator to CPU and GPU solutions.

Degenerative disorders such as Parkinson’s disease (PD) have an influence on daily activities due to rigidity of muscles, tremor or cognitive impairment. Micrographia, speech intensity, and deficient generation of voluntary saccadic eye movements (Pretegiani and Optican in Front Neurol 8:592, 2017) are manifestations of PD that can be used to devise noninvasive and low cost clinical tests. In this context, we have collected a multimodal dataset that we call Parkinson’s disease Multi-Modal Collection (PDMultiMC), which includes online handwriting, speech signals, and eye movements recordings. We present here the handwriting dataset that we call HandPDMultiMC that will be made publicly available. The HandPDMultiMC dataset includes handwriting samples from 42 subjects (21 PD and 21 controls). In this work we investigate the application of various Deep learning architectures, namely the CNN and the CNN-BLSTM, to PD detection through time series classification. Various approaches such as Spectrograms have been applied to encode pen-based signals into images for the CNN model, while the raw time series are directly used in the CNN-BLSTM. In order to train these models for PD detection on large scale data, various data augmentation approaches for pen-based signals are proposed. Experimental results on our dataset show that the best performance for early PD detection (97.62% accuracy) is reached by a combination of CNN-BLSTM models trained with Jittering and Synthetic data augmentation approaches. We also illustrate that deep architectures can surpass the models trained on pre-engineered features even though the available data is small.

The goal of multi-modal learning is to use complementary information on the relevant task provided by the multiple modalities to achieve reliable and robust performance. Recently, deep learning has led significant improvement in multi-modal learning by allowing for fusing high level features obtained at intermediate layers of the deep neural network. This paper addresses a problem of designing robust deep multi-modal learning architecture in the presence of the modalities degraded in quality. We introduce deep fusion architecture for object detection which processes each modality using the separate convolutional neural network (CNN) and constructs the joint feature maps by combining the intermediate features obtained by the CNNs. In order to facilitate the robustness to the degraded modalities, we employ the gated information fusion (GIF) network which weights the contribution from each modality according to the input feature maps to be fused. The combining weights are determined by applying the convolutional layers followed by the sigmoid function to the concatenated intermediate feature maps. The whole network including the CNN backbone and GIF is trained in an end-to-end fashion. Our experiments show that the proposed GIF network offers the additional architectural flexibility to achieve the robust performance in handling some degraded modalities.

In recent decades, deep learning (DL) has become a rapidly growing research direction, redefining the state-of-the-art performances in a wide range of techniques, such as object detection and speech recognition. In the aircraft design, dynamics, and control field, many works hinge on the information-rich data-driven approach, which includes the fusion-based prognostic and health management, the airliner's flight safety monitoring, intelligent sensing, and flight control systems development. While DL provides great potentials to solve these data-driven problems, a systematic review and discussion as to how the DL has been/can be used for these problems are still missing in relation to the rapidly developing and widely used DL techniques. In this article, we aim to address this urgent issue to provide a timely overview of the state-of-the-art for applying DL to the aircraft design, dynamics, and control field. In particular, we briefly introduce five representative DL methods, i.e., deep neural network, deep autoencoder, deep belief network, convolutional neural network, and recurrent neural network. Mathematical definitions for each method are presented, and illustrative applications are also discussed. We then review the existing DL-based works that have appeared in the aircraft design, dynamics, and control field. The review efforts are divided into two major groups, i.e., the own-ship aircraft modeling, wherein the works have been/can be implemented online for the aircraft design/dynamics/control, and other airplanes research works, wherein DL-based schemes provide offline monitoring of the aircraft operation. We then summarize the data sources and DL architectures. Referring to the experiences of DL research works/techniques development in other related fields, future opportunities, challenges, and potential solutions for implementing DL in the aircraft design, dynamics, and control field are also discussed.

<p>A Brain-Computer Interface (BCI) provides an alternative communication interface between the human brain and a computer. The Electroencephalogram (EEG) signals are acquired, processed and machine learning algorithms are further applied to extract useful information. During EEG acquisition, artifacts are induced due to involuntary eye movements or eye blink, casting adverse effects on system performance. The aim of this research is to predict eye states from EEG signals using Deep learning architectures and present improved classifier models. Recent studies reflect that Deep Neural Networks are trending state of the art Machine learning approaches. Therefore, the current work presents the implementation of Deep Belief Network (DBN) and Stacked AutoEncoders (SAE) as Classifiers with encouraging performance accuracy. One of the designed SAE models outperforms the performance of DBN and the models presented in existing research by an impressive error rate of 1.1% on the test set bearing accuracy of 98.9%. The findings in this study, may provide a contribution towards the state of the art performance on the problem of EEG based eye state classification.</p>

The aims of this work are (1) to explore deep learning (DL) architectures, spectroscopic input types, and learning designs toward optimal quantification in MR spectroscopy of simulated pathological spectra; and (2) to demonstrate accuracy and precision of DL predictions in view of inherent bias toward the training distribution. Simulated 1D spectra and 2D spectrograms that mimic an extensive range of pathological in vivo conditions are used to train and test 24 different DL architectures. Active learning through altered training and testing data distributions is probed to optimize quantification performance. Ensembles of networks are explored to improve DL robustness and reduce the variance of estimates. A set of scores compares performances of DL predictions and traditional model fitting (MF). Ensembles of heterogeneous networks that combine 1D frequency-domain and 2D time-frequency domain spectrograms as input perform best. Dataset augmentation with active learning can improve performance, but gains are limited. MF is more accurate, although DL appears to be more precise at low SNR. However, this overall improved precision originates from a strong bias for cases with high uncertainty toward the dataset the network has been trained with, tending toward its average value. MF mostly performs better compared to the faster DL approach. Potential intrinsic biases on training sets are dangerous in a clinical context that requires the algorithm to be unbiased to outliers (i.e., pathological data). Active learning and ensemble of networks are good strategies to improve prediction performances. However, data quality (sufficient SNR) has proven as a bottleneck for adequate unbiased performance-like in the case of MF.

Steering angle prediction plays a crucial role in the control of Autonomous Vehicles (AVs) and has garnered the interest of researchers, manufacturers, and insurance companies within the automotive industry. Various Deep Learning (DL) architectures have been utilized to forecast the steering angle of AVs across different scenarios. An examination focusing on steering angle prediction through deep learning algorithms can assist seasoned researchers in pinpointing areas that necessitate further advancement. Additionally, newcomers in the field can utilize this study as a foundation. This paper offers an extensive analysis of the recent progress achieved in DL architectures concerning the steering angle prediction of AVs. A comprehensive taxonomy outlining the application of DL in steering angle prediction of AVs has been developed. The survey provides a succinct synthesis, summary, and analysis of research findings. It is evident that Convolutional Neural Network (CNN) is the preferred choice among researchers for predicting the steering angle of autonomous vehicles, compared to other DL architectures. Furthermore, existing research challenges have been identified. The primary obstacle encountered in DL-based steering angle prediction of AVs is the scarcity of real-world datasets, leading researchers to heavily rely on data generated from simulated environments. Lastly, potential alternative approaches to address the identified research challenges have been suggested, indicating promising avenues for future research endeavours.

Deep learning based classification of breast tumors with shear-wave elastography

Deep learning is a subfield of machine learning and deep neural architectures can extract high level features automatically without handcraft feature engineering unlike traditional machine learning algorithms. In this paper, we propose a method, which combines feature extraction layers of a convolutional neural network with traditional machine learning algorithms, such as, support vector machine, gradient boosting machines, and random forest. All of the proposed hybrid models and the above mentioned machine learning algorithms are trained on three different datasets: MNIST, Fashion-MNIST, and CIFAR-10. Results show that the proposed hybrid models are more successful than traditional models while they are being trained from raw pixel values. In this study, we empower traditional machine learning algorithms for classification using feature extraction ability of deep neural network architectures and we are inspired by transfer learning methodology to this.