A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces

BackgroundA novel artefact removal algorithm is proposed for a self-paced hybrid brain-computer interface (BCI) system. This hybrid system combines a self-paced BCI with an eye-tracker to operate a virtual keyboard. To select a letter, the user must gaze at the target for at least a specific period of time (dwell time) and then activate the BCI by performing a mental task. Unfortunately, electroencephalogram (EEG) signals are often contaminated with artefacts. Artefacts change the quality of EEG signals and subsequently degrade the BCI’s performance.MethodsTo remove artefacts in EEG signals, the proposed algorithm uses the stationary wavelet transform combined with a new adaptive thresholding mechanism. To evaluate the performance of the proposed algorithm and other artefact handling/removal methods, semi-simulated EEG signals (i.e., real EEG signals mixed with simulated artefacts) and real EEG signals obtained from seven participants are used. For real EEG signals, the hybrid BCI system’s performance is evaluated in an online-like manner, i.e., using the continuous data from the last session as in a real-time environment.ResultsWith semi-simulated EEG signals, we show that the proposed algorithm achieves lower signal distortion in both time and frequency domains. With real EEG signals, we demonstrate that for dwell time of 0.0s, the number of false-positives/minute is 2 and the true positive rate (TPR) achieved by the proposed algorithm is 44.7%, which is more than 15.0% higher compared to other state-of-the-art artefact handling methods. As dwell time increases to 1.0s, the TPR increases to 73.1%.ConclusionsThe proposed artefact removal algorithm greatly improves the BCI’s performance. It also has the following advantages: a) it does not require additional electrooculogram/electromyogram channels, long data segments or a large number of EEG channels, b) it allows real-time processing, and c) it reduces signal distortion.

One of the main problems in Brain Computer Interface (BCI) systems is the non-stationary behavior of the electroencephalography (EEG) signals causing problems in real time applications. Another common problem in BCI systems is the situation where the labeled data are scarce. In this study, we take a semi-supervised learning perspective and propose solving both types of problems by updating the BCI system with labels obtained from the outputs of the classifier. To test the approach, data from motor imagery BCI system are used. Attributes extracted from EEG signals are classified with Linear Discriminant Analysis (LDA) and Support Vector Machines (SVM). With respect to the static classifiers, accuracy was improved approximately 4% using the proposed adaptation approach in the case of a training dataset. Even though the difference between the performance of static and adaptive classifiers decreases as the size of training data increases, the accuracy of our proposed adaptive classifier remains higher. The proposed approach has also improved the performance of a BCI system around 4% in the case of non-stationary signals as well.

Nowadays, brain-computer interface (BCI) systems are frequently used to connect individuals who have lost their mobility with the outside world. These BCI systems enable individuals to control external devices using brain signals. However, these systems have certain disadvantages for users. This paper proposes a novel approach to minimize the disadvantages of visual stimuli on the eye health of system users in BCI systems employing visual evoked potential (VEP) and P300 methods. The approach employs moving objects with different trajectories instead of visual stimuli. It uses a light-emitting diode (LED) with a frequency of 7 Hz as a condition for the BCI system to be active. The LED is assigned to the system to prevent it from being triggered by any involuntary or independent eye movements of the user. Thus, the system user will be able to use a safe BCI system with a single visual stimulus that blinks on the side without needing to focus on any visual stimulus through moving balls. Data were recorded in two phases: when the LED was on and when the LED was off. The recorded data were processed using a Butterworth filter and the power spectral density (PSD) method. In the first classification phase, which was performed for the system to detect the LED in the background, the highest accuracy rate of 99.57% was achieved with the random forest (RF) classification algorithm. In the second classification phase, which involves classifying moving objects within the proposed approach, the highest accuracy rate of 97.89% and an information transfer rate (ITR) value of 36.75 (bits/min) were achieved using the RF classifier.

Non-invasive brain-computer interfaces (BCIs) have been widely used for neural decoding, linking neural signals to control devices. Hybrid BCI systems using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) have received significant attention for overcoming the limitations of EEG- and fNIRS-standalone BCI systems. However, most hybrid EEG-fNIRS BCI studies have focused on late fusion because of discrepancies in their temporal resolutions and recording locations. Despite the enhanced performance of hybrid BCIs, late fusion methods have difficulty in extracting correlated features in both EEG and fNIRS signals. Therefore, in this study, we proposed a deep learning-based early fusion structure, which combines two signals before the fully-connected layer, called the fNIRS-guided attention network (FGANet). First, 1D EEG and fNIRS signals were converted into 3D EEG and fNIRS tensors to spatially align EEG and fNIRS signals at the same time point. The proposed fNIRS-guided attention layer extracted a joint representation of EEG and fNIRS tensors based on neurovascular coupling, in which the spatially important regions were identified from fNIRS signals, and detailed neural patterns were extracted from EEG signals. Finally, the final prediction was obtained by weighting the sum of the prediction scores of the EEG and fNIRS-guided attention features to alleviate performance degradation owing to delayed fNIRS response. In the experimental results, the FGANet significantly outperformed the EEG-standalone network. Furthermore, the FGANet has 4.0% and 2.7% higher accuracy than the state-of-the-art algorithms in mental arithmetic and motor imagery tasks, respectively.

Brain-computer interface (BCI) systems, which provide users with an additional channel to communicate with external devices, have been mainly developed using electroencephalography (EEG) and near-infrared spectroscopy (NIRS). To complement each modality's pros and cons, various hybrid NIRS-EEG studies have been investigated. However, most studies focused on enhancing the classification accuracy rather than analyzing the characteristics of used features. This study aimed to investigate whether EEG features from spatial, temporal, and spectral domains would exhibit the diverse efficacy in hybrid NIRS-EEG BCI. Open access NIRS and EEG recordings of left/ right hand gripping imagery from twenty-nine healthy subjects were utilized. Common spatial patterns (CSP), time domain parameters (TDP), and power spectral density (PSD) were separately employed with NIRS to evaluate the discrimination performance. Within dataset, NIRS with CSP showed the highest classification accuracy with linear support vector machine (LSVM) classifier (mean accuracy, 71.4%). For kernel SVM (KSVM) classifiers, mean accuracy of NIRS with TDP features was lower than accuracy of only NIRS features (mean accuracy, NIRS: 53.1% and NIRS with TDP: 50.5%). The findings suggested that binary motor imagery tasks, which involve localized brain activation, could be enhanced by applying features including rich spatial information.

Motor imagery (MI) is an important brain-computer interface (BCI) paradigm. The traditional MI paradigm (imagining different limbs) limits the intuitive control of the outer devices, while fine MI paradigm (imagining different joint movements from the same limb) can control the mechanical arm without cognitive disconnection. However, the decoding performance of fine MI limits its application. Electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) are widely used in BCI systems because of their portability and easy operation. In this study, a fine MI paradigm including four classes (hand, wrist, shoulder and rest) was designed, and the data of EEG-fNIRS bimodal brain activity was collected from 12 subjects. Event-related desynchronization (ERD) from EEG signals shows a contralateral dominant phenomenon, and there is difference between the ERD of the four classes. For fNIRS signal in the time dimension, the time periods with significant difference can be observed in the activation patterns of four MI tasks. Spatially, the signal peak based brain topographic map also shows difference of these four MI tasks. The EEG signal and fNIRS signal of these four classes are distinguishable. In this study, a bimodal fusion network is proposed to improve the fine MI tasks decoding performance. The features of these two modalities are extracted separately by two feature extractors based on convolutional neural networks (CNN). The recognition performance was significantly improved by the bimodal method proposed in this study, compared with the performance of the single-modal network. The proposed method outperformed all comparison methods, and achieved a four-class accuracy of 58.96%. This paper demonstrates the feasibility of EEG and fNIRS bimodal BCI systems for fine MI, and shows the effectiveness of the proposed bimodal fusion method. This research is supposed to support fine MI-based BCI systems with theories and techniques.

Current EEG based brain computer interface (BCI) systems have achieved successful control in up to 3 dimensions; however, the current sensor-based paradigm is not well suited for many rehabilitative and recreational applications that require motor imagination (MI) tasks of fine motor movements to be recognized. Therefore there is a great need to find complex MI tasks that are intuitive for BCI users to perform and that can be classified with high accuracy. In this paper we present our results on classifying four MI tasks of the right hand, flexion, extension, supination and pronation using a novel EEG source imaging approach. Using this approach we were able to improve the four-class classification of the four tasks by nearly 10% as compared to traditional sensor-based techniques.

A novel hybrid BCI speller based on RSVP and SSVEP paradigm

One of the key components of brain–computer interface (BCI) design is the interfacing paradigm. Progress in BCI research and applications significantly depends on a successful paradigm implementation. BCI paradigm requires simple interpretation and ease of use for end users so that electroencephalogram (EEG) data can be mapped to an application, for example, to drive a prosthetic arm or to send a command to operate an external device. Another remarkable use of the BCI system is to set up a communication network with exterior environment. Such actions necessitate the user to disburse voluntary or involuntary mental or control task without making the subject fatigued or tired. To ensure these criteria, the paradigm design should be leveraged to deliver optimal performance. P300 and SSVEP are two paradigms that have gathered large attentions from BCI communities. Over the last decade, advancement of BCI research has experienced successful design of various hybrid BCI systems. This chapter focuses on the issues and challenges involved during the development of BCI paradigm to harness the advantages associated with recently developed BCI paradigms. In addition, importance of paradigms will be highlighted to open up novel BCI applications for future use, which, in turn, might have a great potential to increase the BCI users to a large extent.

Background Due to the redundant information contained in multichannel electroencephalogram (EEG) signals, the classification accuracy of brain-computer interface (BCI) systems may deteriorate to a large extent. Channel selection methods can help to remove task-independent electroencephalogram (EEG) signals and hence improve the performance of BCI systems. However, in different frequency bands, brain areas associated with motor imagery are not exactly the same, which will result in the inability of traditional channel selection methods to extract effective EEG features. New Method To address the above problem, this paper proposes a novel method based on common spatial pattern- (CSP-) rank channel selection for multifrequency band EEG (CSP-R-MF). It combines the multiband signal decomposition filtering and the CSP-rank channel selection methods to select significant channels, and then linear discriminant analysis (LDA) was used to calculate the classification accuracy. Results The results showed that our proposed CSP-R-MF method could significantly improve the average classification accuracy compared with the CSP-rank channel selection method.

An amplitude-modulated visual stimulation for reducing eye fatigue in SSVEP-based brain–computer interfaces

34 - Electroencephalogram-based brain–computer interface systems for controlling rehabilitative devices

Predictive models have succeeded in distinguishing between individuals with Alcohol use Disorder (AUD) and controls. However, predictive models identifying who is prone to develop AUD and the biomarkers indicating a predisposition to AUD are still unclear. Our sample (n = 656) included offspring and non-offspring of European American (EA) and African American (AA) ancestry from the Collaborative Study of the Genetics of Alcoholism (COGA) who were recruited as early as age 12 and were unaffected at first assessment and reassessed years later as AUD (DSM-5) (n = 328) or unaffected (n = 328). Machine learning analysis was performed for 220 EEG measures, 149 alcohol-related single nucleotide polymorphisms (SNPs) from a recent large Genome-wide Association Study (GWAS) of alcohol use/misuse and two family history (mother DSM-5 AUD and father DSM-5 AUD) features using supervised, Linear Support Vector Machine (SVM) classifier to test which features assessed before developing AUD predict those who go on to develop AUD. Age, gender, and ancestry stratified analyses were performed. Results indicate significant and higher accuracy rates for the AA compared with the EA prediction models and a higher model accuracy trend among females compared with males for both ancestries. Combined EEG and SNP features model outperformed models based on only EEG features or only SNP features for both EA and AA samples. This multidimensional superiority was confirmed in a follow-up analysis in the AA age groups (12-15, 16-19, 20-30) and EA age group (16-19). In both ancestry samples, the youngest age group achieved higher accuracy score than the two other older age groups. Maternal AUD increased the model's accuracy in both ancestries' samples. Several discriminative EEG measures and SNPs features were identified, including lower posterior gamma, higher slow wave connectivity (delta, theta, alpha), higher frontal gamma ratio, higher beta correlation in the parietal area, and 5 SNPs: rs4780836, rs2605140, rs11690265, rs692854, and rs13380649. Results highlight the significance of sampling uniformity followed by stratified (e.g., ancestry, gender, developmental period) analysis, and wider selection of features, to generate better prediction scores allowing a more accurate estimation of AUD development.

White matter hyperintensities (WMH) are a common cause of subcortical vascular cognitive impairment (SVCI). The silent yet progressive nature of WMH in cognitive decline underscores the need for reliable biomarkers for early detection and monitoring of its progression. This study aims to investigate the association between functional near-infrared spectroscopy (fNIRS) signals during mental and physical activities and WMH volume. Additionally, it explores the relationship between fNIRS signals and WMH progression. We recruited 27 patients with mild cognitive impairment (MCI) presenting WMH clinical characteristics. Data from fNIRS and MRI scans were collected during their first visit. Ten of them underwent fNIRS and MRI scans in a second visit two years later. WMH volume analysis used volBrain lesionBrain 1.0 (https://www.volbrain.net). ROC curve analysis was applied to the normalized WMH volume to determine a cut-off value for distinguishing between the subcortical vascular MCI (svMCI) and amnestic MCI (aMCI) groups. We compared fNIRS data during cognitive tests and physical activities between svMCI and aMCI groups at the first visit and in the two-year follow-up. While cognitive profiles were similar between groups, svMCI patients showed significantly reduced fNIRS signals, particularly in the left orbitofrontal cortex (OFC) during verbal fluency tasks (P = 0.005), with further reductions in the left dorsolateral prefrontal cortex (P = 0.049), left OFC (P = 0.012), and right OFC (P = 0.02) over two years. Baseline WMH volume correlated negatively with fNIRS signals during the Stroop test (r = -0.837, P = 0.005). Changes in WMH volume over two years correlated positively with changes in fNIRS signals in the right ventrolateral prefrontal cortex during memory tasks (r = 0.886, P = 0.033) and left OFC during balance tasks (r = 0.786, P = 0.028). Our results suggest that fNIRS signals have the potential to serve as biomarkers for WMH progression.

Many studies have been carried out in order to detect and quantify the level of mental stress by means of different physiological signals. From the physiological point of view, stress promptly affects brain and cardiac function; therefore, stress can be assessed by analyzing the brain- and heart-related signals more efficiently. Signals produced by functional near-infrared spectroscopy (fNIRS) of the brain together with the heart rate (HR) are employed to assess the stress induced by the Montreal Imaging Stress Task. Two different versions of the HR are used in this study. The first one is the commonly used HR derived from the electrocardiogram (ECG) and is considered as the reference HR (RHR). The other is the HR computed from the fNIRS signal (EHR) by means of an effective combinational algorithm. fNIRS and ECG signals were simultaneously recorded from 10 volunteers, and EHR and RHR are derived from them, respectively. Our results showed a high degree of agreement [r > 0.9, BAR (Bland Altman ratio) <5 % ] between the two HR. A principal component analysis/support vector machine-based algorithm for stress classification is developed and applied to the three measurements of fNIRS, EHR, and RHR and a classification accuracy of 78.8%, 94.6%, and 62.2% were obtained for the three measurements, respectively. From these observations, it can be concluded that the EHR carries more useful information with regards to the mental stress than the RHR and fNIRS signals. Therefore, EHR can be used alone or in combination with the fNIRS signal for a more accurate and real-time stress detection and classification.