Canonical Correlation Analysis of Task Related Components as a noise-resistant method in Brain-Computer Interface Speller Systems based on Steady-State Visual Evoked Potential

A classification algorithm of an SSVEP brain-Computer interface based on CCA fusion wavelet coefficients

Objective. Steady-state visual evoked potential (SSVEP) is an important control method of the brain–computer interface (BCI) system. The development of an efficient SSVEP feature decoding algorithm is the core issue in SSVEP-BCI. It has been proposed to use user training data to reduce the spontaneous electroencephalogram activity interference on SSVEP response, thereby improving the feature recognition accuracy of the SSVEP signal. Nevertheless, the tedious data collection process increases the mental fatigue of the user and severely affects the applicability of the BCI system. Approach. A cross-subject spatial filter transfer (CSSFT) method that transfer the existing user model with good SSVEP response to the new user test data without collecting any training data from the new user is proposed. Main results. Experimental results demonstrate that the transfer model increases the distinction of the feature discriminant coefficient between the gaze following target and the non-gaze following target and accurately identifies the wrong target in the fundamental algorithm model. The public datasets show that the CSSFT method significantly increases the recognition performance of canonical correlation analysis (CCA) and filter bank CCA. Additionally, when the data used to calculate the transfer model contains one data block only, the CSSFT method retains its effective feature recognition capabilities. Significance. The proposed method requires no tedious data calibration process for new users, provides an effective technical solution for the transfer of the cross-subject model, and has potential application value for promoting the application of the BCI system.

Multi-frequency-modulated visual stimulation scheme has been shown effective for the steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) recently, especially in increasing the visual target number with less stimulus frequencies and mitigating the visual fatigue. However, the existing calibration-free recognition algorithms based on the traditional canonical correlation analysis (CCA) cannot provide the merited performance. To improve the recognition performance, this study proposes a phase difference constrained CCA (pdCCA), which assumes that the multi-frequency-modulated SSVEPs share a common spatial filter over different frequencies and have a specified phase difference. Specifically, during the CCA computation, the phase differences of the spatially filtered SSVEPs are constrained using the temporal concatenation of the sine-cosine reference signals with the pre-defined initial phases. We evaluate the performance of the proposed pdCCA-based method on three representative multi-frequency-modulated visual stimulation paradigms (i.e., based on the multi-frequency sequential coding, the dual-frequency, and the amplitude modulation). The evaluation results on four SSVEP datasets (Dataset Ia, Ib, II, and III) show that the pdCCA-based method can significantly outperform the current CCA method in terms of recognition accuracy. It improves the accuracy by 22.09% in Dataset Ia, 20.86% in Dataset Ib, 8.61% in Dataset II, and 25.85% in Dataset III. The pdCCA-based method, which actively controls the phase difference of the multi-frequency-modulated SSVEPs after spatial filtering, is a new calibration-free method for multi-frequency-modulated SSVEP-based BCIs.

Decision letter: Neural markers of predictive coding under perceptual uncertainty revealed with Hierarchical Frequency Tagging

Over the years, the driver-vehicle interface has been improved, but interacting with in-vehicle features can still increase distraction and affect road safety. This study aims to introduce brain-machine interface (BMI)- based solution to potentially enhance road safety. To achieve this goal, we evaluated visual stimuli properties (SPs) for a steady state visually evoked potentials (SSVEP)-based BMI system. We used a heads-up display (HUD) as the primary screen to present icons for controlling in-vehicle functions such as music, temperature, settings, and navigation. We investigated the effect of various SPs on SSVEP detection performance including the duty cycle and signal-to-noise ratio of visual stimuli, the size, color, and frequency of the icons, and array configuration and location. The experiments were conducted with 10 volunteers and the signals were analyzed using the canonical correlation analysis (CCA), filter bank CCA (FBCCA), and power spectral density analysis (PSDA). Our experimental results suggest that stimuli with a green color, a duty cycle of 50%, presented at a central location, with a size of 36 cm2 elicit a significantly stronger SSVEP response and enhanced SSVEP detection time. We also observed that lower SNR stimuli significantly affect SSVEP detection performance. There was no statistically significant difference observed in SSVEP response between the use of an LCD monitor and a HUD.

Application of a reconstruction technique in detection of dominant SSVEP frequency

There remains an active investigation on elevating the classification accuracy and information transfer rate of brain-computer interfaces based on steady-state visual evoked potential. However, it has often been ignored that the performance of steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) can be affected through the minor displacement of the electrodes from their optimal locations in practical applications because of the mislocation of electrodes and/or concurrent use of electroencephalography (EEG) devices with external devices, such as virtual reality headsets. In this study, we evaluated the performance robustness of SSVEP-based BCIs with respect to the changes in electrode locations for various channel configurations and classification algorithms. Our experiments involved 21 participants, where EEG signals were recorded from the scalp electrodes densely attached to the occipital area of the participants. The classification accuracies for all the possible cases of electrode location shifts for various channel configurations (1–3 channels) were calculated using five training-free SSVEP classification algorithms, i.e., the canonical correlation analysis (CCA), extended CCA, filter bank CCA, multivariate synchronization index (MSI), and extended MSI (EMSI). Then, the performances of the BCIs were evaluated using two measures, i.e., the average classification accuracy (ACA) across the electrode shifts and robustness to the electrode shift (RES). Our results showed that the ACA increased with an increase in the number of channels regardless of the algorithm. However, the RES was enhanced with an increase in the number of channels only when MSI and EMSI were employed. While both ACA and RES values for the five algorithms were similar under the single-channel condition, both ACA and RES values for MSI and EMSI were higher than those of the other algorithms under the multichannel (i.e., two or three electrodes) conditions. In addition, EMSI outperformed MSI when comparing the ACA and RES values under the multichannel conditions. In conclusion, our results suggested that the use of multichannel configuration and employment of EMSI could make the performance of SSVEP-based BCIs more robust to the electrode shift from the optimal locations.

Spectrum-Enhanced TRCA (SE-TRCA): A novel approach for direction detection in SSVEP-based BCI

Brain-computer interface (BCI) systems translate the human neurophysiological activities into commands through EEG analysis. Improving the BCI performances leads to faster and easier use and less fatigue. In this study, we proposed a new prepossessing approach to increase the robustness of a steady-state visual evoked potential (SSVEP) based BCI. Inspiring from the known properties of the SSVEP frequency components, the goal was to enhance the signal quality by making it more convenient to be interpreted by the decision-making step. We first investigated the potential to detect the deteriorating periods based on the physiological properties of the SSVEP. The proposed system localizes the intervals which can obscure the SSVEP frequencies by a new algorithm founded on the processing and the analysis of the instantaneous phase. The piecewise linear regression allows a sampler comprehension of the phase signal. Then, these intervals are filtered by the moving average filter to enhance the SSVEP quality. Finally, the decision making is made by the canonical correlation analysis (CCA) algorithm. The results of experiments, using real EEG signals from five subjects, show that the proposed approach significantly increases the performances in terms of accuracy and information transfer rate by about 7.3% and 3.85 bits/min, respectively, in case of 2 s segment length. On the other hand, the spatial filtering methods of the literature weaken the system performances.

Steady state visual evoked potential (SSVEP) is widely used in brain computer interface systems because of its steady signal, low number of channels and high recognition rate. In this paper, SSVEP was elicited by sinusoidal stimulation, and Filter Bank canonical correlation analysis (FBCCA) was used to solve the problem that the amplitude and energy of the nonlinear and non-stationary signals at each frequency band varied with time. The experimental results show that the correct recognition rate is 97.5%. And the mean ITR are 152bits/min. The method can effectively improve the identification and control of SSVEP EEG signals. At the same time, the stimulus paradigm displayed on the computer screen reduced the visual fatigue of the subjects and improved the comfort level of the subjects.

Objective. Code-modulated visual evoked potential (c-VEP) based brain–computer interfaces (BCIs) exhibit high encoding efficiency. Nevertheless, the majority of c-VEP based BCIs necessitate an initial training or calibration session, particularly when the number of targets expands, which impedes the practicality. To address this predicament, this study introduces a calibration-free c-VEP based BCI employing narrow-band random sequences. Approach. For the encoding method, a series of random sequences were generated within a specific frequency band. The c-VEP signals were subsequently elicited through the application of on-type grid flashes that were modulated by these sequences. For the calibration-free decoding algorithm, filter-bank canonical correlation analysis (FBCCA) was utilized with the reference templates generated from the original sequences. Thirty-five subjects participated into an online BCI experiment. The performances of c-VEP based BCIs utilizing narrow-band random sequences with frequency bands of 15–25 Hz (NBRS-15) and 8–16 Hz (NBRS-8) were compared with that of a steady-state visual evoked potential (SSVEP) based BCI within a frequency range of 8–15.8 Hz. Main results. The offline analysis results demonstrated a substantial correlation between the c-VEPs and the original narrow-band random sequences. After parameter optimization, the calibration-free system employing the NBRS-15 frequency band achieved an average information transfer rate (ITR) of 78.56 ± 37.03 bits/min, which exhibited no significant difference compared to the performance of the SSVEP based system when utilizing FBCCA. The proposed system achieved an average ITR of 102.1 ± 57.59 bits/min in a simulation of a 1000-target BCI system. Significance. This study introduces a novel calibration-free c-VEP based BCI system employing narrow-band random sequences and shows great potential of the proposed system in achieving a large number of targets and high ITR.

Steady-state visual evoked potential brain-computer interfaces (SSVEP-BCI) have attracted significant attention due to their ease of deployment and high performance in terms of information transfer rate (ITR) and accuracy, making them a promising candidate for integration with consumer electronics devices. However, as SSVEP characteristics are directly associated with visual stimulus attributes, the influence of stereoscopic vision on SSVEP as a critical visual attribute has yet to be fully explored. Meanwhile, the promising combination of virtual reality (VR) devices and BCI applications is hampered by the significant disparity between VR environments and traditional 2D displays. This is not only due to the fact that screen-based SSVEP generally operates under static, stable conditions with simple and unvaried visual stimuli but also because conventional luminance-modulated stimuli can quickly induce visual fatigue. This study attempts to address these research gaps by designing SSVEP paradigms with stereo-related attributes and conducting a comparative analysis with the traditional 2D planar paradigm under the same VR environment. This study proposed two new paradigms: the 3D paradigm and the 3D-Blink paradigm. The 3D paradigm induces SSVEP by modulating the luminance of spherical targets, while the 3D-Blink paradigm employs modulation of the spheres' opacity instead. The results of offline 4-object selection experiments showed that the accuracy of 3D and 2D paradigm was 85.67 and 86.17% with canonical correlation analysis (CCA) and 86.17 and 91.73% with filter bank canonical correlation analysis (FBCCA), which is consistent with the reduction in the signal-to-noise ratio (SNR) of SSVEP harmonics for the 3D paradigm observed in the frequency-domain analysis. The 3D-Blink paradigm achieved 75.00% of detection accuracy and 27.02 bits/min of ITR with 0.8 seconds of stimulus time and task-related component analysis (TRCA) algorithm, demonstrating its effectiveness. These findings demonstrate that the 3D and 3D-Blink paradigms supported by VR can achieve improved user comfort and satisfactory performance, while further algorithmic optimization and feature analysis are required for the stereo-related paradigms. In conclusion, this study contributes to a deeper understanding of the impact of binocular stereoscopic vision mechanisms on SSVEP paradigms and promotes the application of SSVEP-BCI in diverse VR environments.

A steady‐state visually evoked potential (SSVEP) is a brain response to specific frequencies of visual stimuli, including their harmonic frequencies. However, this signal is susceptible to interference from spontaneous and rhythms in electroencephalography (EEG) signals because they overlap from 8 to 40 Hz. This can reduce the recognition accuracy of SSVEP brain–computer interfaces (BCIs). To address this problem, a variational mode decomposition–based filter bank canonical correlation analysis (VMD‐FBCCA) algorithm is proposed, which integrates the adaptive characteristics of VMD and the training‐free nature of the FBCCA algorithm. First, the EEG signal of each channel is transformed into intrinsic mode functions (IMFs) by the VMD algorithm, which extracts frequency components of the SSVEP from each IMF. Next, a particle swarm algorithm is employed to optimize the weights of the IMFs and reconstruct the EEG signals. This reconstruction selectively enhances the IMFs in the target SSVEP frequency band while suppressing interference from other bands. Finally, the reconstructed EEG is classified using FBCCA to decode the SSVEP‐BCI signal. To evaluate its effectiveness, the proposed algorithm is tested on datasets from the BCI Competition. The results demonstrate that VMD‐FBCCA outperforms FBCCA, showing improvements in both the average recognition accuracy and information transmission rate (8.91 bits/min). Moreover, the best recognition accuracy achieved for individual subjects is enhanced by .

An electroencephalogram (EEG) signal projection using kernel discriminative locality preserving canonical correlation analysis (KDLPCCA)-based correlation with steady-state visual evoked potential (SSVEP) templates for frequency recognition is presented in this paper. With KDLPCCA, not only a non-linear correlation but also local properties and discriminative information of each class sample are considered to extract temporal and frequency features of SSVEP signals. The new projected EEG features are classified with classical machine learning algorithms, namely, K-nearest neighbors (KNNs), naive Bayes, and random forest classifiers. To demonstrate the effectiveness of the proposed method, 16-channel SSVEP data corresponding to 4 frequencies collected from 5 subjects were used to evaluate the performance. Compared with the state of the art canonical correlation analysis (CCA), experimental results show significant improvements in classification accuracy and information transfer rate (ITR), achieving 100% and 240 bits/min with 0.5 s sample block. The superior performance demonstrates that this method holds the promising potential to achieve satisfactory performance for high-accuracy SSVEP-based brain-computer interfaces.

Recently, brain-computer interface (BCI) systems developed based on steady-state visual evoked potential (SSVEP) have attracted much attention due to their high information transfer rate (ITR) and increasing number of targets. However, SSVEP-based methods can be improved in terms of their accuracy and target detection time. We propose a new method based on canonical correlation analysis (CCA) to integrate subject-specific models and subject-independent information and enhance BCI performance. We propose to use training data of other subjects to optimize hyperparameters for CCA-based model of a specific subject. An ensemble version of the proposed method is also developed for a fair comparison with ensemble task-related component analysis (TRCA). The proposed method is compared with TRCA and extended CCA methods. A publicly available, 35-subject SSVEP benchmark dataset is used for comparison studies and performance is quantified by classification accuracy and ITR. The ITR of the proposed method is higher than those of TRCA and extended CCA. The proposed method outperforms extended CCA in all conditions and TRCA for time windows greater than 0.3 s. The proposed method also outperforms TRCA when there are limited training blocks and electrodes. This study illustrates that adding subject-independent information to subject-specific models can improve performance of SSVEP-based BCIs.