EEG-based Forward Movement and Turning MI Classification with and without Action Observation in Virtual Reality

Cortical activation and BCI performance during brief tactile imagery: A comparative study with motor imagery

Graph vertex and spectral features for EEG-based motor imagery classification.

Purpose In general, motor imagery and action observation have been distinguished from each other. Recently, several studies demonstrated that combined approach to motor imagery and action observation can be more effective in motor learning. The present study examined the effects of observation learning combined motor imagery and action observation during acquisition basketball shooting skills. Methods We divided with control group, action observation group and observation learning group combined mental image and action observation in the three middle school. Action observation group provided the action observation program, and observation learning group was performed observation learning combined mental image and action observation training. All groups were perform basic basketball skills. Experimental intervention was performed for 10 weeks, and data analysis was performed 3 groups × 2 time repeated ANOVA. Results The results indicated that all group were improve after intervention, and subjects who participated in combined mental image and action observation was significant in the interaction effect on the front shoot. Moreover, the interaction effect on the motor imagery ability was significant. Conclusions These findings suggest that the use of observation learning combined mental image and action observation strategy potentially optimizes motor skills performance and motor image ability by incorporating motor imagery, especially when observing movements with intent to imitate.

The brain-computer interface (BCI) work is to provide humans an alternative channel that allows direct transmission of messages from the brain by analyzing the brain’s mental activities [1–7]. The brain activity is recorded by means of multi-electrode electroencephalographic (EEG) signals that are either invasive or noninvasive. Noninvasive recording is convenient and popular in BCI applications so it is commonly used. According to the definition suggested at the first international meeting for BCI technology, the term BCI is reserved for a system that must not depend on the brain’s normal output pathways of peripheral nerves and muscles [2]. It has become popular for BCI systems on motor imagery (MI) EEG signals in the last decade [8]. It reveals that there are special characteristics of event-related desynchronization (ERD) and synchronization (ERS) in mu and beta rhythms over the sensorimotor cortex during MI tasks by discriminating EEG signals between left and right MIs [9, 10]. ERD/ERS is the task-related or event-related change in the amplitude of the oscillatory behavior of specific cortical areas within various frequency bands. An amplitude (or power) increase is defined as event-related synchronization while an amplitude (or power) decrease is defined as event-related desynchronization. As other event-related potentials, ERD/ERS patterns are associated with sensory processing and motor behavior [2]. The principal objective of this study is to propose a BCI system, which combines neuro-fuzzy prediction and multiresolution fractal feature vectors (MFFVs) with support vector machine, for MI classification.

Brain-Computer Interface (BCI) is an emerging technology that facilitates a pathway between the human brain and external devices. Electroencephalography (EEG) data are mainly employed in BCI systems to reflect the underlying mechanism of different neural activities associated with various limb motions or Motor Imagery (MI) activities. Multichannel EEG signal processing generally results in high-dimensional features, which increases BCI’s overall computational and temporal complexity. We introduce a channel selection methodology using the mutual information-based three-way interaction scheme to reduce this burden due to many channels. Our approach initializes a set of three candidate solutions for a given MI classification task and subsequently determines a highly significant EEG channel set. It effectively balances relevance and redundancy levels in the final channel subset during the selection and rejection of a newly selected channel. The proposed scheme is evaluated on the BCI Competition IV-2008 dataset with four MI classes (left hand, right hand, tongue, and feet) and twenty-two channels. The performance of our scheme is compared with three recently published state-of-the-art methods. The proposed approach realized an average of 86.66% classification accuracy using only nine channels on the data of nine participants. The comparative study shows that our approach realized better performance in terms of higher classification accuracy and channel reduction rate than all three baseline models. The results are promising for the online BCI paradigm that requires low complexity while conducting multiple sessions of BCI experiments for a larger group of participants.

BackgroundMotor planning, imagery or execution is associated with event-related desynchronization (ERD) of mu rhythm oscillations (8-13 Hz) recordable over sensorimotor areas using electroencephalography (EEG). It was shown that motor imagery involving distal muscles, e.g. finger movements, results in contralateral ERD correlating with increased excitability of the contralateral corticospinal tract (c-CST). Following the rationale that purposefully increasing c-CST excitability might facilitate motor recovery after stroke, ERD recently became an attractive target for brain-computer interface (BCI)-based neurorehabilitation training. It was unclear, however, whether ERD would also reflect excitability of the ipsilateral corticospinal tract (i-CST) that mainly innervates proximal muscles involved in e.g. shoulder movements. Such knowledge would be important to optimize and extend ERD-based BCI neurorehabilitation protocols, e.g. to restore shoulder movements after stroke. Here we used single-pulse transcranial magnetic stimulation (TMS) targeting the ipsilateral primary motor cortex to elicit motor evoked potentials (MEPs) of the trapezius muscle. To assess whether ERD reflects excitability of the i-CST, a correlation analysis between between MEP amplitudes and ipsilateral ERD was performed.MethodsExperiment 1 consisted of a motor execution task during which 10 healthy volunteers performed elevations of the shoulder girdle or finger pinching while a 128-channel EEG was recorded. Experiment 2 consisted of a motor imagery task during which 16 healthy volunteers imagined shoulder girdle elevations or finger pinching while an EEG was recorded; the participants simultaneously received randomly timed, single-pulse TMS to the ipsilateral primary motor cortex. The spatial pattern and amplitude of ERD and the amplitude of the agonist muscle’s TMS-induced MEPs were analyzed.ResultsERDs occurred bilaterally during both execution and imagery of shoulder girdle elevations, but were lateralized to the contralateral hemisphere during finger pinching. We found that trapezius MEPs increased during motor imagery of shoulder elevations and correlated with ipsilateral ERD amplitudes.ConclusionsIpsilateral ERD during execution and imagery of shoulder girdle elevations appears to reflect the excitability of uncrossed pathways projecting to the shoulder muscles. As such, ipsilateral ERD could be used for neurofeedback training of shoulder movement, aiming at reanimation of the i-CST.

Objective. Deep transfer learning has been widely used to address the nonstationarity of electroencephalogram (EEG) data during motor imagery (MI) classification. However, previous deep learning approaches suffer from limited classification accuracy because the temporal and spatial features cannot be effectively extracted. Approach. Here, we propose a novel end-to-end deep subject adaptation convolutional neural network (SACNN) to handle the problem of EEG-based MI classification. Our proposed model jointly optimizes three modules, i.e. a feature extractor, a classifier, and a subject adapter. Specifically, the feature extractor simultaneously extracts the temporal and spatial features from the raw EEG data using a parallel multiscale convolution network. In addition, we design a subject adapter to reduce the feature distribution shift between the source and target subjects by using the maximum mean discrepancy. By minimizing the classification loss and the distribution discrepancy, the model is able to extract the temporal-spatial features to the prediction of a new subject. Main results. Extensive experiments are carried out on three EEG-based MI datasets, i.e. brain–computer interface (BCI) competition IV dataset IIb, BCI competition III dataset IVa, and BCI competition IV dataset I, and the average accuracy reaches to 86.42%, 81.71% and 79.35% on the three datasets respectively. Furthermore, the statistical analysis also indicates the significant performance improvement of SACNN. Significance. This paper reveals the importance of the temporal-spatial features on EEG-based MI classification task. Our proposed SACNN model can make fully use of the temporal-spatial information to achieve the purpose.

Brain computer interfaces (BCIs) based on motor imagery (MI) play an important role in helping to improve and restore the loss of physical function. However, traditional MI-BCIs are limited to the motion intention of gross limb, which places many restrictions on their applications. This study proposes a hybrid paradigm based on MI and the steady-state somatosensory evoked potential, with the aim of improving the spatial resolution of MI recognition. Twelve subjects participated in this study. They performed MI tasks under MI and hybrid conditions. In the MI condition, subjects only performed MI tasks, whereas, in the hybrid condition, they received an electrical stimulus while performing the same tasks. Under the hybrid condition, subjects were required not to pay extra attention to the electrical stimulation. The MI task included imagining clenching the right hand and lifting the right forearm. The classifier was built using the filter bank common spatial pattern algorithm and a support vector machine, and online experiments were used to verify the recognition of two MI tasks. During the online experiments, all subjects were able to output different commands at a recognition accuracy far higher than the random level. The average classification accuracy of the hybrid condition reached 76.39%, with a maximum value of 88.34%, which was about 11% higher than that of the MI condition. Moreover, based on offline data, the classification performance using the event-related desynchronization (ERD) feature under the hybrid condition did not differ significantly from that under the MI condition, indicating that the introduction of electrical stimulation did not interfere in the separability of ERD. The proposed paradigm improved the efficiency of decoding multiple MI locations within a single limb. Despite the introduction of external stimuli, users could still drive the new system in the same way as MI in traditional MI-BCI.

BackgroundThe aging of the world population poses a major health challenge, and brain–computer interface (BCI) technology has the potential to provide assistance and rehabilitation for the elderly.ObjectivesThis study aimed to investigate the electroencephalogram (EEG) characteristics during motor imagery by comparing young and elderly, and study Convolutional Neural Networks (CNNs) classification for the elderly population in terms of fatigue analysis in both frontal and parietal regions.MethodsA total of 20 healthy individuals participated in the study, including 10 young and 10 older adults. All participants completed the left- and right-hand motor imagery experiment. The energy changes in the motor imagery process were analyzed using time–frequency graphs and quantified event-related desynchronization (ERD) values. The fatigue level of the motor imagery was assessed by two indicators: (θ + α)/β and θ/β, and fatigue-sensitive channels were distinguished from the parietal region of the brain. Then, rhythm entropy was introduced to analyze the complexity of the cognitive activity. The phase-lock values related to the parietal and frontal lobes were calculated, and their temporal synchronization was discussed. Finally, the motor imagery EEG data was classified by CNNs, and the accuracy was discussed based on the analysis results.ResultFor the young and elderly, ERD was observed in C3 and C4 channels, and their fatigue-sensitive channels in the parietal region were slightly different. During the experiment, the rhythm entropy of the frontal lobe showed a decreasing trend with time for most of the young subjects, while there was an increasing trend for most of the older ones. Using the CNN classification method, the elderly achieved around 70% of the average classification accuracy, which is almost the same for the young adults.ConclusionCompared with the young adults, the elderly are less affected by the level of cognitive fatigue during motor imagery, but the classification accuracy of motor imagery data in the elderly may be slightly lower than that in young persons. At the same time, the deep learning method also provides a potentially feasible option for the application of motor-imagery BCI (MI-BCI) in the elderly by considering the ERD and fatigue phenomenon together.

Objective. We proposed a novel simultaneous hybrid brain–computer interface (BCI) by incorporating electrical stimulation into a motor imagery (MI) based BCI system. The goal of this study was to enhance the overall performance of an MI-based BCI. In addition, the brain oscillatory pattern in the hybrid task was also investigated. Approach. 64-channel electroencephalographic (EEG) data were recorded during MI, selective attention (SA) and hybrid tasks in fourteen healthy subjects. In the hybrid task, subjects performed MI with electrical stimulation which was applied to bilateral median nerve on wrists simultaneously. Main results. The hybrid task clearly presented additional steady-state somatosensory evoked potential (SSSEP) induced by electrical stimulation with MI-induced event-related desynchronization (ERD). By combining ERD and SSSEP features, the performance in the hybrid task was significantly better than in both MI and SA tasks, achieving a ~14% improvement in total relative to the MI task alone and reaching ~89% in mean classification accuracy. On the contrary, there was no significant enhancement obtained in performance while separate ERD feature was utilized in the hybrid task. In terms of the hybrid task, the performance using combined feature was significantly better than using separate ERD or SSSEP feature. Significance. The results in this work validate the feasibility of our proposed approach to form a novel MI-SSSEP hybrid BCI outperforming a conventional MI-based BCI through combing MI with electrical stimulation.

Simultaneous and alternate action observation and motor imagery combinations improve aiming performance

With the continuous development of the brain-computer interface (BCI) technology, the lower limb rehabilitation system based on Motor Imagery (MI) has gradually become a research hotspot in the field of rehabilitation. To recognize the lower limbs MI, this paper designed an experimental paradigm for MI lower limb MI and used the 1D-CNN-LSTM deep learning algorithm to classify lower limb movement features from MI EEG signals. Compared with classical machine learning algorithms, the results showed that 1D-CNN-LSTM has relatively higher accuracy. Meanwhile, the paper built a real-time lower limb rehabilitation system based on the 1D-CNN-LSTM algorithm, which verifies the effectiveness and feasibility of the algorithm. The system provides an advanced and effective solution for brain-computer interfaces based on MI.

EEG and behavioural correlates of different forms of motor imagery during action observation in rhythmical actions

<p>Recently, studies on cycling-based brain–computer interfaces (BCIs) have been standing out due to their potential for lower-limb recovery. In this scenario, the behaviors of the sensory motor rhythms and the brain connectivity present themselves as sources of information that can contribute to interpreting the cortical effect of these technologies. This study aims to analyze how sensory motor rhythms and cortical connectivity behave when volunteers command reactive motor imagery (MI) BCI that provides passive pedaling feedback. We studied 8 healthy subjects who performed pedaling MI to command an electroencephalography (EEG)-based BCI with a motorized pedal to receive passive movements as feedback. The EEG data were analyzed under the following four conditions: resting, MI calibration, MI online, and receiving passive pedaling (on-line phase). Most subjects produced, over the foot area, significant event-related desynchronization (ERD) patterns around Cz when performing MI and receiving passive pedaling. The sharpest decrease was found for the low beta band. The connectivity results revealed an exchange of information between the supplementary motor area (SMA) and parietal regions during MI and passive pedaling. Our findings point to the primary motor cortex activation for most participants and the connectivity between SMA and parietal regions during pedaling MI and passive pedaling.</p>

Brain Computer Interface (BCI) has become one of the most interesting alternatives to support automatic systems able to interpret brain functions. Recently, the Motor Imagery (MI) paradigm is a widely topic of interest as a tool to develop BCI-based systems. Here, we present a relevant feature extraction methodology, termed MI discrimination using kernel relevance analysis (MIDKRA), to support MI classification in BCI systems. For such a purpose, a similarity criterion to rank the contribution of EEG features for classifying an MI paradigm is employed. The introduced approach includes the supervised information regarding the MI paradigm to find out a relevant set of features encoding discriminative information. We model the EEG recordings by considering both time and time-frequency representations. Moreover, a k nearest-neighbor classifier is carried out to validate the proposed feature relevance approach. Experimental results carried out on two different BCI databases, a well-known public MI data and an Emotiv-based dataset built by us, demonstrate that MIDKRA outperforms state of the art methods and it is a suitable alternative to support straightforward BCI systems.