DGSD: Dynamical graph self-distillation for EEG-based auditory spatial attention detection

Recent advances in auditory attention detection from multichannel electroencephalography (EEG) signals encounter the challenges of the scarcity of available online EEG data and the detection of auditory attention with low latency. To this end, we propose a complete deep auditory generative adversarial network auxiliary, named auditory-GAN, designed to handle these challenges while generating EEG data and executing auditory spatial detection. The proposed auditory-GAN system consists of a spectro-spatial feature extraction (SSF) module and an auditory generative adversarial network auxiliary (AD-GAN) classifier. The SSF module extracts the spatial feature maps by learning the topographic specificity of alpha power from EEG signals. The designed AD-GAN network addresses the need for extensive training data by synthesizing augmented versions of original EEG data. We validated the proposed method on the widely used KUL dataset. The model assesses the quality of generated EEG images and the accuracy of auditory spatial attention detection. Results show that the proposed auditory-GAN can produce convincing EEG data and achieves a significant i.e., 98.5% spatial attention detection accuracy for a 10-s decision window of 64-channel EEG data. Comparative analysis reveals that the proposed neural approach outperforms existing state-of-the-art models across EEG data ranging from 64 to 32 channels. The Auditory-GAN model is available at https://github.com/tasleem-hello/Auditory-GAN-/tree/Auditory-GAN.

Attention is one of many human cognitive functions that are essential in everyday life. Given our limited processing capacity, attention helps us focus only on what matters. Focusing attention on one speaker in an environment with many speakers is a critical ability of the human auditory system. This paper proposes a new end-to-end method based on the combined transformer and graph convolutional neural network (TraGCNN) that can effectively detect auditory attention from electroencephalograms (EEGs). This approach eliminates the need for manual feature extraction, which is often time-consuming and subjective. Here, the first EEG signals are converted to graphs. We then extract attention information from these graphs using spatial and temporal approaches. Finally, our models are trained with these data. Our model can detect auditory attention in both the spatial and temporal domains. Here, the EEG input is first processed by transformer layers to obtain a sequential representation of EEG based on attention onsets. Then, a family of graph convolutional layers is used to find the most active electrodes using the spatial position of electrodes. Finally, the corresponding EEG features of active electrodes are fed into the graph attention layers to detect auditory attention. The Fuglsang 2020 dataset is used in the experiments to train and test the proposed and baseline systems. The new TraGCNN approach, as compared with state-of-the-art attention classification methods from the literature, yields the highest performance in terms of accuracy (80.12%) as a classification metric. Additionally, the proposed model results in higher performance than our previously graph-based model for different lengths of EEG segments. The new TraGCNN approach is advantageous because attenuation detection is achieved from EEG signals of subjects without requiring speech stimuli, as is the case with conventional auditory attention detection methods. Furthermore, examining the proposed model for different lengths of EEG segments shows that the model is faster than our previous graph-based detection method in terms of computational complexity. The findings of this study have important implications for the understanding and assessment of auditory attention, which is crucial for many applications, such as brain–computer interface (BCI) systems, speech separation, and neuro-steered hearing aid development.

Objective. We consider the problem of Auditory Attention Detection (AAD), where the goal is to detect which speaker a person is attending to, in a multi-speaker environment, based on neural activity. This work aims to analyze the influence of head-related filtering and ear-specific decoding on the performance of an AAD algorithm. Approach. We recorded high-density EEG of 16 normal-hearing subjects as they listened to two speech streams while tasked to attend to the speaker in either their left or right ear. The attended ear was switched between trials. The speech stimuli were administered either dichotically, or after filtering using Head-Related Transfer Functions (HRTFs). A spatio-temporal decoder was trained and used to reconstruct the attended stimulus envelope, and the correlations between the reconstructed and the original stimulus envelopes were used to perform AAD, and arrive at a percentage correct score over all trials. Main results. We found that the HRTF condition resulted in significantly higher AAD performance than the dichotic condition. However, speech intelligibility, measured under the same set of conditions, was lower for the HRTF filtered stimuli. We also found that decoders trained and tested for a specific attended ear performed better, compared to decoders trained and tested for both left and right attended ear simultaneously. In the context of the decoders supporting hearing prostheses, the former approach is less realistic, and studies in which each subject always had to attend to the same ear may find over-optimistic results. Significance. This work shows the importance of using realistic binaural listening conditions and training on a balanced set of experimental conditions to obtain results that are more representative for the true AAD performance in practical applications.

Focusing on a single source within a complex auditory scene is challenging. M/EEG-based auditory attention detection (AAD) allows to detect which stream an individual is attending to within a set of multiple concurrent streams. The high interindividual variability in the auditory attention detection performance often is attributed to physiological factors and signal-to-noise ratio of neural data. We hypothesize that executive functions-in particular sustained attention, working memory, and attentional inhibition-may partly explain the variability in auditory attention detection performance, because they support the cognitive processes required when listening to complex auditory scenes. We chose a particularly challenging auditory scene by presenting dichotically polyphonic classical piano excerpts that lasted 1min each. Two different excerpts were presented simultaneously, one in each ear. Forty-one participants, with different degrees of musical expertise, listened to these complex auditory scenes focusing on one ear while we recorded the EEG. Participants also completed several tasks assessing executive functions. As expected, EEG-based auditory attention detection was greater for attended than unattended stimuli. Importantly, attentional inhibition ability did explain 6% of the reconstruction accuracy and 8% of the classification accuracy. No other executive function was a significant predictor of reconstruction or classification accuracies. No clear effect of musical expertise was found on reconstruction and classification performance. In conclusion, cognitive factors seem to impact the robustness of the neural auditory representation and hence the performance of EEG-based decoding approaches. Taking advantage of this relation could be useful to improve next-generation hearing aids.

Auditory spatial attention detection (ASAD) is used to determine the direction of a listener's attention to a speaker by analyzing her/his electroencephalographic (EEG) signals. This study aimed to further improve the performance of ASAD with a short decision window (i.e., <1 s) rather than with long decision windows ranging from 1 to 5 seconds in previous studies. An end-to-end temporal attention network (i.e., TAnet) was introduced in this work. TAnet employs a multi-head attention (MHA) mechanism, which can more effectively capture the interactions among time steps in collected EEG signals and efficiently assign corresponding weights to those EEG time steps. Experiments demonstrated that, compared with the CNN-based method and recent ASAD methods, TAnet provided improved decoding performance in the KUL dataset, with decoding accuracies of 92.4% (decision window 0.1 s), 94.9% (0.25 s), 95.1% (0.3 s), 95.4% (0.4 s), and 95.5% (0.5 s) with short decision windows (i.e., <1 s). As a new ASAD model with a short decision window, TAnet can potentially facilitate the design of EEG-controlled intelligent hearing aids and sound recognition systems.

In the context of a multi-speaker "cocktail party" scenario where listeners selectively focus on specific speakers, human auditory attention networks have shown a strong correlation with Electroencephalography (EEG) measurements. However, current EEG-based auditory attention detection (AAD) methods, mostly using artificial neural networks (ANN), face limitations on edge computing platforms due to extended decision windows, high power consumption, and substantial memory requirements linked to multiple EEG channels. This paper introduces a novel hybrid convolutional-spiking neural network (CNN-SNN) architecture, inspired by the auditory cortex, combining EEG data with multi-speaker speech envelopes, enabling effective auditory attention decoding within 0.5-s timeframes. Our approach reduces EEG channels, minimizes computational operations, and quantizes weight parameters while maintaining high accuracy. We validate this approach on our dataset and compare it to state-of-the-art methods on a publicly available dataset. CNN-SNN demonstrates superior performance, achieving up to 10% increase in decoding accuracy, while using 87.5% fewer EEG channels and 75% smaller bit precision for weight quantization compared to existing methods. These results offer promise for edge computing applications, such as hearing aids, emphasizing short decision windows, minimal EEG channels, and strict power and memory constraints.

State-of-the-art hearing prostheses are equipped with acoustic noise reduction algorithms to improve speech intelligibility. However, cocktail party scenarios with multiple speakers pose a major challenge since it is difficult for the algorithm to determine which speaker it should enhance. To address this problem, electroencephalography (EEG) signals can be used to perform auditory attention detection (AAD), i.e., to detect which speaker the listener is attending to. Taking a step further towards realization of a neuro-steered hearing prosthesis, we worked on AAD-assisted noise suppression in a competing-speakers scenario in the presence of babble noise. We use an EEG-informed AAD module in combination with a blind source separation algorithm to extract the per-speaker envelopes from the microphone recordings, as well as a multi-channel Wiener filter to extract the denoised speech signal(s). Using a new algorithm pipeline, we obtain better AAD accuracies, and a better robustness to variations in speaker positions and signal-to-noise ratios (SNR), compared to previously reported results. Furthermore, the algorithm allows to switch more swiftly to the other speaker’s stream when there is a switch in attention.

This paper considers the auditory attention detection (AAD) paradigm, where the goal is to determine which of two simultaneous speakers a person is attending to. The paradigm relies on recordings of the listener's brain activity, e.g., from electroencephalography (EEG). To perform AAD, decoded EEG signals are typically correlated with the temporal envelopes of the speech signals of the separate speakers. In this paper, we study how the inclusion of various degrees of auditory modelling in this speech envelope extraction process affects the AAD performance, where the best performance is found for an auditory-inspired linear filter bank followed by power law compression. These two modelling stages are computationally cheap, which is important for implementation in wearable devices, such as future neuro-steered auditory prostheses. We also introduce a more natural way to combine recordings (over trials and subjects) to train the decoder, which reduces the dependence of the algorithm on regularization parameters. Finally, we investigate the simultaneous design of the EEG decoder and the audio subband envelope recombination weights vector using either a norm-constrained least squares or a canonical correlation analysis, but conclude that this increases computational complexity without improving AAD performance.

The efficiency of a learning process directly depends on how well the students are attentive. Detecting the level of attention can help to improve the learning quality. In recent years, there have been several attempts to leverage EEG signal processing as a tool to detect whether a student is attentive or not. In such work, the level of attention is determined by analyzing the EEG power spectrum, which is mostly followed by machine learning approaches. However, the efficiency of such methods for detecting auditory attention of blind or visually-impaired students has not been analyzed. This study aims to investigate such a scenario. To this end, we propose an EEG recording protocol to simulate the learning process of visually-impaired students as closely as possible. Ten different university students participated in the experiment. In the proposed protocol, the EEG signals were recorded by using EMOTIV EPOC+ wireless EEG headset. Then, the power spectrum of the recorded EEG signals was analyzed, and the most relevant features were extracted using the Fisher feature selection method. Then, Linear SVM, RBF SVM, KNN, and LDA classifiers were used to evaluate the proposed protocol. The results of the classification showed that the level of auditory attention could be detected up to 89% accuracy.

Both visual and auditory spatial selective attention result in lateralized alpha (8–14 Hz) oscillatory power in parietal cortex: alpha increases in the hemisphere ipsilateral to attentional focus. Brain stimulation studies suggest a causal relationship between parietal alpha and suppression of the representation of contralateral visual space. However, there is no evidence that parietal alpha controls auditory spatial attention. Here, we performed high definition transcranial alternating current stimulation (HD-tACS) on human subjects performing an auditory task in which they directed attention based on either spatial or nonspatial features. Alpha (10 Hz) but not theta (6 Hz) HD-tACS of right parietal cortex interfered with attending left but not right auditory space. Parietal stimulation had no effect for nonspatial auditory attention. Moreover, performance in post-stimulation trials returned rapidly to baseline. These results demonstrate a causal, frequency-, hemispheric-, and task-specific effect of parietal alpha brain stimulation on top-down control of auditory spatial attention.

Auditory spatial attention serves important functions in auditory source separation and selection. Although auditory spatial attention mechanisms have been generally investigated, the neural substrates encoding spatial information acted on by attention have not been identified in the human neocortex. We performed functional magnetic resonance imaging experiments to identify cortical regions that support auditory spatial attention and to test 2 hypotheses regarding the coding of auditory spatial attention: 1) auditory spatial attention might recruit the visuospatial maps of the intraparietal sulcus (IPS) to create multimodal spatial attention maps; 2) auditory spatial information might be encoded without explicit cortical maps. We mapped visuotopic IPS regions in individual subjects and measured auditory spatial attention effects within these regions of interest. Contrary to the multimodal map hypothesis, we observed that auditory spatial attentional modulations spared the visuotopic maps of IPS; the parietal regions activated by auditory attention lacked map structure. However, multivoxel pattern analysis revealed that the superior temporal gyrus and the supramarginal gyrus contained significant information about the direction of spatial attention. These findings support the hypothesis that auditory spatial information is coded without a cortical map representation. Our findings suggest that audiospatial and visuospatial attention utilize distinctly different spatial coding schemes.

Auditory attention switch detection (AASD) poses significant challenges for adaptive neurotechnologies, particularly under electroencephalogram (EEG) with low signal-to-noise ratios (SNRs). However, the performance of existing methods is limited due to insufficient feature discriminability and high detection delay. To solve the problem, this paper proposes a Hierarchical Spatiotemporal Network (HSTN) for detecting auditory attention switch from EEG signals. The model employs a hierarchical spatiotemporal encoder to extract spatiotemporal features of EEG signals, integrates short-term transient and long-term dependency information through a multi-scale gated attention mechanism, and synchronously optimizes auditory attention switch detection and auditory attention decoding tasks via a multi-task joint training strategy. Experimental results demonstrate that HSTN significantly outperforms baseline models in both auditory attention switch detection (AASD F1=0.89, accuracy 88.6%) and auditory attention decoding tasks (AAD accuracy 89.3%), with superior model parameter efficiency and inference time. Ablation experiments further validate the critical roles of multi-task learning, gated attention, and multi-scale convolutions. This study provides an efficient solution for auditory attention switch detection in complex auditory scenarios.Clinical Relevance-The study confirms that spatiotemporal feature encoding combined with multi-task joint training significantly enhances performance in EEG attention switch detection, providing a practical technical framework for enabling dynamic sound source enhancement in intelligent hearing aids and auditory brain-computer interface systems.

Existing research has revealed that auditory attention can be tracked from ongoing electroencephalography (EEG) signals. The aim of this novel study was to investigate the identification of peoples’ attention to a specific auditory object from single-trial EEG signals via entropy measures and machine learning. Approximate entropy (ApEn), sample entropy (SampEn), composite multiscale entropy (CmpMSE) and fuzzy entropy (FuzzyEn) were used to extract the informative features of EEG signals under three kinds of auditory object-specific attention (Rest, Auditory Object1 Attention (AOA1) and Auditory Object2 Attention (AOA2)). The linear discriminant analysis and support vector machine (SVM), were used to construct two auditory attention classifiers. The statistical results of entropy measures indicated that there were significant differences in the values of ApEn, SampEn, CmpMSE and FuzzyEn between Rest, AOA1 and AOA2. For the SVM-based auditory attention classifier, the auditory object-specific attention of Rest, AOA1 and AOA2 could be identified from EEG signals using ApEn, SampEn, CmpMSE and FuzzyEn as features and the identification rates were significantly different from chance level. The optimal identification was achieved by the SVM-based auditory attention classifier using CmpMSE with the scale factor τ = 10. This study demonstrated a novel solution to identify the auditory object-specific attention from single-trial EEG signals without the need to access the auditory stimulus.

A short period of adaptation to a prismatic shift of the visual field to the right briefly but significantly improves left unilateral spatial neglect. Additionally, prism adaptation affects multiple modalities, including processes of vision, auditory spatial attention, and sound localization. This non-randomized, single-center, controlled trial aimed to examine the immediate effects of prism adaptation on the sound-localization abilities of patients with left unilateral spatial neglect using a simple source localization test. Subjects were divided by self-allocation into a prism-adaptation group (n = 11) and a control group (n = 12). At baseline, patients with left unilateral spatial neglect showed a rightward deviation tendency in the left space. This tendency to right-sided bias in the left space was attenuated after prism adaptation. However, no changes were observed in the right space of patients with left unilateral spatial neglect after prism adaptation, or in the control group. Our results suggest that prism adaptation improves not only vision and proprioception but also auditory attention in the left space of patients with left unilateral spatial neglect. Our findings demonstrate that a single session of prism adaptation can significantly improve sound localization in patients with left unilateral spatial neglect. However, in this study, it was not possible to accurately determine whether the mechanism was a chronic change in head orientation or a readjustment of the spatial representation of the brain; thus, further studies need to be considered.

Autism Spectrum Disorder (ASD) is a developmental condition characterized by difficulties in social interaction, communication, and sensory processing. The ability to orient towards sounds is a key component of social interactions, yet auditory spatial attention remains relatively understudied in ASD, despite prior research indicating differences in this domain. Here, we investigate the neural signatures associated with passive auditory spatial attention in children with ASD (n = 21, ages 6-17) relative to age- and IQ-matched Typically Developing (TD) children (n = 31), using source-localized magnetoencephalography (MEG). Participants listened passively, while watching a silenced movie, to non-social auditory stimuli designed to either remain lateralized to one hemifield (stay trials) or to change in location from one side to the contralateral hemifield (jump trials). Linear mixed effects modeling showed lower cortical activation in the auditory cortex in the ASD group in response to jump trials, relative to the TD group. Additionally, functional connectivity analyses showed higher alpha-band functional connectivity in the ASD group between left auditory cortex seeds and right prefrontal and left parietal regions known to be recruited during auditory spatial attention. Right prefrontal alpha-band connectivity estimates were associated with behaviorally assessed auditory processing scores, whereas left parietal connectivity estimates were associated with ASD symptomatology. Our results align with the hypothesis that auditory spatial attention generally, and specifically orientation to sounds even when experienced passively, differs in ASD versus TD children.