Decoding Models Research Articles (Page 2)

Working memory, a fundamental cognitive function of the brain, necessitates the evaluation of cognitive load intensity due to limited cognitive resources. Optimizing cognitive load can enhance task performance efficiency by preventing resource waste and overload. Therefore, identifying working memory load is an essential area of research. Deep learning models have demonstrated remarkable potential in identifying the intensity of working memory load. However, existing neural networks based on electroencephalogram (EEG) decoding primarily focus on temporal and spatial characteristics while neglecting frequency characteristics. Drawing inspiration from the role of cross-frequency coupling in the hippocampal region, which plays a crucial role in advanced cognitive processes such as working memory, this study proposes a Multi-Band Multi-Scale Hybrid Sinc Convolutional Neural Network (MBSincNex). This model integrates multi-frequency and multi-scale Sinc convolution to facilitate time-frequency conversion and extract time-frequency information from multiple rhythms and regions of the EEG data with the aim of effectively model the cross-frequency coupling across different cognitive domains. Due to its unique structural design, the proposed model proficiently extracts features in temporal, frequency, and spatial domains while its feature extraction capability is validated through post-hoc interpretability techniques. On our collected three-class working memory dataset, the proposed model achieved higher classification accuracy compared to other state-of-the-art methods. Furthermore, by analyzing the model's classification performance during different stages of working memory processes, this study emphasizes the significance of the encoding phase and confirms that behavioral response does not accurately reflect cognitive load.

In recent years, there has been growing interest among researchers in the field of image captioning, which involves generating one or more descriptions for an image that closely resembles a human-generated description. Most of the existing studies in this area focus on the English language, utilizing CNN and RNN variants as encoder and decoder models, often enhanced by attention mechanisms. Despite Bengali being the fifth most-spoken native language and the seventh most widely spoken language, it has received far less attention in comparison to resource-rich languages like English. This study aims to bridge that gap by introducing a novel approach to image captioning in Bengali. By leveraging state-of-the-art Convolutional Neural Networks such as EfficientNetV2S, ConvNeXtSmall, and InceptionResNetV2 along with an improvised Transformer, the proposed system achieves both computational efficiency and the generation of accurate, contextually relevant captions. Additionally, Bengali text-to-speech synthesis is incorporated into the framework to assist visually impaired Bengali speakers in understanding their environment and visual content more effectively. The model has been evaluated using a chimeric dataset, combining Bengali descriptions from the Ban-Cap dataset with corresponding images from the Flickr 8k dataset. Utilizing EfficientNet, the proposed model attains METEOR, CIDEr, and ROUGE scores of 0.34, 0.30, and 0.40, while BLEU scores for unigram, bigram, trigram, and four-gram matching are 0.66, 0.59, 0.44 and 0.26 respectively. The study demonstrates that the proposed approach produces precise image descriptions, outperforming other state-of-the-art models in generating Bengali descriptions.

Displaying high-quality images on edge devices, such as augmented reality devices, is essential for enhancing the user experience. However, these devices often face power consumption and computing resource limitations, making it challenging to apply many deep learning-based image compression algorithms in this field. Implicit Neural Representation (INR) for image compression is an emerging technology that offers two key benefits compared to cutting-edge autoencoder models: low computational complexity and parameter-free decoding. It also outperforms many traditional and early neural compression methods in terms of quality. In this study, we introduce a new Mixed AutoRegressive Model (MARM) to significantly reduce the decoding time for the current INR codec, along with a new synthesis network to enhance reconstruction quality. MARM includes our proposed AutoRegressive Upsampler (ARU) blocks, which are highly computationally efficient, and ARM from previous work to balance decoding time and reconstruction quality. We also propose enhancing ARU's performance using a checkerboard two-stage decoding strategy. Moreover, the ratio of different modules can be adjusted to maintain a balance between quality and speed. Comprehensive experiments demonstrate that our method significantly improves computational efficiency while preserving image quality. With different parameter settings, our method can achieve over a magnitude acceleration in decoding time without industrial level optimization or achieve state-of-the-art reconstruction quality compared with other INR codecs. To the best of our knowledge, our method is the first INR-based codec comparable with Ballé et al. [1] in both decoding speed and quality while maintaining low complexity.

Auditory attention decoding (AAD) is the process of identifying the attended speech in a multi-talker environment using brain signals, typically recorded through electroencephalography (EEG). Over the past decade, AAD has undergone continuous development, driven by its promising application in neuro-steered hearing devices. Most AAD algorithms are relying on the increase in neural entrainment to the envelope of attended speech, as compared to unattended speech, typically using a two-step approach. First, the algorithm predicts representations of the attended speech signal envelopes; second, it identifies the attended speech by finding the highest correlation between the predictions and the representations of the actual speech signals. In this study, we proposed a novel end-to-end neural network architecture, named AADNet, which combines these two stages into a direct approach to address the AAD problem. We compare the proposed network against traditional stimulus decoding-based approaches, including linear stimulus reconstruction, canonical correlation analysis, and an alternative non-linear stimulus reconstruction using three different datasets. AADNet shows a significant performance improvement for both subject-specific and subject-independent models. Notably, the average subject-independent classification accuracies for different analysis window lengths range from 56.3% (1 s) to 78.1% (20 s), 57.5% (1 s) to 89.4% (40 s), and 56.0% (1 s) to 82.6% (40 s) for three validated datasets, respectively, showing a significantly improved ability to generalize to data from unseen subjects. These results highlight the potential of deep learning models for advancing AAD, with promising implications for future hearing aids, assistive devices, and clinical assessments.

Breast cancer remains a formidable health challenge requiring advanced computational tools for accurate diagnosis and treatment planning. This study hypothesizes that modifica-tions to the DeepLabV3+ architecture, such as incorporating an attention layer and replacing the ResNet50 backbone with Xcep-tion, can significantly enhance segmentation accuracy and model stability for breast cancer histopathological images. To test this hy-pothesis, we evaluated the performance of the original DeepLabV3+ and three modified versions for semantic segmenta-tion using the “Breast Cancer Semantic Segmentation” (BCSS) da-taset, which provides pixel-wise annotations of breast cancer tis-sues. The proposed modifications include integrating an attention layer between the encoder and decoder (Model 1), replacing the ResNet50 backbone with an Xception backbone up to 'block5' (Model 2), and combining the Xception backbone with the atten-tion layer (CancerSeg-XA). The models were implemented and trained in the Kaggle Notebook environment, and their perfor-mance was assessed based on training and validation accuracy. The results show that Model 1 improved the model stability and accuracy compared to DeepLabV3+, whereas Model 2 and Can-cerSeg-XA achieved significant accuracy improvements of 91.47% and 91.57%, respectively, over the baseline DeepLabV3+ accuracy of 85.7%. CancerSeg-XA demonstrated enhanced training stabil-ity, making it a promising approach for clinical application in breast cancer diagnosis and treatment.

In recent years, advancements in wearable device technology have facilitated the shift from clinical epilepsy monitoring units to home-based EEG systems. This transition has driven the increasing demand for personalized precision treatments in ultra-long-term dynamic EEG data. However, existing EEG decoding models employ static architectures (with fixed parameters after training), which struggle to adapt to data distribution shifts caused by physiological, pharmacological, and environmental factors, limiting their applicability in long-term monitoring. To address this issue, we propose SAR-LTS, a novel incremental learning framework for seizure detection. Based on the significant similarity exhibited in short-term neural activity, SAR-LTS utilizes the similarity-aware mechanism with local temporal sampling to construct an experience pool for dynamically storing representative EEG samples. These samples are then replayed periodically via stratified random (SR) retrieval to reinforce historical knowledge. Experimental results on CHB-MIT and Siena datasets demonstrate that SAR-LTS significantly outperforms baseline models in patient-specific incremental learning scenarios. Compared to frozen models, it achieves an average accuracy improvement of 9.0-20.8%. This study provides a promising technical solution for long-term personalized epilepsy management. Source codes are available at: https://github.com/kaixindamaoxian/Incremental-Learning-for-Patient-Specific-EEG-Based-Seizure-Detection.

Computational models of auditory processing can be valuable tools for research and technology development. Models of the cochlea are highly accurate and widely used, but models of the auditory brain lag far behind in both performance and penetration. Here we present ICNet, a convolutional encoder–decoder model of neural coding in the inferior colliculus. We developed ICNet using large-scale intracranial recordings from anaesthetized gerbils, addressing three key modelling challenges that are common across all sensory systems: capturing the full statistical structure of neuronal response patterns; accounting for physiological and experimental non-stationarity; and extracting features of sensory processing that are shared across different brains. ICNet provides highly accurate simulation of multi-unit neural responses to a wide range of complex sounds, including near-perfect responses to speech. It also reproduces key neurophysiological phenomena such as forward masking and dynamic range adaptation. ICNet can be used to simulate activity from thousands of neural units or to provide a compact representation of early central auditory processing through its latent dynamics, facilitating a wide range of hearing and audio applications. It can also serve as a foundation core, providing a baseline neural representation for models of active listening or higher-level auditory processing.

IntroductionIn our complex world, the auditory system plays a crucial role in perceiving and processing our environment. Humans are able to segment and stream concurrent auditory objects, allowing them to focus on specific sounds, such as speech, and suppress irrelevant auditory objects. The attentional enhancement or suppression of sound processing is evident in neural data through a phenomenon called neural speech tracking. Previous studies have identified correlates of neural speech tracking in electroencephalography (EEG) data, but EEG measures are susceptible to motion artefacts, and the association between neural data and auditory objects is vulnerable to distraction.MethodsThe current study investigated EEG-based auditory attention decoding in realistic everyday scenarios. N=20 participants were exposed to the sound of a busy cafeteria or walked along busy and quiet streets while listening to one or two simultaneous speech streams. We also investigated the robustness of neural speech tracking estimates within subjects. Linear decoding models were used to determine the magnitude of neural speech tracking.ResultsThe results confirmed that neural speech tracking was strongest in single speaker scenarios. In dual speaker conditions, there was significantly stronger neural speech tracking for the attended speaker compared to the ignored speaker, even in complex environments such as a busy cafeteria or outdoor settings.DiscussionIn conclusion, EEG-based attention decoding is feasible in highly complex and realistic everyday conditions while humans behave naturally.

Image quality improvement in single plane-wave imaging using deep learning

Plant disease identification is a crucial issue in agriculture, and with the advancement of deep learning techniques, early and accurate identification of plant diseases has become increasingly critical. In recent years, the rise of vision transformers has attracted significant attention from researchers in various vision-based application areas. We designed a model with an encoder–decoder architecture to efficiently classify plant diseases using a transfer learning approach, which effectively recognizes a large number of plant diseases in multiple crops. The model was tested on the “PlantVillage”, “FGVC8”, and “EMBRAPA” datasets, which contain leaf information from crops such as apples, soybeans, tomatoes, and potatoes. These datasets cover diseases caused by fungi, including rust, spot, and scab, as well as viral diseases such as leaf curl. The model’s performance was rigorously evaluated on datasets, and the results demonstrated its high accuracy. The model achieved 99.9% accuracy on the “PlantVillage” dataset, 97.4% on the “EMBRAPA” dataset, and 91.5% on the “FGVC8” dataset, showcasing its competitiveness with other state-of-the-art models. This study provides a robust and reliable solution for plant disease classification and contributes to the advancement of precision agriculture.

Segmentation and Estimation of Fetal Biometric Parameters using an Attention Gate Double U-Net with Guided Decoder Architecture

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

Brain-Computer Interface (BCI) is a promising neu-rotechnology offering non-muscular control of external devices, such as neuroprostheses and robotic exoskeletons. A new yet under-explored BCI control paradigm is Motion Trajectory Prediction (MTP). While MTP provides continuous control signals suitable for high-precision tasks, its feasibility and applications are challenged by the low signal-to-noise ratio, especially in noninvasive settings. Previous research has predominantly focused on kinematic reconstruction of upper (e.g., arm reaching) and lower limbs (e.g., gait). However, finger movements have received much less attention, despite their crucial role in daily activities. To address this gap, our study explores the potential of noninvasive Electroencephalography (EEG) for reconstructing finger movements, specifically during hand grasping actions. A new experimental paradigm to collect multichannel EEG data from 20 healthy subjects, while performing full, natural hand opening and closing movements, was designed. Employing state-of-the-art deep learning algorithms, continuous decoding models were constructed for eight key finger joints. The Convolutional Neural Network with Attention approach achieved an average decoding performance of r=0.63. Furthermore, a post-hoc metric was proposed for hand grasp cycle detection, and 83.5% of hand grasps were successfully detected from the reconstructed motion signals, which can potentially serve as a new BCI command. Explainable AI algorithm was also applied to analyze the topographical relevance of trained features. Our findings demonstrate the feasibility of using EEG to reconstruct hand joint movements and highlight the potential of MTP-BCI in control and rehabilitation applications.

A novel hybrid decoding neural network for EEG signal representation

This article proposes an Advanced Persistent Threat (APT) attack detection method based on traceability graphs, aimed at addressing the complexity and concealment of APT attacks. This method describes system behavior by constructing a traceability graph, optimizing it to reduce redundant information, converting the traceability graph sequence into a feature vector sequence, and using an encoder decoder model to train the GRU (Gate Recurrent Unit) model to extract long-term features of the sequence. Finally, a normal behavior model is established through clustering to detect APT attacks.

Alterations of mismatch responses (ie, neural activity evoked by unexpected stimuli) are often considered a potential biomarker of schizophrenia. Going beyond establishing the type of observed alterations found in diagnosed patients and related cohorts, computational methods can yield valuable insights into the underlying disruptions of neural mechanisms and cognitive function. Here, we adopt a typology of model-based approaches from computational cognitive neuroscience, providing an overview of the study of mismatch responses and their alterations in schizophrenia from four complementary perspectives: (a) connectivity models, (b) decoding models, (c) neural network models, and (d) cognitive models. Connectivity models aim at inferring the effective connectivity patterns between brain regions that may underlie mismatch responses measured at the sensor level. Decoding models use multivariate spatiotemporal mismatch response patterns to infer the type of sensory violations or to classify participants based on their diagnosis. Neural network models such as deep convolutional neural networks can be used for improved classification performance as well as for a systematic study of various aspects of empirical data. Finally, cognitive models quantify mismatch responses in terms of signaling and updating perceptual predictions over time. In addition to describing the available methodology and reviewing the results of recent computational psychiatry studies, we offer suggestions for future work applying model-based techniques to advance the study of mismatch responses in schizophrenia.

Yes, I am afraid of the sharks and also wild lions!: A multitask framework for enhancing dialogue generation via knowledge and emotion grounding

The rapid serial visual presentation-based brain–computer interface (RSVP-BCI) system achieves the recognition of target images by extracting event-related potential (ERP) features from electroencephalogram (EEG) signals and then building target classification models. Currently, how to reduce the training and calibration time for classification models across different subjects is a crucial issue in the practical application of RSVP. To address this issue, a zero-calibration (ZC) method termed Attention-ProNet, which involves meta-learning with a prototype network integrating multiple attention mechanisms, was proposed in this study. In particular, multiscale attention mechanisms were used for efficient EEG feature extraction. Furthermore, a hybrid attention mechanism was introduced to enhance model generalization, and attempts were made to incorporate suitable data augmentation and channel selection methods to develop an innovative and high-performance ZC RSVP-BCI decoding model algorithm. The experimental results demonstrated that our method achieved a balance accuracy (BA) of 86.33% in the decoding task for new subjects. Moreover, appropriate channel selection and data augmentation methods further enhanced the performance of the network by affording an additional 2.3% increase in BA. The model generated by the meta-learning prototype network Attention-ProNet, which incorporates multiple attention mechanisms, allows for the efficient and accurate decoding of new subjects without the need for recalibration or retraining.

Dimension reduction on neural activity paves a way for unsupervised neural decoding by dissociating the measurement of internal neural state repetition from the measurement of external variable tuning. With assumptions only on the smoothness of latent dynamics and of internal tuning curves, the Poisson Gaussian-process latent variable model (P-GPLVM) (Wu et al., 2017) is a powerful tool to discover the low-dimensional latent structure for high-dimensional spike trains. However, when given novel neural data, the original model lacks a method to infer their latent trajectories in the learned latent space, limiting its ability for estimating the internal state repetition. Here, we extend the P-GPLVM to enable the latent variable inference of new data constrained by previously learned smoothness and mapping information. We also describe a principled approach for the constrained latent variable inference for temporally-compressed patterns of activity, such as those found in population burst events (PBEs) during hippocampal sharp-wave ripples, as well as metrics for assessing whether the inferred new latent variables are congruent with a previously learned manifold in the latent space. Applying these approaches to hippocampal ensemble recordings during active maze exploration, we replicate the result that P-GPLVM learns a latent space encoding the animal’s position. We further demonstrate that this latent space can differentiate one maze context from another. By inferring the latent variables of new neural data during running, certain internal neural states are observed to repeat, which is in accordance with the similarity of experiences encoded by its nearby neural trajectories in the training data manifold. Finally, repetition of internal neural states can be estimated for neural activity during PBEs as well, allowing the identification for replay events of versatile behaviors and more general experiences. Thus, our extension of the P-GPLVM framework for unsupervised analysis of neural activity can be used to answer critical questions related to scientific discovery.

Decoding consciousness from different time-scale spatiotemporal dynamics in resting-state electroencephalogram