model of decoder Research Articles (Page 1)

Purpose This paper aims to enhance the quality of images generated by electroencephalography (EEG) encoding and decoding models; this paper improves the neurological plausibility of the generated images by emulating key mechanisms of human visual processing. Design/methodology/approach Motivated by the center-periphery organization of visual perception, an EEG segmentation module and a feature fusion module are introduced into a diffusion-based EEG-to-image generation framework. The EEG segmentation module decomposes the input signal into a segment with high fluctuation amplitude and the remaining signal, which are separately encoded and subsequently fused to enable differentiated visual modeling. In addition, a neuro-inspired application framework is proposed to extend the EEG-to-image generation approach to Web search scenarios, where EEG-generated images serve as implicit visual representations of user intent. Findings Experimental results demonstrate that integrating EEG segmentation and feature fusion leads to measurable improvements in the perceptual quality and structural coherence of EEG-generated images. Originality/value An EEG segmentation module and a feature fusion module are introduced into a diffusion-based EEG-to-image generation framework. The EEG segmentation module decomposes the input signal into a segment with high fluctuation amplitude and the remaining signal, which are separately encoded and subsequently fused to enable differentiated visual modeling. A neuro-inspired application framework is proposed to extend the EEG-to-image generation approach to Web search scenarios, where EEG-generated images serve as implicit visual representations of user intent.

With the development of artificial intelligence technology, text-driven image generation has gradually become a research hotspot. In terms of the application of traditional Chinese painting generation, due to the particularity of traditional Chinese painting themes, techniques, and artistic conception, existing models have problems such as cross-modal alignment deviation. To enhance the model's understanding of text semantics and improve the matching degree between text and generated Chinese painting images, a multimodal dataset of Chinese paintings is studied, and the Vector Quantized Generative Adversarial Network (VQGAN) is improved. A new multimodal text generation method is constructed by combining transformer-based bidirectional encoder representation, Transformer decoder, and contrastive language-image pre-training models. The results showed that when trained for 100 rounds on the general object dataset and Flickr30k dataset in context, the Inception Score (IS) values of this method were 39.6 and 5.1, respectively, and the Fréchet Inception Distance (FID) values were 2.3 and 1.7, respectively, which were better than models such as VQGAN. In the ablation experiment, the IS was 5.8, the FID was 10.3, and the cosine similarity was 0.82. The convergence was achieved after 60 rounds, which was better than other variants. The method proposed in the study has good adaptability to the generation of traditional Chinese painting. Although the expansion of complex scenes is slightly weak, the overall performance is excellent, filling the gap in traditional artistic semantic mapping and providing support for the digital dissemination and innovation of traditional Chinese painting.

Steady-state visual evoked potential (SSVEP)-based brain–computer interfaces (BCIs) offer a high information transfer rate (ITR), but are often constrained by the computational burden and limited performance of existing decoding models. To address these challenges, we propose LARSNet, a lightweight asymmetric residual Siamese network tailored for efficient and accurate multiclass SSVEP classification. LARSNet employs an asymmetric architecture comprising a residual-enhanced signal branch for robust feature extraction from low signal-to-noise ratio (SNR) data and a reference branch for high-fidelity template construction. A correlation analysis layer with a parameter-efficient approximate weight mechanism further refines the discriminative capability by modeling joint stimulus interference. Comprehensive evaluations on 40-class and 12-class datasets demonstrated that LARSNet significantly outperformed traditional and deep learning methods, particularly for short data segments, while reducing model complexity to 491 parameters and 1.00 mega floating-point operations (MFLOPs). In particular, with 0.4-s data, the 40-class dataset achieved an ITR of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$254.15~\pm ~69.51$ </tex-math></inline-formula> bits/min, while the 12-class dataset reached <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$125.78~\pm ~34.26$ </tex-math></inline-formula> bits/min. Feature visualization further revealed a progressive SNR enhancement of SSVEP characteristic frequencies across layers. Collectively, these results highlight the potential of LARSNet for SSVEP-BCI applications, offering a robust balance of high performance and low complexity.

As a core component of the maritime Internet of Things (IoT), the Automatic Identification System (AIS) continuously collects dynamic vessel navigation data, providing a solid foundation for addressing complex maritime traffic prediction tasks that support intelligent Vessel Traffic Services (VTS), such as vessel trajectory prediction and vessel arrival time (VAT) estimation. However, existing methods typically focus on single prediction objectives, falling short of meeting practical multi-task requirements. To address this gap, this study proposes a geospatial grid-constrained deep learning framework based on AIS data to simultaneously handle three key prediction tasks: vessel trajectory prediction, whether the vessel arrives within the specified time, and VAT. The framework incorporates a dynamic patch construction method and a Graph Soft Evolution (GSE) module to capture temporal correlations among observations under spatial grid constraints. An encoder-decoder architecture is introduced, where the encoder employs a Squeeze-and-Excitation (SE) block to adaptively select feature channels, and the decoder models dependencies across both variable and temporal dimensions. In a case study of New York Harbor, the model achieved an R² of 0.8386 and RMSE of 0.0329 for latitude increment prediction, and an R² of 0.8432 with RMSE of 0.0322 for longitude increment prediction. It also attained 99.83% accuracy in arrival status prediction and an R² of 0.9350 with RMSE of 0.0701 for VAT prediction. The framework demonstrated effectiveness in port scheduling and robust generalizability in cross-validation experiments at the Port of Los Angeles, thereby demonstrating its substantial potential to enhance the operational efficiency of VTS within maritime IoT systems.

Individual differences pose a significant challenge in brain-computer interface (BCI) research. Designing a universally applicable network architecture is impractical due to the variability in human brain structure and function. We propose Filter-Bank Neural Architecture Search (FBNAS), an EEG decoding framework that automates network architecture design for individuals. FBNAS uses three temporal cells to process different frequency EEG signals, with dilated convolution kernels in their search spaces. A multi-path NAS algorithm determines optimal architectures for multi-scale feature extraction. We benchmarked FBNAS on three EEG datasets across two BCI paradigms, comparing it to six state-of-the-art deep learning algorithms. FBNAS achieved cross-session decoding accuracies of 79.78%, 70.66%, and 68.38% on the BCIC-IV-2a, OpenBMI, and SEED datasets, respectively, outperforming other methods. Our results show that FBNAS customizes decoding models to address individual differences, enhancing decoding performance and shifting model design from expert-driven to machine-aided. The source code can be found at https://github.com/wang1239435478/FBNAS-master.

Brain-computer interfaces (BCIs) attempt to establish a connection between the human mind and a computer system. While recent computational advances continue to improve these interfaces, human factors have been overlooked. Factors such as fatigue and attention play a key role in brain signal modulation. This arises the need for paradigms designed and implemented in terms of human factors. Therefore, it is proposed to improve the level of engagement to diminish fatigue and increase attention by a video game-based paradigm for an imagined speech BCI. For this purpose, a sample of 15 volunteers (females = 7) was recruited to study the quality of their imagined speech when it is evoked under an abstract scenario (traditional paradigm) and a video-game paradigm. This dataset helps to study the differences in imagined speech signals when using two different paradigms: (1) one that does not consider human factors, and (2) one that does. Additional applications may include designing imagined speech decoding models for BCI and studying the relationship between users' profile and their imagined speech signals.

Rapid Serial Visual Presentation (RSVP)-based Brain-Computer Interface (BCI) can efficiently detect rare target images by capturing event-related potentials (ERPs) evoked by target images in electroencephalogram (EEG) signals. RSVP-based BCI systems require subject-specific data collection to train the decoding model for one new user. Their performances depend on the amount of each new subject’s data for training a model, which limits practical application. Thus, it is necessary to develop the zero-calibration method in RSVP-based BCIs, requiring only historical collected data to train models for a new user. In this work, we introduce the pupil modality and develop a multi-modal method combining the EEG and pupil signals to enhance zero-calibration RSVP-based target detection performance. The proposed ResVPNet, based on the residual network tailored for RSVP signal decoding, extracts signal features for each modality with a two-branch structure. A MixSub strategy is designed to implicitly generate new signal samples by mixing subject-related information between subjects to enhance the diversity of training distributions. Also, a Prototype-based Cross-Modal Alignment (PCMA) module is employed to align multi-modal signal features and separate different class samples. Our proposed network achieved a balanced accuracy of 91.89%, surpassing the performance of the compared methods. The ablation studies and visualizations revealed the effectiveness of the proposed modules. This work provides an effective method for the zero-calibration RSVP-based target detection system.

Brain-computer interfaces (BCIs) enable direct communication with computers by encoding and decoding brain electrical signals to generate control signals and interact directly with external devices, thereby assisting patients with damaged motor nerves. Research on brain electrical signals based on motor imagery has always received widespread attention. To effectively extract and classify complicated brain electrical signal features, a system architecture that can withstand a low ratio between signal and noise, instability, and physiological artifacts must be constructed. Advancements in motor imagery electroencephalography have been made possible by the rapid development of deep learning over the past few years. This paper aims to introduce and analyze three decoding models that use convolutional neural network architectures (CNN) and three combinations of CNN and Transformer architectures. Through similar comparisons and cross-comparisons, by analyzing the accuracy of each model on the current mainstream motor imagery-related datasets BCI IV 2A and 2B, the advantages and disadvantages of each model are explored. Through comparison, it can be seen that models based on the CNN architecture still occupy a dominant position due to their fewer parameters and better adaptability to various situations. However, the confusion decoding model CTNet performs very well on the BCI IV 2A and 2B, indicating that the decoding model with the Transformer architecture has higher performance development potential and is the main direction of future development.

Accurate reservoir inflow forecasting is vital for effective water resource management. Reliable forecasts enable operators to optimize storage and release strategies to meet competing sectoral demands—such as water supply, irrigation, and hydropower scheduling—while also mitigating flood and drought risks. To address this need, in this study, we propose a novel time-variant encoder–decoder (ED) model designed specifically to improve multi-step reservoir inflow forecasting, enabling accurate predictions of reservoir inflows up to seven days ahead. Unlike conventional ED-LSTM and recursive ED-LSTM models, which use fixed encoder parameters or recursively propagate predictions, our model incorporates an adaptive encoder structure that dynamically adjusts to evolving conditions at each forecast horizon. Additionally, we introduce the Expected Baseline Integrated Gradients (EB-IGs) method for variable importance analysis, enhancing interpretability of inflow by incorporating multiple baselines to capture a broader range of hydrometeorological conditions. The proposed methods are demonstrated at several diverse reservoirs across the United States. Our results show that they outperform traditional methods, particularly at longer lead times, while also offering insights into the key drivers of inflow forecasting. These advancements contribute to enhanced reservoir management through improved forecasting accuracy and practical decision-making insights under complex hydroclimatic conditions.

The hippocampus is crucial for forming new episodic memories. While its role in encoding spatial and temporal information (where and when) is well understood, how it encodes objects (what) remains unclear due to the high dimensionality of object space. Rather than encoding each object separately, the hippocampus may encode object categories to reduce complexity. Here, an experimental‐modeling approach to investigate how the hippocampus encodes visual memory categories in humans is developed. Spikes are recorded from hippocampal CA3 and CA1 neurons in 24 epilepsy patients performing a delayed match‐to‐sample task involving five image categories. An interpretable memory decoding model is employed to decode memory categories from hippocampal spiking activity and identify the spatio‐temporal characteristics of hippocampal encoding. Using this model, the optimal temporal resolutions for decoding each visual memory category per neuron are estimated. Results indicate that visual memory categories can be decoded from hippocampal spike patterns, supporting the presence of category‐specific coding. Hippocampal neuron ensembles encode memory categories in a distributed manner, akin to a population code, while individual neurons use a temporal code. Additionally, CA3 and CA1 neurons exhibit similar and redundant memory category information, likely due to strong and diffuse feedforward synaptic connections from CA3 to CA1 regions.

Intracranial electroencephalography (iEEG) is increasingly used for clinical and brain-computer interface applications due to its high spatial and temporal resolution. However, inter-subject variability in electrode implantation poses a challenge for developing generalized neural decoders. To address this, we introduce a novel decoder model that is robust to inter-subject electrode implantation variability. We call this model RISE-iEEG, which stands for Robust to Inter-Subject Electrode Implantation Variability iEEG Classifier. RISE-iEEG employs a deep neural network structure preceded by a participant-specific projection network. The projection network maps the neural data of individual participants onto a common low-dimensional space, compensating for the implantation variability. In other words, we developed an iEEG decoder model that can be applied across multiple participants' data without requiring the coordinates of electrode for each participant. The performance of RISE-iEEG across multiple datasets, including the Music Reconstruction dataset, and AJILE12 dataset, surpasses that of advanced iEEG decoder models such as HTNet and EEGNet. Our analysis shows that the performance of RISE-iEEG is about 7% higher than that of HTNet and EEGNet in terms of F1 score, with an average F1 score of 0.83, which is the highest result among the evaluation methods defined. Furthermore, Our analysis of the projection network weights reveals that the Superior Temporal and Postcentral lobes are key encoding nodes for the Music Reconstruction and AJILE12 datasets, which aligns with the primary physiological principles governing these regions. This model improves decoding accuracy while maintaining interpretability and generalization.

Characterizing biobehavioral time series data recorded under different task conditions is a critical step in neuroscience data analysis. The inherent complexity and stochasticity present in these data pose significant modeling challenges. Deep Neural Network (DNN) models have been widely adopted for analyzing such data. Despite their success, DNNs have significant limitations: they are sensitive to noise in training data and require large training datasets. These issues limit the applicability of DNNs to neuroscience data - where sample size is often smaller and data are volatile and feature noise artifacts. The present manuscript introduces a novel framework that embeds a state-space model (SSM) in the classic DNN structure. We demonstrate that this framework, which we call SSM-DNN, can overcome the sample size and noise sensitivity issues that plague classic DNNs. For this purpose, we illustrate training and inference steps when SSM-DNN is applied to a dataset recorded during a Death Implicit Association Test (D-IAT) task. This task was designed to produce biobehavioral data that facilitate decoding of participant phenotypes (e.g., person with depression vs. psychologically healthy person). We show SSM-DNN performance reaches an accuracy of 78%, which is 20% higher than state-of-art DNN decoder models. Its area under curve (AUC) is 0.8, which reflects its high specificity and sensitivity. The SSM-DNN modeling framework is scalable to high-dimensional time-series data and it can be applied broadly to neuroscience data, providing a robust and accurate decoding performance.

Feature extraction and model training are critical steps in developing machine learning models for electromyography (EMG) based motion decoding. Traditionally, these processes require domain expertise and programming knowledge to implement signal processing algorithms with optimized model training pipelines. In this work, we investigate the feasibility of using Large Language Models (LLMs) to automate both the extraction of features from EMG data and the development of machine learning models for decoding human motion with minimal human intervention. More specifically, we compare LLM extracted features and their corresponding motion decoding models against those developed using manually developed code. Our results indicate that LLM extracted features and their corresponding trained models achieve performance comparable to traditional methods, demonstrating the potential of accelerating research and scientific investigations with AI-driven biosignal processing. This study highlights LLMs' capabilities and limitations in replacing manual coding for developing muscle-machine interfaces and provides insights into their integration into biomedical signal analysis workflows.

With rapid advances in image-generative AI, interest has grown in applying these technologies across diverse domains. While current models excel at producing high-quality final images, they rarely model the intermediate stages of the drawing process. This study proposes a drawing support system that leverages generative AI to sequentially generate and modify images during the drawing process. To train the system, we constructed a custom dataset of time-series drawing images, which are typically unavailable in existing AI models. We developed an encoder–decoder model based on convolutional neural networks to predict the next frame from a current input image. To address error accumulation during recursive generation, we introduced a noise-augmented training method, incorporating noisy images into the dataset. Experimental results show that standard training suffers from image degradation over time, while the noise-augmented approach significantly improves stability and quality throughout the sequence. Averaged across all generated frames, the noise-augmented training achieved a PSNR of 20.48, SSIM of 0.81, and LPIPS of 0.13. As a benchmark, we compared this approach to PredRNN, a representative temporal model, which achieved a PSNR of 24.05, SSIM of 0.88, and LPIPS of 0.08. These results demonstrate the effectiveness of noise-augmented learning while also highlighting potential performance gains using temporal architectures. PredRNN also required more computation, with 6.41 M parameters and a 0.188 s inference time per sequence, compared to 5.20 M and 0.005 s for our model. Furthermore, in a sample-level analysis of 10 final images, the proposed model outperformed PredRNN in three cases across all metrics, suggesting its robustness in certain sequences despite architectural simplicity. Future directions include using more advanced models such as Variational Autoencoders and diffusion models, and enhancing consistency in long-term sequences. Our work confirms the feasibility of generating interactive drawing sequences using AI and sets the foundation for more robust and creative drawing support tools.

With the rapid development of smart cities and intelligent transportation, the level of informatization and intelligence in traffic systems is continuously improving. Studies have shown that the accident rate in foggy conditions is more than 30% higher than under normal weather. Fog significantly affects drivers' visibility, reducing safety and increasing driving risks. To address this issue, this paper proposes an improved Transformer-based model, Retinexformer, aimed at enhancing multi-object detection performance in foggy environments. First, to enhance the model's adaptability in complex environments, the Retinexformer model is optimized by incorporating a multi-branch Transformer module. This module enables the model to handle diverse input conditions and process information from different types of scenes more effectively. In addition, an adaptive parameter estimation module is designed to automatically adjust dehazing and image enhancement strategies based on the image’s brightness and contrast. By dynamically setting different dehazing parameters, this module significantly improves the robustness of Retinexformer under varying foggy conditions.In the model's Decoder section, a multi-level feature enhancement module is introduced. After the IGT (Image Gradient Transformation) output, this module fuses multi-scale features, with a particular focus on enhancing details and small object detection capabilities, ensuring accurate identification of targets such as pedestrians and vehicles.The experimental findings indicate that the improved Retinexformer model demonstrates strong robustness and superior detection accuracy in foggy multi-object detection tasks, significantly enhancing traffic safety in foggy environments.

Objective.Electroencephalography (EEG) decoding is challenging because of its temporal variability and low signal-to-noise ratio, which complicate the extraction of meaningful information from signals. Although convolutional neural networks (CNNs) effectively extract local features from EEG signals, they are constrained by restricted receptive fields. In contrast, transformers excel at capturing global dependencies through self-attention mechanisms but often require extensive training data and computational resources, which limits their efficiency on EEG datasets with limited samples.Approach.In this paper, we propose LGFormer, a hybrid network designed to efficiently learn both local and global representations for EEG decoding. LGFormer employs a deep attention module to extract global information from EEG signals, dynamically adjusting the focus of CNNs. Subsequently, LGFormer incorporates a local-enhanced transformer, combining the strengths of CNNs and transformers to achieve multiscale perception from local to global. Despite integrating multiple advanced techniques, LGFormer maintains a lightweight design and training efficiency.Main results.LGFormer achieves state-of-the-art performance within 200 training epochs across four public datasets, including motor imagery, cognitive workload, and error-related negativity decoding tasks. Additionally, we propose a novel spatial and temporal attention visualization method, revealing that LGFormer captures discriminative spatial and temporal features, enhancing model interpretability and providing insights into its decision-making process.Significance.In summary, LGFormer demonstrates superior performance while maintaining high training efficiency across different tasks, highlighting its potential as a versatile and practical model for EEG decoding.

Decoding the human brain using non-invasive methods is a significant challenge. This study aims to enhance electroencephalography (EEG) decoding by developing of machine learning methods. Specifically, we propose the novel, attention-based Patched Brain Transformer model to achieve this goal. The model exhibits flexibility regarding the number of EEG channels and recording duration, enabling effective pre-training across diverse datasets. We investigate the effect of data augmentation methods and pre-training on the training process. To gain insights into the training behavior, we incorporate an inspection of the architecture. We compare our model with state-of-the-art models and demonstrate superior performance using only a fraction of the parameters. The results are achieved with supervised pre-training, coupled with time shifts as data augmentation for multi-participant classification on motor imagery datasets.

Tropical forests are essential ecosystems recognized for their carbon sequestration and biodiversity benefits. As the world undergoes a simultaneous data revolution and climate crisis, accurate data on the world’s forests are increasingly important. Completely novel in approach, this study proposes a methodology encompassing two bespoke deep learning models: (1) a single encoder, double decoder (SEDD) model to generate a species segmentation map, regularized by a distance map in training, and (2) an XGBoost model that estimates the diameter at breast height (DBH) based on tree species and crown measurements. These models operate sequentially: RGB images from the ReforesTree dataset undergo preprocessing before species identification, followed by tree crown detection using a fine-tuned DeepForest model. Post-processing applies the XGBoost model and custom allometric equations alongside standard carbon accounting formulas to generate final sequestration estimates. Unlike previous approaches that treat individual tree identification as an isolated task, this study directly integrates species-level identification into carbon accounting. Moreover, unlike traditional carbon estimation methods that rely on regional estimations via satellite imagery, this study leverages high-resolution, drone-captured RGB imagery, offering improved accuracy without sacrificing accessibility for resource-constrained regions. The model correctly identifies 67% of trees in the dataset, with accuracy rising to 84% for the two most common species. In terms of carbon accounting, this study achieves a relative error of just 2% compared to ground-truth carbon sequestration potential across the test set.

Nowadays, grammatical error correction (GEC) has a significant role in writing since even native speakers often face challenges with proficient writing. This research is focused on developing a methodology to correct grammatical errors in the Romanian language, a less-resourced language for which there are currently no up-to-date GEC solutions. Our main contributions include an open-source synthetic dataset of 345,403 Romanian sentences, a manually curated dataset of 3054 social media comments, a two-phased GEC approach, and a comparison with several Romanian models, including RoMistral and RoLama3, but also LanguageTool, GPT-4o mini, and GPT-4o. We consider a synthetic dataset to finetune our models, while we rely on two real-life datasets with genuine human mistakes (i.e., CNA and RoComments) to evaluate performance. Building an artificial dataset was necessary because of the scarcity of real-life mistake datasets, whereas introducing RoComments, a new genuine dataset, is argued by the necessity to cover errors amongst native speakers encountered in social media comments. We also introduce a two-phased approach, where we first identify the location of erroneous tokens in the sentence; next, the erroneous tokens are replaced by an encoder–decoder model. Our approach achieved an F0.5 of 0.57 on CNA and 0.64 on RoComments, surpassing by a considerable margin LanguageTool as well as an end-to-end version based on Flan-T5 and mT0 in most setups. While our two-phased method did not outperform GPT-4o, arguably by its smaller size and language exposure, it obtained on-par results with GPT-4o mini and achieved higher performance than all Romanian LLMs.

IntroductionRecently, numerous studies have focused on the semantic decoding of perceived images based on functional magnetic resonance imaging (fMRI) activities. However, it remains unclear whether it is possible to establish relationships between brain activities and semantic features of human actions in video stimuli. Here we construct a framework for decoding action semantics by establishing relationships between brain activities and semantic features of human actions.MethodsTo effectively use a small amount of available brain activity data, our proposed method employs a pre-trained image action recognition network model based on an expanding three-dimensional (X3D) deep neural network framework (DNN). To apply brain activities to the image action recognition network, we train regression models that learn the relationship between brain activities and deep-layer image features. To improve decoding accuracy, we join by adding the nonlocal-attention mechanism module to the X3D model to capture long-range temporal and spatial dependence, proposing a multilayer perceptron (MLP) module of multi-task loss constraint to build a more accurate regression mapping approach and performing data enhancement through linear interpolation to expand the amount of data to reduce the impact of a small sample.Results and discussionOur findings indicate that the features in the X3D-DNN are biologically relevant, and capture information useful for perception. The proposed method enriches the semantic decoding model. We have also conducted several experiments with data from different subsets of brain regions known to process visual stimuli. The results suggest that semantic information for human actions is widespread across the entire visual cortex.