• Home
  • Search
  • Exploring the Potential of Electroencephalography Signal–Based Image Generation Using Diffusion Models: Integrative Framework Combining Mixed Methods and Multimodal Analysis
  • https://doi.org/10.2196/72027Copy DOI Icon

Exploring the Potential of Electroencephalography Signal–Based Image Generation Using Diffusion Models: Integrative Framework Combining Mixed Methods and Multimodal Analysis

Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

BackgroundElectroencephalography (EEG) has been widely used to measure brain activity, but its potential to generate accurate images from neural signals remains a challenge. Most EEG-decoding research has focused on tasks such as motor imagery, emotion recognition, and brain wave classification, which involve EEG signal analysis and classification. Some studies have explored the correlation between EEG and images, primarily focusing on EEG-image pair classification or transformation. However, EEG-based image generation remains underexplored.ObjectiveThe primary goal of this study was to extend EEG-based classification to image generation, addressing the limitations of previous methods and unlocking the full potential of EEG for image synthesis. To achieve more meaningful EEG-to-image generation, we developed a novel framework, Neural-Cognitive Multimodal EEG-Informed Image (NECOMIMI), which was specifically designed to generate images directly from EEG signals.MethodsWe developed a 2-stage NECOMIMI method, which integrated the novel Neural Encoding Representation Vectorizer (NERV) EEG encoder that we designed with a diffusion-based generative model. The Category-Based Assessment Table (CAT) score was introduced to evaluate the semantic quality of EEG-generated images. In addition, the ThingsEEG dataset was used to validate and benchmark the CAT score, providing a standardized measure for assessing EEG-to-image generation performance.ResultsThe NERV EEG encoder achieved state-of-the-art performance in several zero-shot classification tasks, with an average accuracy of 94.8% (SD 1.7%) in the 2-way task and 86.8% (SD 3.4%) in the 4-way task, outperforming models such as Natural Image Contrast EEG, Multimodal Similarity-Keeping Contrastive Learning, and Adaptive Thinking Mapper ShallowNet. This highlighted its superiority as a feature extraction tool for EEG signals. In a 1-stage image generation framework, EEG embeddings often resulted in abstract or generalized images such as landscapes instead of specific objects. Our proposed 2-stage NECOMIMI architecture effectively extracted semantic information from noisy EEG signals, showing its ability to capture and represent underlying concepts derived from brain wave activity. We further conducted a perturbation study to test whether the model overly depended on visual cortex EEG signals for scene-based image generation. The perturbation of visual cortex EEG channels led to a notable increase in Fréchet inception distance scores, suggesting that our model relied heavily on posterior brain signals to generate semantically coherent images.ConclusionsNECOMIMI demonstrated the potential of EEG-to-image generation, revealing the challenges of translating noisy EEG data into accurate visual representations. The novel NERV EEG encoder for multimodal contrastive learning reached state-of-the-art performance both on n-way zero-shot and EEG-informed image generation. The introduction of the CAT score provided a new evaluation metric, paving the way for future research to refine generative models. In addition, this study highlighted the significant clinical potential of EEG-to-image generation, particularly in enhancing brain-machine interface systems and improving quality of life for individuals with motor impairments.

Similar Papers
  • PDF
  • Research Article
  • Citations8

Developing Innovative Feature Extraction Techniques from the Emotion Recognition Field on Motor Imagery Using Brain–Computer Interface EEG Signals

  • Dec 04, 2024
  • Applied Sciences
  • Amr F Mohamed +1
  • Research Article
  • Citations2

MSAttNet: Multi-scale attention convolutional neural network for motor imagery classification.

  • Dec 01, 2025
  • Journal of neuroscience methods
  • Ruiyu Zhao +7
  • Research Article
  • Citations42

EEG-based motor imagery classification using convolutional neural networks with local reparameterization trick

  • Sep 27, 2021
  • Expert Systems with Applications
  • Wenqie Huang +5
  • Research Article

EEG Signal Classification for Multitasking Motor Imagery Using Multi-Layer Time-Varying Functional Brain Network Features

  • Jan 01, 2024
  • Journal of Electronics and Information Science
  • Hongzhen Ruan
  • Conference Article
  • Citations29

Removal of EMG artifacts from single channel EEG signal using singular spectrum analysis

  • Sep 01, 2015
  • Ajay Kumar Maddirala +1
  • Research Article

Analysis of the Relationship between Brain Waves and Learning Readiness of Students with Disabilities Using Electroencephalography (EEG) Signals

  • Jun 25, 2025
  • Jurnal Penelitian Pendidikan IPA
  • Tining Haryanti +2
  • Research Article
  • Citations2

TFANet: a temporal fusion attention neural network for motor imagery decoding

  • Sep 22, 2025
  • Frontiers in Neuroscience
  • Chao Zhang +2
  • Research Article
  • Citations164

A Channel-Fused Dense Convolutional Network for EEG-Based Emotion Recognition

  • Dec 01, 2021
  • IEEE Transactions on Cognitive and Developmental Systems
  • Zhongke Gao +5
  • Research Article
  • Citations38

Convolutional neural network based features for motor imagery EEG signals classification in brain–computer interface system

  • Mar 05, 2020
  • SN Applied Sciences
  • Samaneh Taheri +2
  • Research Article
  • Citations3

Two Class Motor Imagery EEG Signal Classification for BCI Using LDA and SVM

  • Oct 31, 2024
  • Traitement du Signal
  • Venkatesh Kanagaluru +1
  • PDF
  • Conference Article
  • Citations3

Optimal Sensor Set for Decoding Motor Imagery from EEG

  • Apr 24, 2023
  • Arnau Dillen +6
  • Research Article
  • Citations4

Neural Network Classification of Brainwave Alpha Signals in Cognitive Activities

  • Dec 23, 2019
  • Knowledge Engineering and Data Science
  • Ahmad Azhari +3
  • PDF
  • Research Article
  • Citations176

A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification

  • Nov 26, 2019
  • Frontiers in Neuroscience
  • Hao Wu +6
  • PDF
  • Research Article
  • Citations3

Experimental design and signal selection for construction of a robot control system based on EEG signals

  • Nov 26, 2014
  • Robotics and Biomimetics
  • Masataka Yoshioka +6
  • Research Article

Decoding EEG Signals for Brain-Computer Interfaces.

  • Oct 03, 2025
  • Studies in health technology and informatics
  • Hamza Amrani +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.