Speech is a signal rich in information about cognitive and affective states, which can be of high clinical utility for detecting and monitoring mental health conditions. Numerous studies have employed natural language processing (NLP) and AI-based language models to derive potential psychological and neurocognitive insights from spontaneous speech as a marker. However, only few studies have investigated the test-retest reliability of commonly used features, a basic psychometric property crucial to clinical applications. In the present study, we use a crowdsourcing approach to test the reliability of a comprehensive set of speech- and language markers across three speech elicitation tasks (free speech, picture and cartoon descriptions) and four time points, using the intra-class correlation coefficient (ICC). We also explore the underlying factor structure of the feature space through an exploratory factor analysis (EFA). Results indicate that acoustic-prosodic features exhibit high test-retest reliability across all sessions. In contrast, semantic measures (e.g., semantic similarity, information density, and perplexity), speech quantity metrics, and syntactic complexity, exhibit low reliability, even when stimulus materials used for speech elicitation were kept identical. Although semantic features showed strong within-subject variability, EFA across the feature space revealed a latent factor specifically comprising BERT-based semantic features with a moderate-to-high ICC of 0.76. There was some limited evidence of free speech showing lower ICCs across tasks. Demographic, emotional and physical state factors contributed negligibly to ICC variance, indicating that these external factors had minimal impact on the consistency of the acoustic and semantic features. Overall, we find that acoustic-prosodic and text-based features have crucially different psychometric properties, with the latter showing low test-retest reliability individually, though semantic features form intercorrelated clusters that are more stable and capture significant aspects of the variance.

We present UGR-MINDVOICE, the University of Granada (UGR) multimodal electroencephalography (EEG) and audio dataset for overt and covert speech in Iberian Spanish intended for basic neuroscience and brain-computer interface (BCI) research. The dataset features EEG and audio recordings from 15 native Spanish speakers engaged in both overt and covert speech production tasks. This dataset is unique in its inclusion of all Spanish phonemes and a diverse set of words spanning various semantic categories and different usage frequencies. Validation of the dataset confirmed the presence of robust sensory event-related potentials, including the visual P100 and the auditory N1 (N100), indicating reliable early perceptual processing and sustained participant attention to both visual and auditory stimuli. Additionally, the EEG data were classified into rest, covert speech, and overt speech conditions with an accuracy of 81.40%, demonstrating active participant engagement in the tasks. By providing synchronised EEG and audio data for overt speech, along with EEG data for the same stimuli during covert speech, UGR-MINDVOICE constitutes a valuable resource for advancing research in basic neuroscience and brain-computer interfaces, particularly in the domain of silent speech communication. The full dataset is openly available on the Open Science Framework (OSF) ( https://osf.io/6sh5d ), and all accompanying code and analysis scripts are provided in a public GitHub repository ( https://github.com/owaismujtaba/mind-voice ). • UGR-MINDVOICE is the first open-access EEG-audio dataset designed specifically for Iberian Spanish, capturing both overt and covert speech across all Spanish phonemes and a wide range of lexical items. • The dataset supports the development of brain-computer interfaces for silent speech decoding, offering a non-invasive alternative to intracranial methods and enabling communication for individuals with severe motor impairments. • With high-density EEG, multimodal stimuli (text, audio, and images), and carefully controlled tasks, UGR-MINDVOICE enables robust modelling of speech production and perception.