Current limitations in technology-based cognitive assessment for severe mental illnesses: a focus on feasibility, reliability, and ecological validity

Schizophrenia is a major cause of disability worldwide. As new treatments for functioning are tested, the need grows to demonstrate real-world functioning gains. Ecological momentary assessment (EMA) may provide a more ecologically valid measure of functioning. In this study, smartphone-based EMA was used to signal participants with schizophrenia (N = 100) and controls (N = 71) 7 times a day for 7 days to respond to brief questionnaires about social interactions and functioning behaviors. Excellent adherence was found, with both groups completing an average of 85% of surveys and only 3% of participants with schizophrenia excluded for poor adherence. Four-week test-retest reliability was high (r = .83 for total productive behaviors). Relative to controls, participants with schizophrenia reported significantly less total productive activity (d = 1.2), fewer social interactions (d = 0.3), more nonproductive behaviors (d = 1.0; watching TV, resting), and more time at home (d = 0.8). Within the schizophrenia group, participants living independently showed better functioning on EMA relative to participants in supported housing (d = 0.8) and participants engaged in vocational activities showed better functioning than individuals not engaged in vocational activities (d = 0.55). Modest correlations were found between EMA and an in-lab self-report measure of functioning activities performed in the community, but not between EMA and measures of functional capacity or potential. This study demonstrated the feasibility, sensitivity reliability, and validity of EMA methods to assess functioning in schizophrenia. EMA provides a much-needed measure of what individuals with schizophrenia are actually doing in real-world contexts. These results also suggest that there may be important disjunctions between indices of abilities and actual real-world functioning.

Introduction: Ecological Momentary Assessment (EMA) is a real-time data capture longitudinal methodology which is conducted through smartphones or wearable sensors. This methodology uses prompts to gather information on the current state, behavior and experience of a person in their natural environment. The purpose of this study is to explore the feasibility of using EMA as a methodology in measuring behavioural contexts around physical activity. Utility: EMA is advantageous in reducing recall errors, enhancing the validity of self-reports by actively recording a participant’s dynamic interaction with their environment, while accounting for intra- and inter-personal variation. EMA can provide researchers with more accurate information that is generalizable to real-life routines, and provides insight on processes that can undermine behavior change. Additionally, EMA is convenient due to the omnipresent accessibility of smartphones or related technologies, which are easy to use and can quickly collect data from large populations remotely. The use of EMA can answer researchers’ questions regarding participant current context, affective states, and psychological processes. This can ultimately help create innovative and feasible solutions which can be implemented into participant’s natural environments and daily lives to benefit their physical, mental and emotional well-being. Challenges: EMA requires smart technology equipment which can be expensive to supply, repair, or replace. Real-time prompts pose the challenge of subjects’ full compliance to prompts, struggling to respond in the case of competing activities, not carrying the device or device malfunctions such as battery drainage or software problems. Moreover, EMA raises concerns in its practicality with low-socioeconomic populations that cannot afford such technology, elderly populations who cannot operate these devices, or clinical populations whose psychopathology may interfere with their responses. Limitations: The use of EMA is associated with biases concerning ecological validity. For example, consistent prompts on a certain activity may cause an individual to think about the activity more or alter their behavior. In the absence of researchers, it is difficult to verify data reported by participants. It is possible to mitigate such biases by seeking confirmation through reliable sources who are in contact with the participants, to approve a subset of the data.

Various populations with chronic conditions are at risk for decreased cognitive performance, making assessment of their cognition important. Formal mobile cognitive assessments measure cognitive performance with greater ecological validity than traditional laboratory-based testing but add to participant task demands. Given that responding to a survey is considered a cognitively demanding task itself, information that is passively collected as a by-product of ecological momentary assessment (EMA) may be a means through which people's cognitive performance in their natural environment can be estimated when formal ambulatory cognitive assessment is not feasible. We specifically examined whether the item response times (RTs) to EMA questions (eg, mood) can serve as approximations of cognitive processing speed. This study aims to investigate whether the RTs from noncognitive EMA surveys can serve as approximate indicators of between-person (BP) differences and momentary within-person (WP) variability in cognitive processing speed. Data from a 2-week EMA study investigating the relationships among glucose, emotion, and functioning in adults with type 1 diabetes were analyzed. Validated mobile cognitive tests assessing processing speed (Symbol Search task) and sustained attention (Go-No Go task) were administered together with noncognitive EMA surveys 5 to 6 times per day via smartphones. Multilevel modeling was used to examine the reliability of EMA RTs, their convergent validity with the Symbol Search task, and their divergent validity with the Go-No Go task. Other tests of the validity of EMA RTs included the examination of their associations with age, depression, fatigue, and the time of day. Overall, in BP analyses, evidence was found supporting the reliability and convergent validity of EMA question RTs from even a single repeatedly administered EMA item as a measure of average processing speed. BP correlations between the Symbol Search task and EMA RTs ranged from 0.43 to 0.58 (P<.001). EMA RTs had significant BP associations with age (P<.001), as expected, but not with depression (P=.20) or average fatigue (P=.18). In WP analyses, the RTs to 16 slider items and all 22 EMA items (including the 16 slider items) had acceptable (>0.70) WP reliability. After correcting for unreliability in multilevel models, EMA RTs from most combinations of items showed moderate WP correlations with the Symbol Search task (ranged from 0.29 to 0.58; P<.001) and demonstrated theoretically expected relationships with momentary fatigue and the time of day. The associations between EMA RTs and the Symbol Search task were greater than those between EMA RTs and the Go-No Go task at both the BP and WP levels, providing evidence of divergent validity. Assessing the RTs to EMA items (eg, mood) may be a method of approximating people's average levels of and momentary fluctuations in processing speed without adding tasks beyond the survey questions.

BackgroundWith the increasing use of mobile technology, ecological momentary assessments (EMAs) may enable routine monitoring of patient health outcomes and patient experiences of care by health agencies. This rapid review aims to synthesise the evidence on the use of EMAs to monitor health outcomes after traumatic unintentional injury.MethodA rapid systematic review of nine databases (MEDLINE, Web of Science, Embase, CINAHL, Academic Search Premier, PsychINFO, Psychology and Behavioural Sciences Collection, Scopus, SportDiscus) for English-language articles from January 2010–September 2021 was conducted. s and full-text were screened by two reviewers and each article critically appraised. Key information was extracted by population characteristics, age and sample size, follow-up time period(s), type of EMA tools, physical health or pain outcome(s), psychological health outcome(s), general health or social outcome(s), and facilitators or barriers of EMA methods. Narrative synthesis was undertaken to identify key EMA facilitator and barrier themes.ResultsThere were 29 articles using data from 25 unique studies. Almost all (84.0%) were prospective cohort studies and 11 (44.0%) were EMA feasibility trials with an injured cohort. Traumatic and acquired brain injuries and concussion (64.0%) were the most common injuries examined. The most common EMA type was interval (40.0%). There were 10 key facilitator themes (e.g. feasibility, ecological validity, compliance) and 10 key barrier themes (e.g. complex technology, response consistency, ability to capture a participant’s full experience, compliance decline) identified in studies using EMA to examine health outcomes post-injury.ConclusionsThis review highlighted the usefulness of EMA to capture ecologically valid participant responses of their experiences post-injury. EMAs have the potential to assist in routine follow-up of the health outcomes of patients post-injury and their use should be further explored.

BackgroundWe present a digital phenotyping protocol designed to continuously and objectively measure behavioral, physiological, and contextual data during pregnancy and the postpartum period using passive sensing from Garmin smartwatches and smartphones, along with active ecological momentary assessments (EMAs). This novel protocol uniquely adapts to the unpredictable timing of childbirth, spanning from the third trimester through 6 weeks post partum, to accurately capture critical temporal changes and maternal-infant outcomes. By providing high-frequency real-time data, this methodology offers comprehensive insights into pregnancy-related behaviors and physiological processes, overcoming the limitations of traditional retrospective self-report methods.ObjectiveWe aim to develop a protocol for longitudinal data collection supporting digital phenotyping that is optimized for pregnancy and the postpartum period. This protocol leverages the pregnant population’s heightened interest in health and tracking. It aims to minimize the burden on the participants, increase retention, and assess the value of wearables compared to smartphones to determine the appropriate data collection methods.MethodsData will be collected from 30 nulliparous participants from the start of the third trimester through 6 weeks post partum. This protocol uses 3 distinct 1-time surveys, alongside daily and weekly EMAs, to capture real-time maternal experience data. Passive maternal data—such as activity, vitals, sleep, and location—are collected via smartphones and Garmin smartwatches. Participants are expected to log data about the newborn after delivery through the mobile app Huckleberry. This protocol was developed in collaboration with the Northeastern University Sath Laboratory, which focuses on digital phenotyping and longitudinal data collection, and the Tufts Medical Center’s obstetrics and gynecology department, which has expertise in working with the pregnant population.ResultsThis study was funded in August 2024. Data collection is projected to run from October 2025 to July 2026. As of September 2025, the study has been approved, and recruitment and data collection are to begin. The results are expected to be published by August 2026. We plan to assess the retention rates, survey and EMA completion rates, wear time of the smartwatch without intervention, and data volume logged in the Huckleberry app. In addition, we will perform digital phenotyping to determine whether the data collected during pregnancy can be used to predict breastfeeding outcomes, delivery outcomes, and maternal-infant well-being.ConclusionsThis protocol integrates the use of digital phenotyping in pregnancy and postpartum research, providing a novel method for capturing real-time indicators of maternal well-being. It will determine the expected rates of data completion and appropriate sample size using a power analysis for a more extensive future study. By integrating smartphone and wearable sensor data, this protocol has the potential to transform the way maternal health clinical interventions are designed and implemented in the future.International Registered Report Identifier (IRRID)PRR1-10.2196/77175

BACKGROUND We present a digital phenotyping protocol designed to continuously and objectively measure behavioral, physiological, and contextual data during pregnancy and postpartum periods using passive sensing from Garmin smartwatches and smartphones, along with active ecological momentary assessments (EMAs). This novel protocol uniquely adapts to the unpredictable timing of childbirth, spanning from the third trimester through six weeks postpartum, to accurately capture critical temporal changes and maternal-infant outcomes. By providing high-frequency real-time data, this methodology offers comprehensive insights into pregnancy-related behaviors and physiological processes, overcoming limitations of traditional retrospective self-report methods. OBJECTIVE The objective is to develop a protocol for longitudinal data collection supporting digital phenotyping that is optimized for pregnancy and postpartum. This protocol leverages the pregnant population’s heightened interest in health and tracking with over 50% using mobile health apps [1]. This protocol aims to minimize burden on the participants, increase retention, and assess the value of wearables compared to smartphones to determine appropriate data collection methods. METHODS Data will be collected on 30 nulliparous participants from the start of the third trimester through 6 weeks postpartum. This protocol utilizes three distinct one-time surveys, alongside daily and weekly Ecological Momentary Assessments (EMA), to capture real-time maternal experience data. Passive maternal data - such as activity, vitals, sleep, location - are collected via smartphone and Garmin smartwatch. Participants are expected to log data about the newborn after delivery through the mobile application Huckleberry. This protocol was developed in collaboration between the Northeastern University SATH Lab who focus on digital phenotyping and longitudinal data collection and Tufts Medical Center Obstetrics and Gynecology who have expertise working with the pregnant population. RESULTS The planned completion date is December 2026, with a manuscript published afterward. We plan to assess retention rates, survey and EMA completion rates, track wear time of smartwatch without intervention, and data volume logged in Huckleberry. CONCLUSIONS This protocol integrates the use of digital phenotyping in pregnancy and postpartum research, providing a novel method for capturing real-time maternal well-being indicators. It will determine expected rates of data completion and appropriate sample size using a power analysis for a more extensive future study. By integrating smartphone and wearable sensor data, this protocol has the potential to transform the way maternal health clinical interventions are designed and implemented in the future.

BackgroundIncreasing attention is being paid to lifestyle factors, such as nutrition and physical activity (PA), as potential complementary treatment options in attention-deficit/hyperactivity disorder (ADHD). Previous research indicates that sugar and saturated fat intake may be linked to increased impulsivity, a core symptom of ADHD, whereas protein intake and PA may be related to reduced impulsivity. However, most studies rely on cross-sectional data that lack microtemporal resolution and ecological validity, wherefore questions of microtemporal dynamics (eg, is the consumption of foods high in sugar associated with increased impulsivity within minutes or hours?) remain largely unanswered. Ecological momentary assessment (EMA) has the potential to bridge this gap.ObjectiveThis study is the first to apply EMA to assess microtemporal associations among macronutrient intake, PA, and state impulsivity in the daily life of adults with and without ADHD.MethodsOver a 3-day period, participants reported state impulsivity 8 times per day (signal-contingent), recorded food and drink intake (event-contingent), and wore an accelerometer. Multilevel 2-part models were used to study the association among macronutrient intake, PA, and the probability to be impulsive as well as the intensity of impulsivity (ADHD: n=36; control: n=137).ResultsNo association between macronutrient intake and state impulsivity was found. PA was not related to the intensity of impulsivity but to a higher probability to be impulsive (ADHD: β=−.09, 95% CI −0.14 to −0.04; control: β=−.03, 95% CI −0.05 to −0.01). No evidence was found that the combined intake of saturated fat and sugar amplified the increase in state impulsivity and that PA alleviated the positive association between sugar or fat intake and state impulsivity.ConclusionsImportant methodological considerations are discussed that can contribute to the optimization of future EMA protocols. EMA research in the emerging field of nutritional psychiatry is still in its infancy; however, EMA is a highly promising and innovative approach as it offers insights into the microtemporal dynamics of psychiatric symptomology, dietary intake, and PA in daily life.

Digital phenotyping using wearable devices and ecological momentary assessment (EMA) enables continuous, real-world monitoring of physiological and emotional states, but identifying high-risk stress states in real time remains challenging. We examined day-level associations between emotional distress and heart rate variability (HRV), and assessed whether daily physical activity modifies this relationship using longitudinal wearable and EMA data. The Smart Momentary Interactive Longitudinal Evaluation Study (SMILES) was a prospective cohort study conducted among STEM graduate students in the U.S. in 2025. Participants wore an Oura Ring Generation 3 continuously for five months and completed daily EMA surveys assessing emotional distress. The primary outcome was nightly HRV measured as the root mean square of successive differences and log-transformed for analysis. Quantile regression within a quadratic inference function framework was used to estimate associations at the 25th, 50th, and 75th percentiles of HRV, accounting for within-participant correlation and time-varying covariates. Thirty-one participants contributed 1,724 person-days of observation. High emotional distress was associated with lower HRV across the HRV distribution, with the strongest association observed at the lower HRV quantile (β = -0.094, 95\% CI: [-0.111, -0.078]). A significant interaction between daily step count and emotional distress was observed across quantiles, such that higher physical activity was associated with higher HRV on high emotional distress days but not on low-to-moderate distress days. Integration of wearable-derived physiological data with EMA enables real-time identification of high-risk stress states in naturalistic settings. The observed buffering effect of physical activity during periods of elevated emotional distress suggests that wearable-guided, personalized just-in-time adaptive interventions, such as physical activity prompts, could be deployed to improve autonomic regulation and mental health.

To describe the diverse methodologies for researching daily life a variety of terms have been coined: ecological momentary assessment (EMA), ambulatory assessment, experience sampling method (ESM), real-time data capture, or digital phenotyping, just to name a few. In line with the definition of the international society devoted to researching daily life Society for Ambulatory Assessment (SAA), these various terms serve to highlight the different origins and ancestors. Researchers in the Netherlands, Germany, and the United States started developing cutting-edge methods to assess dynamics of behavior and experience in everyday life in the 1970s and 1980s. Although the diverse backgrounds can be clearly differentiated, with Ambulatory Assessment having a focus on combining e-diaries with physiological and behavioral monitoring—ESM, by using paper-pencil diaries and pagers, and EMA, by utilizing e-diaries early on—these research groups coined their terms to describe a broad set of tools to assess affective experiences, cognition, behavior, and physiological processes in daily life. Accordingly, the Society for Ambulatory Assessment introduced “ambulatory assessment” as the umbrella term, covering a range of real-time data capture methodologies that originate from different scientific disciplines (psychology, medicine, computer science, etc.), and it focuses on the common goal, namely, to assess the ongoing behavior, physiology, experience, and environmental aspects of people in naturalistic settings. Distinct features of ambulatory assessment, which differentiates it from traditional assessment approaches like retrospective questionnaires or laboratory-based techniques, include: (1) assessment of data in the real world, increasing the ecological validity and generalizability of the method, (2) focus on individuals’ momentary or very recent states to minimize retrospective biases, (3) ability to study within-subject processes and dynamics, (4) possibility for multimodal and context-specific assessments, and (5) use of real-time analyses to trigger situational assessments and interventions.

Evaluating psychometric validity of ecological momentary assessment for youth irritability.

Assessing the risk of suicidal outcomes is challenging, particularly in older people. Smartphone-based digital phenotyping may help to monitor suicide risk through ecological momentary assessment (EMA) applications. In this real-world study, we investigated if age and other clinical factors were associated with participation in EMA at baseline, and with retention in EMA monitoring among patients at risk of suicide. Participation in EMA was determined by quantifying the installation of the MEmind mobile application in individuals involved in the SmartCrisis 1.0 and 2.0 studies. The patients were followed-up over a 6-month period. N = 512 patients met inclusion criteria, of which 387 installed the MEmind application on their smartphone. While age as a continuous variable was not associated with using EMA at baseline, being aged older than 50 and being engaged in an intimate relationship were independently associated with longer participation in EMA (OR 2.070, 95%CI [1.054-4.066], and OR 2.103, 95%CI [1.076-4.110], respectively). In an exploratory survival analysis, we found that EMA retention increased with age (p < 0.001). Feasibility of EMA seems warranted in older people at risk of suicide. Clinicians should be encouraged to offer EMA monitoring to older adults, as they commonly face limitations in their access to healthcare facilities.

IntroductionVestibular disorders have a high prevalence and impact on general health. Currently questionnaires to measure vestibular symptoms and the impact on daily life, do not capture disease-specific symptoms or induce recall bias. The DizzyQuest was introduced, to measure vestibular symptoms in the flow of daily life, using the ecological momentary assessment (EMA) strategy. The development and content validity of the DizzyQuest are described in this article.MethodsThe development process consisted of two stages: First, the development of a concept version of the DizzyQuest based on the conceptual model of positive health, semi-structured interviews with patients diagnosed with bilateral vestibulopathy (BV) and expert input. Second, the assessment of content validity through a 4-week pilot trial, followed by focus group meetings with patients and professionals separately.ResultsPatient interviews, an expert meeting and a conceptual model resulted in the first pilot version of the DizzyQuest, which included a morning-, within-day-, evening- and attack-questionnaire. To report patient characteristics, complimentary patient- and doctor-questionnaires were added. The pilot trial (N = 5) and two patient focus group meetings resulted in multiple suggestions for response options. Two professional focus group meetings provided recommendations for sampling time and sampling schedule. As a result, two versions of the DizzyQuest were developed, consisting of a regular DizzyQuest (DQ) and an extended DizzyQuest-XL (DQ-XL).ConclusionThe DizzyQuest enables the administration of multiple questionnaires during the flow of daily life, therefore increasing reliability and ecological validity. The DQ and DQ-XL, using a big data approach, has the potential to improve precision medicine.

BackgroundMood disorders, including bipolar disorder (BP) and major depressive disorder (MDD), are characterized by significant psychological and behavioral fluctuations, with mobility patterns serving as potential markers of emotional states.ObjectiveThis study explores the diagnostic and monitoring capabilities of Fourier transform, a frequency-domain analysis method, in mood disorders by leveraging GPS data as an objective measure.MethodsA total of 62 participants (BP: n=20, MDD: n=27, and healthy controls: n=15) contributed 5177 person-days of data over observation periods ranging from 5 days to 6 months. Key GPS indicators—location variance (LV), transition time (TT), and entropy—were identified as reflective of mood fluctuations and diagnostic differences between BP and MDD.ResultsFourier transform analysis revealed that the maximum power spectra of LV and entropy differed significantly between BP and MDD groups, with patients with BP exhibiting greater periodicity and intensity in mobility patterns. Notably, participants with BP demonstrated consistent periodic waves (eg, 1-d, 4-d, and 9-d cycles), while such patterns were absent in those with MDD. In addition, after adjusting for age, gender, and employment status, only the power spectrum of LV remained a significant predictor of depressed mood (odds ratio [OR] 0.9976, 95% CI 0.9956‐0.9996; P=.02). Daily GPS data showed stronger correlations with ecological momentary assessment (EMA)–reported mood states compared to weekly or monthly aggregations, emphasizing the importance of day-to-day monitoring. Depressive states were associated with reduced LV (OR 0.975, 95% CI 0.957‐0.993; P=.008) and TT (OR 0.048, 95% CI 0.012‐0.200; P<.001) on weekdays, and lower entropy (OR 0.662, 95% CI 0.520‐0.842; P=.001) on weekends, indicating that mobility features vary with social and temporal contexts.ConclusionsThis study underscores the potential of GPS-derived mobility data, analyzed through Fourier transform, as a noninvasive and real-time diagnostic and monitoring tool for mood disorders. The findings suggest that the intensity of mobility patterns, rather than their frequency, may better differentiate BP from MDD. Integrating GPS data with EMAs could enhance the precision of clinical assessments, provide early warnings for mood episodes, and support personalized interventions, ultimately improving mental health outcomes. This approach represents a promising step toward digital phenotyping and advanced mental health monitoring strategies.

Mobile device-assisted dietary ecological momentary assessments (EMAs) have emerged as a new tool allowing the evaluation of dietary intake in real time, in a real-world setting and in a continuous manner. They have the potential to minimize recall bias, participant, and investigator burden, and decrease economic and time investment while maximizing ecological validity. We developed a set of EMAs aimed at evaluating continuous adherence to the MedDiet. Four multiple-choice EMAs are sent daily in a randomized manner from a total of eight questions. The EMAs enquire about the consumption of 11 key food groups of the Mediterranean diet in the last 24-48 h in a semi-quantitative way. EMAs capture the daily frequency of consumption of fruits, vegetables, and extra virgin olive oil on different days of the week. Additionally, EMAs capture the weekly frequency of consumption of whole grain products, sugary drinks, nuts, legumes, sweets, fish and seafood, and red and processed meats. A designed scoring system behind the EMAs extracts the percentage of adherence to the MedDiet recommendations and calculates a quality index of the diet every week. Individualized reports are sent periodically to the volunteers highlighting the strengths and weaknesses of their diet. EMAs are also expected to have a behavioral effect, reinforcing the choice of Mediterranean foods.

Digital technologies are reshaping human behavior, health care delivery, and population health; however, their cumulative effects across the lifespan remain underexplored. This viewpoint argues that exposures arising from interactions with digital technologies should be formally integrated into exposome science as a distinct, measurable component of the human environment. Our aims are to (1) redefine the digital component of the exposome (the digital exposome) within the broader exposome framework, (2) examine its life course implications for health and equity, and (3) outline a research and policy agenda to enable its systematic measurement and integration into clinical and public health practice. Digital technology-related exposures can confer benefits such as enhanced health monitoring, personalized interventions, improved access to care, and the promotion of healthy behaviors. However, they may also introduce potential risks, including mental health challenges, cognitive and circadian disruptions, sedentary lifestyles, exposure to misinformation, and widening inequities among vulnerable populations. Despite their ubiquity, digital technology-related exposures remain poorly integrated into clinical medicine, epidemiology, or public and global health policies. Drawing on interdisciplinary evidence from exposure science, epidemiology, and digital phenotyping research, we propose a refined conceptual definition of the digital exposome grounded in the classical exposome domains. We propose redefining the digital exposome as the full spectrum of exposures resulting from interactions or proximities with digital technologies and their combined influence on health across the lifespan. This framework conceptualizes digital technology-related exposures as a dynamic set of environmental influences operating through sociotechnical, behavioral, and biological pathways over the life course. To operationalize this framework, we discuss practical approaches using validated behavioral instruments, objective device use logs, ecological momentary assessments, smartphone-based digital phenotyping, and wearable sensing technologies. Systematic measurement, large-scale longitudinal studies, and harmonized exposure metrics are needed to characterize the cumulative health impacts of digital environments more accurately. Emerging tools such as digital markers or biomarkers and digital phenotypes offer promising opportunities to link real-world technology use with physiological and biological outcomes, thereby supporting precision medicine and population health strategies. Ethical governance, privacy safeguards, and equity considerations must be embedded from the start, drawing on emerging exposomethics frameworks. Recognizing the digital exposome as a modifiable determinant of health offers a foundation for evidence-based guidance, prevention strategies, and policy interventions suited to increasingly digital societies. By integrating digital technology-related exposures into exposome science, clinical practice, and public health research, this viewpoint seeks to foster interdisciplinary dialogue, guide future empirical work, and support the development of safer and more equitable digital environments across the lifespan.