• Home
  • Search
  • WavFace: A Multimodal Transformer-Based Model for Depression Screening.
  • Cite Icon15
  • https://doi.org/10.1109/jbhi.2025.3529348Copy DOI Icon

WavFace: A Multimodal Transformer-Based Model for Depression Screening.

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Depression, a prevalent mental health disorder with severe health and economic consequences, can be costly and difficult to detect. To alleviate this burden, recent research has been exploring the depression screening capabilities of deep learning (DL) models trained on videos of clinical interviews conducted by a virtual agent. Such DL models need to consider the challenges of modality representation, alignment, and fusion as well as small sample sizes. To address them, we propose WavFace, a multimodal deep learning model that inputs audio and temporal facial features. WavFace adds an encoder-transformer layer over pre-trained models to improve the unimodal representation. It also applies an explicit alignment method for both modalities and then uses sequential and spatial self-attention over the alignment. Finally, WavFace fuses the sequential and spatial self-attentions among the two modality embeddings, inspired by how mental health professionals simultaneously observe visual and vocal presentation during clinical interviews. By leveraging sequential and spatial self-attention, WavFace outperforms pre-trained unimodal and multimodal models from the literature. With a single interview question, WaveFace screened for depression with a balanced accuracy of 0.81. This presents a valuable modeling approach for audio-visual mental health screening.

Similar Papers
  • Research Article
  • Citations14

Predicting microvascular invasion in hepatocellular carcinoma with a CT- and MRI-based multimodal deep learning model.

  • Mar 03, 2024
  • Abdominal radiology (New York)
  • Yan Lei +10
  • Preprint Article

Application of multimodal deep learning using radar and water level data for water level prediction

  • May 15, 2023
  • Seongsim Yoon +2
  • Research Article
  • Citations24

Predicting Obstructive Sleep Apnea Based on Computed Tomography Scans Using Deep Learning Models.

  • Jul 15, 2024
  • American journal of respiratory and critical care medicine
  • Jeong-Whun Kim +7
  • Research Article
  • Citations22

Multimodal deep learning model on interim [18F]FDG PET/CT for predicting primary treatment failure in diffuse large B-cell lymphoma.

  • Aug 27, 2022
  • European Radiology
  • Cheng Yuan +7
  • Research Article

Development and validation of a multimodal deep learning model for the radiographic classification of pediatric femoral neck fractures.

  • Jul 11, 2026
  • Orthopaedics & traumatology, surgery & research : OTSR
  • Wentao Wang +13
  • Research Article
  • Citations6

Multimodal deep learning model for prediction of breast cancer recurrence risk and correlation with oncotype DX

  • Jan 01, 2025
  • Breast Cancer Research : BCR
  • Ruixin Zhang +7
  • Research Article
  • Citations1

Deep multi-instance learning model based on gadoxetic acid-enhanced MRI for predicting microvascular invasion of hepatocellular carcinoma: a multicenter, retrospective study

  • Oct 22, 2025
  • BMC Cancer
  • Yi Luo +7
  • Research Article
  • Citations6

Multi-modal deep learning for automated assembly of periapical radiographs

  • Jun 21, 2023
  • Journal of Dentistry
  • L Pfänder +5
  • Abstract

Development of a deep learning model using multimodal data for dementia diagnosis

  • Dec 01, 2025
  • Alzheimer's & Dementia
  • Hee Won Yang +2
  • Research Article
  • Citations7

Multicenter Development and Validation of a Multimodal Deep Learning Model to Predict Moderate to Severe AKI.

  • Apr 15, 2025
  • Clinical journal of the American Society of Nephrology : CJASN
  • Jay L Koyner +8
  • Research Article
  • Citations11

PSMA PET/CT based multimodal deep learning model for accurate prediction of pelvic lymph-node metastases in prostate cancer patients identified as candidates for extended pelvic lymph node dissection by preoperative nomograms.

  • Jan 27, 2025
  • European journal of nuclear medicine and molecular imaging
  • Qiaoke Ma +10
  • PDF
  • Research Article
  • Citations32

Accessory pathway analysis using a multimodal deep learning model

  • Apr 13, 2021
  • Scientific Reports
  • Makoto Nishimori +17
  • Research Article
  • Citations3

Deep learning-based multimodal integration of imaging and clinical data for predicting surgical approach in percutaneous transforaminal endoscopic discectomy.

  • Feb 08, 2025
  • European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society
  • Yefu Xu +6
  • PDF
  • Research Article
  • Citations3

AI in Stock Market Forecasting: A Bibliometric Analysis

  • Jan 01, 2024
  • SHS Web of Conferences
  • Hong N.Dao +5
  • Research Article

Multimodal Artificial Intelligence for Predicting 3- and 5-Year Risks of Myopic Choroidal Neovascularization in High Myopia.

  • Feb 01, 2026
  • Ophthalmology. Retina
  • Yining Wang +6
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.