- Conference Article
- 10.1109/iccsnt67644.2025.11291307
Research on a VxL-based Multimodal Big Five Personality Prediction Method
- Oct 18, 2025
- Jin Zhang + 2 more +2
With the growing use of short-video and audio data in personalized psychological assessment, multimodal information provides richer feature support for Big Five personality prediction tasks. However, effective fusion and modeling of multimodal data remain challenging. This paper proposes a multimodal Big Five prediction model based on an improved Vision-Mamba visual encoder and an S6-enhanced xLSTM temporal module. In the visual branch we retain Vision-Mamba’s bidirectional state-space module for global dependency modeling and innovatively append lightweight channel-wise 1D depth-separable convolution modules after layers 1, 7, 13 and 18 to enhance local texture consistency between adjacent patches, compensating for the global model’s limited sensitivity to local details. In the temporal branch we employ xLSTM for sequence modeling and extend it by introducing a selective state-space module, which improves long-range dependency capture efficiency without increasing computational complexity. For the audio branch we extract 40-dim MFCCs, time-frequency spectral and chroma features, map them via multi-layer fully connected networks to 128-dim embeddings. Finally, a feature-level fusion strategy concatenates visual and audio representations and feeds them to a multi-layer perceptron that outputs five personality scores. Experiments on the publicly available and widely used First Impressions V2 dataset [1] show that the VxL model achieves comparable prediction accuracy while significantly outperforming traditional multimodal Big Five models in training speed and GPU memory usage, validating the effectiveness of the proposed improvements.
Read more