- Research Article
- 10.1109/taffc.2026.3672547
Exploring Spontaneous Facial Micro-expressions in On-road Driver Behavior
- Jan 01, 2026
- IEEE Transactions on Affective Computing
- Xinqi Fan + 4 more +4
The development of driver monitoring systems (DMS) has emerged as a pivotal advancement in automotive safety to monitor driver behaviors and prevent road accidents. Driver emotions are one of the critical factors that affect driving safety. While numerous studies have explored facial macro-expressions (FMaEs) of drivers, these facial expressions can be consciously controlled, allowing individuals to conceal their true negative emotions. However, inaccurate capturing of true emotions can lead to severe accidents. In this paper, we focus on facial micro-expressions (FMEs), which are brief and involuntary facial movements that reveal deeper insights into genuine emotions. Thereby, they provide a more accurate measure of a driver's emotional condition, and are crucial for assessing driver readiness and safety. To facilitate FME studies of drivers, we analyzed driver videos from an affective driver-pedestrian interaction experiment, providing corresponding facial action units and FME annotations, and built a real driver FME dataset. By analyzing the driver data, we found that there are more FMEs when drivers respond to negative stimuli compared with positive stimuli, and there are more FMaEs when drivers respond to positive stimuli compared with negative stimuli. To protect driver identities while maintaining the integrity of the facial movements, we released a synthesized driver FME dataset. Furthermore, we benchmarked deep learning-based architectures on the real and synthetic driver FME datasets, uncovering the challenges inherent in recognizing FMEs in real-world driving scenarios.
Read more