Real-time Gaze Estimation via Face Mesh and Machine Learning: A Case Study

Abstract

Translate Article

The quest for more intuitive human-computer interaction continuously seeks pathways to bridge the cognitive gap between human intentions and digital execution. Among these, gaze tracking stands as a beacon, promising to unlock hands-free control and enhanced user experiences. However, hardware-based eye trackers often compete with the inflated cost and impractical setups of standard hardware-based eye trackers. The present work focuses on the development of an open source gaze estimation system, creating a user-calibrated, real-time focus prediction system based on a commonly available web camera and the robust technology of Google’s MediaPipe Face Mesh pipeline. The proposed solution is based on Support Vector Regression (SVR) models with custom feature engineering that incorporates both facial landmarks and head pose dynamics. The paper details a calibration methodology designed to forge a bespoke mapping for each user, adapting to individual biometric and environmental variations. Furthermore, this work addresses practical implementation considerations, including the crucial step of feature scaling and the process of hyperparameter optimization via exhaustive grid search. The presented results, as illustrated in the accompanying scatter plots, demonstrate the SVR models’ ability to capture complex nonlinear relationships, marking a promising step towards robust and accessible gaze tracking in diverse interactive paradigms.