• Home
  • Search
  • ShouldAR: Detecting Shoulder Surfing Attacks Using Multimodal Eye Tracking and Augmented Reality
  • Cite Icon9
  • https://doi.org/10.1145/3678573Copy DOI Icon

ShouldAR: Detecting Shoulder Surfing Attacks Using Multimodal Eye Tracking and Augmented Reality

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Shoulder surfing attacks (SSAs) are a type of observation attack designed to illicitly gather sensitive data from "over the shoulder" of victims. This attack can be directed at mobile devices, desktop screens, Personal Identification Number (PIN) pads at an Automated Teller Machine (ATM), or written text. Existing solutions are generally focused on authentication techniques (e.g., logins) and are limited to specific attack scenarios (e.g., mobile devices or PIN Pads). We present ShotjldAR, a mobile and usable system to detect SSAs using multimodal eye gaze information (i.e., from both the potential attacker and victim). ShouldAR uses an augmented reality headset as a platform to incorporate user eye gaze tracking, rear-facing image collection and eye gaze analysis, and user notification of potential attacks. In a 24-participant study, we show that the prototype is capable of detecting 87.28% of SSAs against both physical and digital targets, a two-fold improvement on the baseline solution using a rear-facing mirror, a widely used solution to the SSA problem. The ShouldAR approach provides an AR-based, active SSA defense that applies to both digital and physical information entry in sensitive environments.

Similar Papers
  • Research Article
  • Citations2

비밀번호 훔쳐보기로부터 안전한 기술을 내장시킨 비밀번호 입력기의 설계 및 구현

  • Apr 30, 2010
  • The KIPS Transactions:PartD
  • Moon-Seol Kang +1
  • Book Chapter
  • Citations2

Secure and Enhanced Bank Transactions Using Biometric ATM Security System

  • Nov 07, 2019
  • A J Bhuvaneshwari +1
  • Book Chapter
  • Citations1

The Impact of Financial Security Enhancement Through One Time Password in ATM Cash Transactions

  • Apr 10, 2025
  • Nitin Bansal
  • Conference Article
  • Citations4

Expression-based Security Framework for ATM Networks

  • Dec 01, 2022
  • Sajid Ali Khan +1
  • Research Article
  • Citations42

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

  • Jul 01, 2021
  • The Journal of neuropsychiatry and clinical neurosciences
  • Wilfredo López-Ojeda +1
  • PDF
  • Research Article
  • Citations26

Evaluating authentication options for mobile health applications in younger and older adults

  • Jan 04, 2018
  • PLoS ONE
  • Kelly Grindrod +7
  • Research Article
  • Citations35

A holistic approach to augmented reality-related research in tourism: through bibliometric analysis

  • Feb 16, 2023
  • Journal of Hospitality and Tourism Insights
  • Shafeeque M Hanaa +1
  • Conference Article
  • Citations69

Towards understanding ATM security

  • Jul 14, 2010
  • Alexander De Luca +2
  • Research Article

The Surfing Attacks Secured Password Authentication System

  • Jan 05, 2020
  • INTERNATIONAL JOURNAL OF ADVANCED INFORMATION AND COMMUNICATION TECHNOLOGY
  • Arulprakash P +4
  • Book Chapter
  • Citations5

Automated Teller Machine

  • Jan 01, 2018
  • Gerard O’Regan
  • PDF
  • Research Article
  • Citations5

An Efficient Login Authentication System against Multiple Attacks in Mobile Devices

  • Jan 13, 2021
  • Symmetry
  • Yang Li +3
  • Research Article
  • Citations4

AR Smart Glasses: The Feeling of Groundedness Mediator Effect

  • Dec 05, 2024
  • International Journal of Human–Computer Interaction
  • Sara Quintal +3
  • Research Article

Evaluating the Effectiveness of Smart Glasses in Reducing Patient Care Time in Emergency Departments: Cohort Study From the Hangzhou Asian Games

  • Jun 30, 2025
  • JMIR Formative Research
  • Xinwei Jiang +37
  • Research Article

Development of a bimodal biometric authentication system for automated teller machine using gray level co-occurrence matrix

  • Dec 25, 2025
  • LAUTECH Journal of Engineering and Technology
  • Y O Mojeed, +4
  • Conference Article
  • Citations16

Advanced ATM System Using Iris Scanner

  • Mar 01, 2019
  • Indranil Banerjee +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.