• Home
  • Search
  • When Vision Fails: Text Attacks Against ViT and OCR
  • https://doi.org/10.1145/3733800.3763269Copy DOI Icon

When Vision Fails: Text Attacks Against ViT and OCR

  • Oct 13, 2025
  • Nicholas Boucher +4 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

Text-based machine learning models are vulnerable to an emerging class of Unicode-based adversarial examples capable of tricking a model into misreading text with potentially disastrous effects. The primary existing defense against these attacks is to preprocess potentially malicious text inputs using optical character recognition (OCR). In theory, OCR models will ignore any malicious Unicode characters and will extract the visually correct input to be fed to the model. In this work, we show that these visual defenses fail to prevent this type of attack. We use a genetic algorithm to generate visual adversarial examples (i.e., OCR outputs) in a black-box setting, demonstrating a highly effective novel attack that substantially reduces the accuracy of OCR and other visual models. Specifically, we use the Unicode functionality of combining characters (e.g., \(\tilde{n}\) which combines the characters n and ∼) to manipulate text inputs so that small visual perturbations appear when the text is displayed. We demonstrate the effectiveness of these attacks in the real world by creating adversarial examples against production models published by Meta, Microsoft, IBM, and Google. We additionally conduct a user study to establish that the model-fooling adversarial examples do not affect human comprehension of the text, showing that language models are uniquely vulnerable to this type of text attack.

Similar Papers
  • Research Article
  • Citations5

Smart Parking System based on Improved OCR Model

  • Nov 01, 2020
  • IOP Conference Series: Materials Science and Engineering
  • Rami Bassam +1
  • Book Chapter
  • Citations2

TrOCR Meets Language Models: An End-to-End Post-correction Approach

  • Jan 01, 2024
  • Yung-Hsin Chen +1
  • Research Article
  • Citations4

Enhancing image-based virtual try-on with Multi-Controlled Diffusion Models.

  • Sep 01, 2025
  • Neural networks : the official journal of the International Neural Network Society
  • Weihao Luo +2
  • Research Article

A comparative analysis of optical character recognition models for extracting and classifying texts in natural scenes

  • Apr 01, 2025
  • IAES International Journal of Artificial Intelligence (IJ-AI)
  • Puneeth Prakash +1
  • Research Article
  • Citations1

Development of Optical Character Recognition-Based Quality Control Process of Paper-Based Consent Forms.

  • Feb 17, 2026
  • Biopreservation and biobanking
  • Juyoung Lee +5
  • Conference Article
  • Citations6

Unsupervised OCR Model Evaluation Using GAN

  • Sep 01, 2019
  • Abhash Sinha +3
  • Research Article
  • Citations1

Artificial intelligence powered OCR models for digitizing ayurveda manuscripts

  • Oct 16, 2024
  • Journal of Ayurveda and Holistic Medicine (JAHM)
  • Haritha T Jayaraj +2
  • PDF
  • Research Article
  • Citations8

A Smart Visual Sensing Concept Involving Deep Learning for a Robust Optical Character Recognition under Hard Real-World Conditions

  • Aug 12, 2022
  • Sensors (Basel, Switzerland)
  • Kabeh Mohsenzadegan +2
  • Research Article
  • Citations5

Robust named entity detection from optical character recognition output

  • Apr 13, 2011
  • International Journal on Document Analysis and Recognition (IJDAR)
  • Krishna Subramanian +2
  • Research Article
  • Citations1

Comparative Study of Character Recognition for Handwritten Characters

  • Apr 24, 2025
  • Communications on Applied Nonlinear Analysis
  • Tarun Kumar
  • Conference Article
  • Citations3

Developing and evaluating a pipeline for Setswana OCR

  • Nov 01, 2017
  • Gideon Kotze +1
  • Conference Article
  • Citations1

Enhancing Text Recognition of Damaged Documents through Synergistic OCR and Large Language Models

  • Nov 09, 2024
  • Thomas Asselborn +4
  • Conference Article
  • Citations1

Towards a Higher Accuracy of Optical Character Recognition of Chinese Rare Books in Making Use of Text Model

  • May 08, 2019
  • Hsiang-An Wang +1
  • Research Article
  • Citations7

Systematic Literature Review of Machine Learning Models and Applications for Text Recognition

  • Jan 01, 2025
  • IEEE Access
  • Nuzhat Khan +7
  • PDF
  • Research Article
  • Citations73

Optical character recognition with neural networks and post-correction with finite state methods

  • Aug 20, 2020
  • International Journal on Document Analysis and Recognition (IJDAR)
  • Senka Drobac +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.