• Home
  • Search
  • MixGen: A New Multi-Modal Data Augmentation
  • Open Access IconOpen Access
  • Cite Icon95
  • https://doi.org/10.1109/wacvw58289.2023.00042Copy DOI Icon

MixGen: A New Multi-Modal Data Augmentation

  • Jan 1, 2023
  • Xiaoshuai Hao +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Data augmentation is a necessity to enhance data efficiency in deep learning. For vision-language pre-training, data is only augmented either for images or for text in previous works. In this paper, we present MixGen: a joint data augmentation for vision-language representation learning to further improve data efficiency. It generates new image-text pairs with semantic relationships preserved by interpolating images and concatenating text. It's simple, and can be plug-and-played into existing pipelines. We evaluate MixGen on four architectures, including CLIP, ViLT, ALBEF and TCL, across five downstream vision-language tasks to show its versatility and effectiveness. For example, adding MixGen in ALBEF pre-training leads to absolute performance improvements on downstream tasks: image-text retrieval (+6.2% on COCO fine-tuned and +5.3% on Flicker30K zero-shot), visual grounding (+0.9% on Re-fCOCO+), visual reasoning (+0.9% on NLVR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), visual question answering (+0.3% on VQA2.0), and visual entail-ment (+0.4% on SNLI-VE).

Similar Papers
  • Conference Article
  • Citations9

MAMO: Fine-Grained Vision-Language Representations Learning with Masked Multimodal Modeling

  • Jul 18, 2023
  • Zijia Zhao +5
  • PDF
  • Research Article
  • Citations3

Clustering swap prediction for image-text pre-training

  • May 24, 2024
  • Scientific Reports
  • Sun Fayou +3
  • Conference Article
  • Citations6

Semi-supervised Grounding Alignment for Multi-modal Feature Learning

  • May 01, 2022
  • Shih-Han Chou +3
  • PDF
  • Conference Article
  • Citations7

Representation, Learning and Reasoning on Spatial Language for Downstream NLP Tasks

  • Jan 01, 2020
  • Parisa Kordjamshidi +2
  • Conference Article
  • Citations37

GilBERT: Generative Vision-Language Pre-Training for Image-Text Retrieval

  • Jul 11, 2021
  • Weixiang Hong +5
  • Research Article
  • Citations8

Enhancing Visual Grounding in Vision-Language Pre-Training With Position-Guided Text Prompts.

  • May 01, 2024
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Alex Jinpeng Wang +3
  • Book Chapter

Neurosymbolic Visual Reasoning with Scene Graphs and Multimodal LLMs

  • Mar 17, 2025
  • Frontiers in artificial intelligence and applications
  • M Jaleed Khan +2
  • Conference Article
  • Citations271

Vision-Language Pre-Training with Triple Contrastive Learning

  • Jun 01, 2022
  • Jinyu Yang +8
  • Dissertation

Backdoor in deep learning: new threats and opportunities

  • Jan 01, 2025
  • Kangjie Chen
  • Research Article
  • Citations1

A survey on pre-training and transfer learning for multimodal Vision-Language Models

  • Jun 10, 2025
  • Advances in Engineering Innovation
  • Zhongren Liang
  • Dissertation
  • Citations10

Vision language representation learning

  • Jan 01, 2023
  • Xiaofeng Yang
  • Research Article
  • Citations2

Boosting Generic Visual-Linguistic Representation With Dynamic Contexts

  • Jan 01, 2023
  • IEEE Transactions on Multimedia
  • Guoqing Ma +5
  • PDF
  • Research Article
  • Citations5

Advancing Accuracy in Multimodal Medical Tasks Through Bootstrapped Language-Image Pretraining (BioMedBLIP): Performance Evaluation Study

  • Aug 05, 2024
  • JMIR Medical Informatics
  • Usman Naseem +2
  • PDF
  • Research Article
  • Citations14

On the Limitations of Visual-Semantic Embedding Networks for Image-to-Text Information Retrieval

  • Jul 26, 2021
  • Journal of Imaging
  • Yan Gong +2
  • Conference Article
  • Citations14

Proposal-free One-stage Referring Expression via Grid-Word Cross-Attention

  • Aug 01, 2021
  • Wei Suo +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.