• Home
  • Search
  • Patch-Level Consistency Regularization in Self-Supervised Transfer Learning for Fine-Grained Image Recognition
  • Cite Icon1
  • https://doi.org/10.3390/app131810493Copy DOI Icon

Patch-Level Consistency Regularization in Self-Supervised Transfer Learning for Fine-Grained Image Recognition

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Fine-grained image recognition aims to classify fine subcategories belonging to the same parent category, such as vehicle model or bird species classification. This is an inherently challenging task because a classifier must capture subtle interclass differences under large intraclass variances. Most previous approaches are based on supervised learning, which requires a large-scale labeled dataset. However, such large-scale annotated datasets for fine-grained image recognition are difficult to collect because they generally require domain expertise during the labeling process. In this study, we propose a self-supervised transfer learning method based on Vision Transformer (ViT) to learn finer representations without human annotations. Interestingly, it is observed that existing self-supervised learning methods using ViT (e.g., DINO) show poor patch-level semantic consistency, which may be detrimental to learning finer representations. Motivated by this observation, we propose a consistency loss function that encourages patch embeddings of the overlapping area between two augmented views to be similar to each other during self-supervised learning on fine-grained datasets. In addition, we explore effective transfer learning strategies to fully leverage existing self-supervised models trained on large-scale labeled datasets. Contrary to the previous literature, our findings indicate that training only the last block of ViT is effective for self-supervised transfer learning. We demonstrate the effectiveness of our proposed approach through extensive experiments using six fine-grained image classification benchmark datasets, including FGVC Aircraft, CUB-200-2011, Food-101, Oxford 102 Flowers, Stanford Cars, and Stanford Dogs. Under the linear evaluation protocol, our method achieves an average accuracy of 78.5%, outperforming the existing transfer learning method, which yields 77.2%.

Loading PDF

Similar Papers
  • Research Article
  • Citations2063

Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey.

  • May 04, 2020
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Longlong Jing +1
  • Research Article
  • Citations14

Benchmarking Self-Supervised Contrastive Learning Methods for Image-Based Plant Phenotyping

  • Jan 01, 2023
  • Plant Phenomics
  • Franklin C Ogidi +2
  • Research Article
  • Citations48

CS-CO: A Hybrid Self-Supervised Visual Representation Learning Method for H&E-stained Histopathological Images.

  • Oct 01, 2022
  • Medical Image Analysis
  • Pengshuai Yang +6
  • Research Article
  • Citations20

Self-supervised visual learning in the low-data regime: A comparative evaluation

  • Mar 01, 2025
  • Neurocomputing
  • Sotirios Konstantakos +6
  • Conference Article
  • Citations18

Self-supervised Visual Attribute Learning for Fashion Compatibility

  • Oct 01, 2021
  • Donghyun Kim +5
  • Research Article
  • Citations20

In-Distribution and Out-of-Distribution Self-Supervised ECG Representation Learning for Arrhythmia Detection.

  • Feb 01, 2024
  • IEEE journal of biomedical and health informatics
  • Sahar Soltanieh +2
  • Research Article
  • Citations16

Considering three elements of aesthetics: Multi-task self-supervised feature learning for image style classification

  • Nov 04, 2022
  • Neurocomputing
  • Hua Zhang +5
  • Research Article
  • Citations2

Research on Self-Supervised Comparative Learning for Computer Vision

  • Aug 17, 2021
  • Journal of Electronic Research and Application
  • Yuanyuan Liu +1
  • Conference Article
  • Citations2

A Novel Multi-Task Self-Supervised Representation Learning Paradigm

  • May 28, 2021
  • Control theory & applications
  • Yinggang Li +5
  • Research Article
  • Citations47

Self-supervised reinforcement learning-based energy management for a hybrid electric vehicle

  • Dec 01, 2021
  • Journal of Power Sources
  • Chunyang Qi +7
  • Research Article

Study on Various Self-supervised Video Representation Learning Methods

  • Aug 31, 2022
  • Moving Image & Technology (MINT)
  • Soohyun Park +1
  • Research Article
  • Citations7

Randomly shuffled convolution for self-supervised representation learning

  • Nov 14, 2022
  • Information Sciences
  • Youngjin Oh +3
  • Research Article
  • Citations17

Self‐supervised learning improves classification of agriculturally important insect pests in plants

  • Jul 18, 2023
  • The Plant Phenome Journal
  • Soumyashree Kar +11
  • Book Chapter
  • Citations12

Case Discrimination: Self-supervised Feature Learning for the Classification of Focal Liver Lesions

  • Jan 01, 2021
  • Haohua Dong +6
  • Research Article

Enhancing Slice-Wise Brain MRI Tasks using Self-Supervised and Auxiliary Learning.

  • Jan 01, 2025
  • IEEE journal of biomedical and health informatics
  • Giovanni Castiglioni +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.