Enhancing fine-grained image classification through attentive batch training

Although fine-grained image classification is able to classify more fine-grained sub-categories compared to its coarse-grained counterpart, it often fails to identify individual instances. Therefore, we propose a new instance-level image classification task which further refines the granularity of fine-grained classification in order to identify unique instances rather than a sub-category containing multiple instances. In addition, we introduce an instance-level image classification dataset, AircraftCarrier, which contains 20 global aircraft carrier classes, as the first publically available dataset for instance-level image classification. The classification of instance-level aircraft carriers can prove to be a challenging task due to large intra-category differences as well as variations in the camera view, illumination, scale, and the presence of complex backgrounds. The AircraftCarrier dataset put forward here has the potential to improve the development of instance-level image classification. At the same time, we provide a Simple Classification Head (SCH) technique for the classification of aircraft carriers, with classical convolutional neural network models as the backbone network. The SCH has better performance than a direct classification head, and these results provide a benchmark performance result for researchers. Furthermore, we evaluate several fine-grained image classification methods and give their benchmark results. Finally, we present the challenges of instance-level classification and discuss further directions. This study provides the first publicly available instance-level image classification dataset and a performance benchmark for further research. The dataset and codes can be downloaded at https://github.com/tsingqsu/AircraftCarrier_Dataset/.

To efficiently capture feature information in tasks of fine-grained image classification, this study introduces a new network model for fine-grained image classification, which utilizes a hybrid attention approach. The model is built upon a hybrid attention module (MA), and with the assistance of the attention erasure module (EA), it can adaptively enhance the prominent areas in the image and capture more detailed image information. Specifically, for tasks involving fine-grained image classification, this study designs an attention module capable of applying the attention mechanism to both the channel and spatial dimensions. This highlights the important regions and key feature channels in the image, allowing for the extraction of distinct local features. Furthermore, this study presents an attention erasure module (EA) that can remove significant areas in the image based on the features identified; thus, shifting focus to additional feature details within the image and improving the diversity and completeness of the features. Moreover, this study enhances the pooling layer of ResNet50 to augment the perceptual region and the capability to extract features from the network's less deep layers. For the objective of fine-grained image classification, this study extracts a variety of features and merges them effectively to create the final feature representation. To assess the effectiveness of the proposed model, experiments were conducted on three publicly available fine-grained image classification datasets: Stanford Cars, FGVC-Aircraft, and CUB-200-2011. The method achieved classification accuracies of 92.8, 94.0, and 88.2% on these datasets, respectively. In comparison with existing approaches, the efficiency of this method has significantly improved, demonstrating higher accuracy and robustness.

This paper studies the real-time water quality prediction model based on fine-grained image classification and home aerobic exercise rehabilitation for the elderly. Fine-grained image classification refers to the classification of sub-categories under image classification. In the past, image classification mainly classified images into large categories. In contrast, the most important feature of fine-grained image classification is the large differences within the class and the small differences between the classes, which are very difficult. And because it has a wide range of application requirements in human production and life, it has attracted a large number of scholars to conduct in-depth research on it. Based on the above research, the performance of the proposed model is analyzed and verified for specific water quality parameter prediction scenarios. The water quality model is an important part of the study of water environment simulation and water environment planning and management. Starting from the research review of water quality models, this paper introduces the development history of water quality models and several major non-mechanical water quality models and application research progress. It recognizes the objective facts of uncertainty in the water environment system and the limitations of organic water quality model in the application of water environment system in China. Finally, based on the above conclusions, elastic band resistance training combined with cardiac rehabilitation education improved the physical function of patients with coronary heart disease in the community, especially muscle health and cardiopulmonary rehabilitation training, as well as the quality of life of patients. The Heart Rehabilitation Training Program at Home can improve the awareness of cardiac rehabilitation of patients with coronary heart disease in the region, strengthen exercise compliance, popularize and apply to the home care of patients with coronary heart disease in the region, and also provide theoretical reference materials for regional medical personnel helping patients at home with training.

Fine-grained image classification has drawn increasing attention as it is much closer to practical applications than generic image classification. The majority of current fine-grained approaches locate the discriminative regions and leverage the features of these regions for classification as their magic weapons. However, these approaches simply ignore the internal semantic region correlation. As is well known, the correlation reveals the salient information of images, which can further boost the performance of fine-grained image classification. To this end, we propose an Object Decoupling with Graph Correlation network (ODGC) to explore the informative potentials of region correlation. A Responsive Object Location Module (ROLM) is first introduced to obtain the fine-grained object within a bounding box automatically. A Semantic Decoupling Module (SDM) then segments the object into different parts. ODGC learns the representations of these parts by transferring these part features into a Graph Correlation Module (GCM). Consists of these three main modules, ODGC is trained for fine-grained image classification in an end-to-end way. Extensive experiments conducted on CUB-200-2011 demonstrate that the aforementioned modules significantly improve the ODGC, and it achieves a new state-of-the-art performance to 88.2% top-1 accuracy. Besides, we collect a practical business e-commercial dataset, named Ecom-15K. The evaluation on it further validates the applicability of our method in practical scenarios.

Text contained in an image carries high-level semantics that can be exploited to achieve richer image understanding. In particular, the mere presence of text provides strong guiding content that should be employed to tackle a diversity of computer vision tasks such as image retrieval, fine-grained classification, and visual question answering. In this paper, we address the problem of fine-grained classification and image retrieval by leveraging textual information along with visual cues to comprehend the existing intrinsic relation between the two modalities. The novelty of the proposed model consists of the usage of a PHOC descriptor to construct a bag of textual words along with a Fisher Vector Encoding that captures the morphology of text. This approach provides a stronger multimodal representation for this task and as our experiments demonstrate, it achieves state-of-the-art results on two different tasks, fine-grained classification and image retrieval. The code of this model will be publicly available at1.

Thus far, few studies have been conducted on fine-grained classification tasks for the latest convolutional neural network ConvNeXt, and no effective optimization method has been made available. To achieve more accurate fine-grained classification, this paper proposes two attention embedding methods based on ConvNeXt network and designs a new bilinear CBAM; simultaneously, a multiscale, multi-perspective and all-around attention framework is proposed, which is then applied in ConvNeXt. Experimental verification shows that the accuracy rate of the improved ConvNeXt for fine-grained image classification reaches 87.8%, 91.2%, and 93.2% on fine-grained classification datasets CUB-200-2011, Stanford Cars, and FGVC Aircraft, respectively, showing increases of 2.7%, 0.3% and 0.4%, respectively, compared to those of the original network without optimization, and increases of 3.7%, 8.0% and 2.0%, respectively, compared to those of the traditional BCNN. In addition, ablation experiments are set up to verify the effectiveness of the proposed attention framework.

In recent years, deep learning techniques have demonstrated their powerful feature extraction capabilities in a variety of visual analysis tasks. Especially in the field of fine-grained image classification, deep feature extraction technology has achieved great performance improvement. The depth features extracted based on the deep learning model have strong expressive power. Fine-grained image classification is intended to subclass large categories, such as different types of birds. This identification task is more challenging because these sub-classes have large interclass differences and small inter-class differences. This paper first introduces the research progress of fine-grained image classification. Then, the data set used in the experiment and the extended data set are introduced. A variable convolution method for fine-grained image classification is proposed, and the research direction of the next step is summarized and explored.

Feature relocation network for fine-grained image classification

Fine-grained image classification is a difficult problem, and previous studies mainly overcome this problem by locating multiple discriminative regions in different scales and then aggregating complementary information explored from the located regions. However, locating discriminative regions introduces heavy overhead and is not suitable for real-world application. In this paper, we propose the recursive multi-scale channel-spatial attention module (RMCSAM) for addressing this problem. Following the experience of previous research on fine-grained image classification, RMCSAM explores multi-scale attentional information. However, the attentional information is explored by recursively refining the deep feature maps of a convolutional neural network (CNN) to better correspond to multi-scale channel-wise and spatial-wise attention, instead of localizing attention regions. In this way, RMCSAM provides a lightweight module that can be inserted into standard CNNs. Experimental results show that RMCSAM can improve the classification accuracy and attention capturing ability over baselines. Also, RMCSAM performs better than other state-of-the-art attention modules in fine-grained image classification, and is complementary to some state-of-the-art approaches for fine-grained image classification. Code is available at https://github.com/Dichao-Liu/Recursive-Multi-Scale-Channel-Spatial-Attention-Module.

The public has long been interested in computer vision research projects on image classification. Over the past 20 years, fine-grained image classification (FGIC) has advanced quickly because of the ongoing development of deep neural network models. The FDIC is based on the traditional image classification and further identifies the subtle differences between subclasses within the same category. Deep learning-based image categorization techniques are separated into two groups in this article: FGIC based on intensely supervised learning and weakly supervised learning. Briefly, it introduces the algorithms included in each category. Additionally, this article lists the performance of several methods on the well-known CUB-200 dataset and gives typical fine-grained picture datasets. By comparing several algorithms' outputs, it is determined that weakly supervised learning has the advantages of lower cost and higher accuracy than intensely supervised learning. Finally, the paper proposes a summary and a discussion of fine-grained images' potential future development prospects.

Recent research has shown the initial success of sparse coding (Sc) in solving many computer vision tasks. Motivated by the fact that kernel trick can capture the nonlinear similarity of features, which helps in finding a sparse representation of nonlinear features, we propose kernel sparse representation (KSR). Essentially, KSR is a sparse coding technique in a high dimensional feature space mapped by an implicit mapping function. We apply KSR to feature coding in image classification, face recognition, and kernel matrix approximation. More specifically, by incorporating KSR into spatial pyramid matching (SPM), we develop KSRSPM, which achieves a good performance for image classification. Moreover, KSR-based feature coding can be shown as a generalization of efficient match kernel and an extension of Sc-based SPM. We further show that our proposed KSR using a histogram intersection kernel (HIK) can be considered a soft assignment extension of HIK-based feature quantization in the feature coding process. Besides feature coding, comparing with sparse coding, KSR can learn more discriminative sparse codes and achieve higher accuracy for face recognition. Moreover, KSR can also be applied to kernel matrix approximation in large scale learning tasks, and it demonstrates its robustness to kernel matrix approximation, especially when a small fraction of the data is used. Extensive experimental results demonstrate promising results of KSR in image classification, face recognition, and kernel matrix approximation. All these applications prove the effectiveness of KSR in computer vision and machine learning tasks.

Fine-grained image classification is a challenging problem, due to the small inter-class variance caused by highly similar subordinate categories and large intra-class variance in poses, viewpoints and rotations. In this paper, we propose a novel end-to-end model for fine-grained image classification(FGIC). The proposed model consists of two sub-networks: detection sub-network and classification sub-network. The detection sub-network is constructed on the basis of R-FCN, and the classification sub-network contains a two-steam CNN for feature extraction and three fully connected layers for object classification. In addition, the network compression technology is adopted in both of the sub-networks to improve efficiency and reduce storage space. Experimental results on the CUB-200-2011 shows that the accuracy of our method is close to state-of-the-art with higher efficiency and lower storage requirement than the other compared methods (10 frames/sec during inference on TitanX). The proposed high-efficiency framework is believed to be effective in some of the practical applications, especially in the applications of mobile terminals.

Fine-grained image classification is one of the key basic tasks of computer vision. The appearance of traditional deep convolutional neural network (DCNN) combined with attention mechanism can focus on partial and local features of fine-grained images, but it still lacks the consideration of the embedding mode of different attention modules in the network, leading to the unsatisfactory result of classification model. To solve the above problems, three different attention mechanisms are introduced into the DCNN network (like ResNet, VGGNet, etc.), including SE, CBAM and ECA modules, so that DCNN could better focus on the key local features of salient regions in the image. At the same time, we adopt three different embedding modes of attention modules, including serial, residual and parallel modes, to further improve the performance of the classification model. The experimental results show that the three attention modules combined with three different embedding modes can improve the performance of DCNN network effectively. Moreover, compared with SE and ECA, CBAM has stronger feature extraction capability. Among them, the parallelly embedded CBAM can make the local information paid attention to by DCNN richer and more accurate, and bring the optimal effect for DCNN, which is 1.98% and 1.57% higher than that of original VGG16 and Resnet34 in CUB-200-2011 dataset, respectively. The visualization analysis also indicates that the attention modules can be easily embedded into DCNN networks, especially in the parallel mode, with stronger generality and universality.

Attribute hierarchy based multi-task learning for fine-grained image classification

Transfer learning is a cornerstone of computer vision, yet little work has been done to evaluate the relationship between architecture and transfer. An implicit hypothesis in modern computer vision research is that models that perform better on ImageNet necessarily perform better on other vision tasks. However, this hypothesis has never been systematically tested. Here, we compare the performance of 16 classification networks on 12 image classification datasets. We find that, when networks are used as fixed feature extractors or fine-tuned, there is a strong correlation between ImageNet accuracy and transfer accuracy (r = 0.99 and 0.96, respectively). In the former setting, we find that this relationship is very sensitive to the way in which networks are trained on ImageNet; many common forms of regularization slightly improve ImageNet accuracy but yield features that are much worse for transfer learning. Additionally, we find that, on two small fine-grained image classification datasets, pretraining on ImageNet provides minimal benefits, indicating the learned features from ImageNet do not transfer well to fine-grained tasks. Together, our results show that ImageNet architectures generalize well across datasets, but ImageNet features are less general than previously suggested.