Weakly Supervised Fine-Grained Image Classification via Salient Region Localization and Different Layer Feature Fusion

In recent years, deep learning techniques have demonstrated their powerful feature extraction capabilities in a variety of visual analysis tasks. Especially in the field of fine-grained image classification, deep feature extraction technology has achieved great performance improvement. The depth features extracted based on the deep learning model have strong expressive power. Fine-grained image classification is intended to subclass large categories, such as different types of birds. This identification task is more challenging because these sub-classes have large interclass differences and small inter-class differences. This paper first introduces the research progress of fine-grained image classification. Then, the data set used in the experiment and the extended data set are introduced. A variable convolution method for fine-grained image classification is proposed, and the research direction of the next step is summarized and explored.

Fine-grained image classification is quite challenging due to high inter-class similarity and large intra-class variations. Another issue is the small amount of training images with a large number of classes to be identified. To address the challenges, we propose a model for fine-grained image classification with its application to bird species recognition. Based on the features extracted by bilinear convolutional neural network (BCNN), we propose an on-line dictionary learning algorithm where the principle of sparsity is integrated into classification. The features extracted by BCNN encode pairwise neuron interaction in a translation-invariant manner. This property is valuable to fine-grained classification. The proposed algorithm for dictionary learning further carries out sparsity based classification, where training data can be represented with a less number of dictionary atoms. It alleviates the problems caused by insufficient training data, and makes classification much more efficient. Our approach is evaluated and compared with the state-of-the-art approaches on the CUB-200-2011 dataset. The promising experimental results demonstrate its efficacy and superiority.

To efficiently capture feature information in tasks of fine-grained image classification, this study introduces a new network model for fine-grained image classification, which utilizes a hybrid attention approach. The model is built upon a hybrid attention module (MA), and with the assistance of the attention erasure module (EA), it can adaptively enhance the prominent areas in the image and capture more detailed image information. Specifically, for tasks involving fine-grained image classification, this study designs an attention module capable of applying the attention mechanism to both the channel and spatial dimensions. This highlights the important regions and key feature channels in the image, allowing for the extraction of distinct local features. Furthermore, this study presents an attention erasure module (EA) that can remove significant areas in the image based on the features identified; thus, shifting focus to additional feature details within the image and improving the diversity and completeness of the features. Moreover, this study enhances the pooling layer of ResNet50 to augment the perceptual region and the capability to extract features from the network's less deep layers. For the objective of fine-grained image classification, this study extracts a variety of features and merges them effectively to create the final feature representation. To assess the effectiveness of the proposed model, experiments were conducted on three publicly available fine-grained image classification datasets: Stanford Cars, FGVC-Aircraft, and CUB-200-2011. The method achieved classification accuracies of 92.8, 94.0, and 88.2% on these datasets, respectively. In comparison with existing approaches, the efficiency of this method has significantly improved, demonstrating higher accuracy and robustness.

Thus far, few studies have been conducted on fine-grained classification tasks for the latest convolutional neural network ConvNeXt, and no effective optimization method has been made available. To achieve more accurate fine-grained classification, this paper proposes two attention embedding methods based on ConvNeXt network and designs a new bilinear CBAM; simultaneously, a multiscale, multi-perspective and all-around attention framework is proposed, which is then applied in ConvNeXt. Experimental verification shows that the accuracy rate of the improved ConvNeXt for fine-grained image classification reaches 87.8%, 91.2%, and 93.2% on fine-grained classification datasets CUB-200-2011, Stanford Cars, and FGVC Aircraft, respectively, showing increases of 2.7%, 0.3% and 0.4%, respectively, compared to those of the original network without optimization, and increases of 3.7%, 8.0% and 2.0%, respectively, compared to those of the traditional BCNN. In addition, ablation experiments are set up to verify the effectiveness of the proposed attention framework.

Fine-grained image classification is a challenging problem, due to the small inter-class variance caused by highly similar subordinate categories and large intra-class variance in poses, viewpoints and rotations. In this paper, we propose a novel end-to-end model for fine-grained image classification(FGIC). The proposed model consists of two sub-networks: detection sub-network and classification sub-network. The detection sub-network is constructed on the basis of R-FCN, and the classification sub-network contains a two-steam CNN for feature extraction and three fully connected layers for object classification. In addition, the network compression technology is adopted in both of the sub-networks to improve efficiency and reduce storage space. Experimental results on the CUB-200-2011 shows that the accuracy of our method is close to state-of-the-art with higher efficiency and lower storage requirement than the other compared methods (10 frames/sec during inference on TitanX). The proposed high-efficiency framework is believed to be effective in some of the practical applications, especially in the applications of mobile terminals.

Feature relocation network for fine-grained image classification

Fine-grained image categorization is to recognize subcategories from each other. In actual photographs, there is often large intra-class variation and small inter-class variation, which make fine-grained image categorization a challenging task. Existing bilinear-pooling-based methods are less interpretable, and visual-attention-based methods often suffer from limited number of salient part. In this paper, we propose a saliency-based weakly-supervised network, a simple approach that can effectively preserve all the salient details of the original image. Based on the saliency sampler backbone, we apply feature pyramid network structure to combine the saliency information with high-level features and use KL-divergence for knowledge distillation. Experiments on Caltech-UCSD Birds dataset show that components of our network are efficient and reliable.

Fine-grained image classification aims to distinguish sub-categories under a general category, characterized by small inter-class differences and large intra-class variations. This paper proposes an improved fine-grained image classification model based on the Squeeze and Excitation attention mechanism, Generalized Mean pooling, and multi-level feature fusion. The SE module enhances the discriminative power of important feature channels, GeM pooling replaces the traditional average pooling to retain more global context, and multi-level feature fusion integrates representations from different depths of the network to improve local detail perception. Experiments conducted on the CUB-200-2011 dataset demonstrate that the proposed method achieves higher accuracy compared with the baseline ResNet50, validating the effectiveness of the combined strategies.

Learning delicate feature representation of object parts plays a critical role in fine-grained visual classification tasks. However, advanced deep convolutional neural networks trained for general visual classification tasks usually tend to focus on the coarse-grained information while ignoring the fine-grained one, which is of great significance for learning discriminative representation. In this work, we explore the great merit of multi-modal data in introducing semantic knowledge and sequential analysis techniques in learning hierarchical feature representation for generating discriminative fine-grained features. To this end, we propose a novel approach, termed Channel Cusum Attention ResNet (CCA-ResNet ), for multi-modal joint learning of fine-grained representation. Specifically, we use feature-level multi-modal alignment to connect image and text classification models for joint multi-modal training. Through joint training, image classification models trained with semantic level labels tend to focus on the most discriminative parts, which enhances the cognitive ability of the model. Then, we propose a Channel Cusum Attention (CCA ) mechanism to equip feature maps with hierarchical properties through unsupervised reconstruction of local and global features. The benefits brought by the CCA are in two folds: a) allowing fine-grained features from early layers to be preserved in the forward propagation of deep networks; b) leveraging the hierarchical properties to facilitate multi-modal feature alignment. We conduct extensive experiments to verify that our proposed model can achieve state-of-the-art performance on a series of fine-grained visual classification benchmarks.

The focus of fine-grained image classification tasks is to ignore interference information and grasp local features. This challenge is what the visual attention mechanism excels at. Firstly, we have constructed a two-level attention convolutional network, which characterizes the object-level attention and the pixel-level attention. Then, we combine the two kinds of attention through a second-order response transform algorithm. Furthermore, we propose a clustering-based grouping attention model, which implies the part-level attention. The grouping attention method is to stretch all the semantic features, in a deeper convolution layer of the network, into vectors. These vectors are clustered by a vector dot product, and each category represents a special semantic. The grouping attention algorithm implements the functions of group convolution and feature clustering, which can greatly reduce the network parameters and improve the recognition rate and interpretability of the network. Finally, the low-level visual features and high-level semantic information are merged by a multi-level feature fusion method to accurately classify fine-grained images. We have achieved good results without using pre-training networks and fine-tuning techniques.

Fine-grained image classification has drawn increasing attention as it is much closer to practical applications than generic image classification. The majority of current fine-grained approaches locate the discriminative regions and leverage the features of these regions for classification as their magic weapons. However, these approaches simply ignore the internal semantic region correlation. As is well known, the correlation reveals the salient information of images, which can further boost the performance of fine-grained image classification. To this end, we propose an Object Decoupling with Graph Correlation network (ODGC) to explore the informative potentials of region correlation. A Responsive Object Location Module (ROLM) is first introduced to obtain the fine-grained object within a bounding box automatically. A Semantic Decoupling Module (SDM) then segments the object into different parts. ODGC learns the representations of these parts by transferring these part features into a Graph Correlation Module (GCM). Consists of these three main modules, ODGC is trained for fine-grained image classification in an end-to-end way. Extensive experiments conducted on CUB-200-2011 demonstrate that the aforementioned modules significantly improve the ODGC, and it achieves a new state-of-the-art performance to 88.2% top-1 accuracy. Besides, we collect a practical business e-commercial dataset, named Ecom-15K. The evaluation on it further validates the applicability of our method in practical scenarios.

Fine-grained image classification is a difficult problem, and previous studies mainly overcome this problem by locating multiple discriminative regions in different scales and then aggregating complementary information explored from the located regions. However, locating discriminative regions introduces heavy overhead and is not suitable for real-world application. In this paper, we propose the recursive multi-scale channel-spatial attention module (RMCSAM) for addressing this problem. Following the experience of previous research on fine-grained image classification, RMCSAM explores multi-scale attentional information. However, the attentional information is explored by recursively refining the deep feature maps of a convolutional neural network (CNN) to better correspond to multi-scale channel-wise and spatial-wise attention, instead of localizing attention regions. In this way, RMCSAM provides a lightweight module that can be inserted into standard CNNs. Experimental results show that RMCSAM can improve the classification accuracy and attention capturing ability over baselines. Also, RMCSAM performs better than other state-of-the-art attention modules in fine-grained image classification, and is complementary to some state-of-the-art approaches for fine-grained image classification. Code is available at https://github.com/Dichao-Liu/Recursive-Multi-Scale-Channel-Spatial-Attention-Module.

This paper studies the real-time water quality prediction model based on fine-grained image classification and home aerobic exercise rehabilitation for the elderly. Fine-grained image classification refers to the classification of sub-categories under image classification. In the past, image classification mainly classified images into large categories. In contrast, the most important feature of fine-grained image classification is the large differences within the class and the small differences between the classes, which are very difficult. And because it has a wide range of application requirements in human production and life, it has attracted a large number of scholars to conduct in-depth research on it. Based on the above research, the performance of the proposed model is analyzed and verified for specific water quality parameter prediction scenarios. The water quality model is an important part of the study of water environment simulation and water environment planning and management. Starting from the research review of water quality models, this paper introduces the development history of water quality models and several major non-mechanical water quality models and application research progress. It recognizes the objective facts of uncertainty in the water environment system and the limitations of organic water quality model in the application of water environment system in China. Finally, based on the above conclusions, elastic band resistance training combined with cardiac rehabilitation education improved the physical function of patients with coronary heart disease in the community, especially muscle health and cardiopulmonary rehabilitation training, as well as the quality of life of patients. The Heart Rehabilitation Training Program at Home can improve the awareness of cardiac rehabilitation of patients with coronary heart disease in the region, strengthen exercise compliance, popularize and apply to the home care of patients with coronary heart disease in the region, and also provide theoretical reference materials for regional medical personnel helping patients at home with training.

Text contained in an image carries high-level semantics that can be exploited to achieve richer image understanding. In particular, the mere presence of text provides strong guiding content that should be employed to tackle a diversity of computer vision tasks such as image retrieval, fine-grained classification, and visual question answering. In this paper, we address the problem of fine-grained classification and image retrieval by leveraging textual information along with visual cues to comprehend the existing intrinsic relation between the two modalities. The novelty of the proposed model consists of the usage of a PHOC descriptor to construct a bag of textual words along with a Fisher Vector Encoding that captures the morphology of text. This approach provides a stronger multimodal representation for this task and as our experiments demonstrate, it achieves state-of-the-art results on two different tasks, fine-grained classification and image retrieval. The code of this model will be publicly available at1.

CariesFG: A fine-grained RGB image classification framework with attention mechanism for dental caries