Fine-grained Image Classification and Retrieval by Combining Visual and Locally Pooled Textual Features

Thus far, few studies have been conducted on fine-grained classification tasks for the latest convolutional neural network ConvNeXt, and no effective optimization method has been made available. To achieve more accurate fine-grained classification, this paper proposes two attention embedding methods based on ConvNeXt network and designs a new bilinear CBAM; simultaneously, a multiscale, multi-perspective and all-around attention framework is proposed, which is then applied in ConvNeXt. Experimental verification shows that the accuracy rate of the improved ConvNeXt for fine-grained image classification reaches 87.8%, 91.2%, and 93.2% on fine-grained classification datasets CUB-200-2011, Stanford Cars, and FGVC Aircraft, respectively, showing increases of 2.7%, 0.3% and 0.4%, respectively, compared to those of the original network without optimization, and increases of 3.7%, 8.0% and 2.0%, respectively, compared to those of the traditional BCNN. In addition, ablation experiments are set up to verify the effectiveness of the proposed attention framework.

To efficiently capture feature information in tasks of fine-grained image classification, this study introduces a new network model for fine-grained image classification, which utilizes a hybrid attention approach. The model is built upon a hybrid attention module (MA), and with the assistance of the attention erasure module (EA), it can adaptively enhance the prominent areas in the image and capture more detailed image information. Specifically, for tasks involving fine-grained image classification, this study designs an attention module capable of applying the attention mechanism to both the channel and spatial dimensions. This highlights the important regions and key feature channels in the image, allowing for the extraction of distinct local features. Furthermore, this study presents an attention erasure module (EA) that can remove significant areas in the image based on the features identified; thus, shifting focus to additional feature details within the image and improving the diversity and completeness of the features. Moreover, this study enhances the pooling layer of ResNet50 to augment the perceptual region and the capability to extract features from the network's less deep layers. For the objective of fine-grained image classification, this study extracts a variety of features and merges them effectively to create the final feature representation. To assess the effectiveness of the proposed model, experiments were conducted on three publicly available fine-grained image classification datasets: Stanford Cars, FGVC-Aircraft, and CUB-200-2011. The method achieved classification accuracies of 92.8, 94.0, and 88.2% on these datasets, respectively. In comparison with existing approaches, the efficiency of this method has significantly improved, demonstrating higher accuracy and robustness.

Feature relocation network for fine-grained image classification

Fine-grained image classification is a difficult problem, and previous studies mainly overcome this problem by locating multiple discriminative regions in different scales and then aggregating complementary information explored from the located regions. However, locating discriminative regions introduces heavy overhead and is not suitable for real-world application. In this paper, we propose the recursive multi-scale channel-spatial attention module (RMCSAM) for addressing this problem. Following the experience of previous research on fine-grained image classification, RMCSAM explores multi-scale attentional information. However, the attentional information is explored by recursively refining the deep feature maps of a convolutional neural network (CNN) to better correspond to multi-scale channel-wise and spatial-wise attention, instead of localizing attention regions. In this way, RMCSAM provides a lightweight module that can be inserted into standard CNNs. Experimental results show that RMCSAM can improve the classification accuracy and attention capturing ability over baselines. Also, RMCSAM performs better than other state-of-the-art attention modules in fine-grained image classification, and is complementary to some state-of-the-art approaches for fine-grained image classification. Code is available at https://github.com/Dichao-Liu/Recursive-Multi-Scale-Channel-Spatial-Attention-Module.

This paper studies the real-time water quality prediction model based on fine-grained image classification and home aerobic exercise rehabilitation for the elderly. Fine-grained image classification refers to the classification of sub-categories under image classification. In the past, image classification mainly classified images into large categories. In contrast, the most important feature of fine-grained image classification is the large differences within the class and the small differences between the classes, which are very difficult. And because it has a wide range of application requirements in human production and life, it has attracted a large number of scholars to conduct in-depth research on it. Based on the above research, the performance of the proposed model is analyzed and verified for specific water quality parameter prediction scenarios. The water quality model is an important part of the study of water environment simulation and water environment planning and management. Starting from the research review of water quality models, this paper introduces the development history of water quality models and several major non-mechanical water quality models and application research progress. It recognizes the objective facts of uncertainty in the water environment system and the limitations of organic water quality model in the application of water environment system in China. Finally, based on the above conclusions, elastic band resistance training combined with cardiac rehabilitation education improved the physical function of patients with coronary heart disease in the community, especially muscle health and cardiopulmonary rehabilitation training, as well as the quality of life of patients. The Heart Rehabilitation Training Program at Home can improve the awareness of cardiac rehabilitation of patients with coronary heart disease in the region, strengthen exercise compliance, popularize and apply to the home care of patients with coronary heart disease in the region, and also provide theoretical reference materials for regional medical personnel helping patients at home with training.

The fine-grained image classification task is about differentiating between different object classes. The difficulties of the task are large intra-class variance and small inter-class variance. For this reason, improving models’ accuracies on the task heavily relies on discriminative parts’ annotations and regional parts’ annotations. Such delicate annotations’ dependency causes the restriction on models’ practicability. To tackle this issue, a saliency module based on a weakly supervised fine-grained image classification model is proposed by this article. Through our salient region localization module, the proposed model can localize essential regional parts with the use of saliency maps, while only image class annotations are provided. Besides, the bilinear attention module can improve the performance on feature extraction by using higher- and lower-level layers of the network to fuse regional features with global features. With the application of the bilinear attention architecture, we propose the different layer feature fusion module to improve the expression ability of model features. We tested and verified our model on public datasets released specifically for fine-grained image classification. The results of our test show that our proposed model can achieve close to state-of-the-art classification performance on various datasets, while only the least training data are provided. Such a result indicates that the practicality of our model is incredibly improved since fine-grained image datasets are expensive.

In recent years, deep learning techniques have demonstrated their powerful feature extraction capabilities in a variety of visual analysis tasks. Especially in the field of fine-grained image classification, deep feature extraction technology has achieved great performance improvement. The depth features extracted based on the deep learning model have strong expressive power. Fine-grained image classification is intended to subclass large categories, such as different types of birds. This identification task is more challenging because these sub-classes have large interclass differences and small inter-class differences. This paper first introduces the research progress of fine-grained image classification. Then, the data set used in the experiment and the extended data set are introduced. A variable convolution method for fine-grained image classification is proposed, and the research direction of the next step is summarized and explored.

Fine-grained image classification has drawn increasing attention as it is much closer to practical applications than generic image classification. The majority of current fine-grained approaches locate the discriminative regions and leverage the features of these regions for classification as their magic weapons. However, these approaches simply ignore the internal semantic region correlation. As is well known, the correlation reveals the salient information of images, which can further boost the performance of fine-grained image classification. To this end, we propose an Object Decoupling with Graph Correlation network (ODGC) to explore the informative potentials of region correlation. A Responsive Object Location Module (ROLM) is first introduced to obtain the fine-grained object within a bounding box automatically. A Semantic Decoupling Module (SDM) then segments the object into different parts. ODGC learns the representations of these parts by transferring these part features into a Graph Correlation Module (GCM). Consists of these three main modules, ODGC is trained for fine-grained image classification in an end-to-end way. Extensive experiments conducted on CUB-200-2011 demonstrate that the aforementioned modules significantly improve the ODGC, and it achieves a new state-of-the-art performance to 88.2% top-1 accuracy. Besides, we collect a practical business e-commercial dataset, named Ecom-15K. The evaluation on it further validates the applicability of our method in practical scenarios.

In this article, we propose a novel system for feature selection, which is one of the key problems in content-based image indexing and retrieval as well as various other research fields such as pattern classification and genomic data analysis. The proposed system aims at enhancing semantic image retrieval results, decreasing retrieval process complexity, and improving the overall system usability for end-users of multimedia search engines. Three feature selection criteria and a decision method construct the feature selection system. Two novel feature selection criteria based on inner-cluster and intercluster relations are proposed in the article. A majority voting-based method is adapted for efficient selection of features and feature combinations. The performance of the proposed criteria is assessed over a large image database and a number of features, and is compared against competing techniques from the literature. Experiments show that the proposed feature selection system improves semantic performance results in image retrieval systems.

Fine-grained image classification is quite challenging due to high inter-class similarity and large intra-class variations. Another issue is the small amount of training images with a large number of classes to be identified. To address the challenges, we propose a model for fine-grained image classification with its application to bird species recognition. Based on the features extracted by bilinear convolutional neural network (BCNN), we propose an on-line dictionary learning algorithm where the principle of sparsity is integrated into classification. The features extracted by BCNN encode pairwise neuron interaction in a translation-invariant manner. This property is valuable to fine-grained classification. The proposed algorithm for dictionary learning further carries out sparsity based classification, where training data can be represented with a less number of dictionary atoms. It alleviates the problems caused by insufficient training data, and makes classification much more efficient. Our approach is evaluated and compared with the state-of-the-art approaches on the CUB-200-2011 dataset. The promising experimental results demonstrate its efficacy and superiority.

Retail products belonging to the same category usually have extremely similar appearance characteristics such as colors, shapes, and sizes, which cannot be distinguished by conventional classification methods. Currently, the most effective way to solve this problem is fine-grained classification methods, which utilize machine vision + scene to perform fine feature representations on a target local region, thereby achieving fine-grained classification. Fine-grained classification methods have been widely used for recognizing birds, cars, airplanes, and many others. However, the existing fine-grained classification methods still have some drawbacks. In this paper, we propose an improved fine-grained classification method based on self-attention destruction and construction learning (SADCL) for retail product recognition. Specifically, the proposed method utilizes a self-attention mechanism in the destruction and construction of image information in an end-to-end fashion so that to calculate a precise fine-grained classification prediction and large information areas in the reasoning process. We test the proposed method on the Retail Product Checkout (RPC) dataset. Experimental results demonstrate that the proposed method achieved an accuracy above 80% in retail commodity recognition reasoning, which is much higher than the results of other fine-grained classification methods.

Fine-grained image categorization is to recognize subcategories from each other. In actual photographs, there is often large intra-class variation and small inter-class variation, which make fine-grained image categorization a challenging task. Existing bilinear-pooling-based methods are less interpretable, and visual-attention-based methods often suffer from limited number of salient part. In this paper, we propose a saliency-based weakly-supervised network, a simple approach that can effectively preserve all the salient details of the original image. Based on the saliency sampler backbone, we apply feature pyramid network structure to combine the saliency information with high-level features and use KL-divergence for knowledge distillation. Experiments on Caltech-UCSD Birds dataset show that components of our network are efficient and reliable.

Fine-grained image classification is a challenging problem, due to the small inter-class variance caused by highly similar subordinate categories and large intra-class variance in poses, viewpoints and rotations. In this paper, we propose a novel end-to-end model for fine-grained image classification(FGIC). The proposed model consists of two sub-networks: detection sub-network and classification sub-network. The detection sub-network is constructed on the basis of R-FCN, and the classification sub-network contains a two-steam CNN for feature extraction and three fully connected layers for object classification. In addition, the network compression technology is adopted in both of the sub-networks to improve efficiency and reduce storage space. Experimental results on the CUB-200-2011 shows that the accuracy of our method is close to state-of-the-art with higher efficiency and lower storage requirement than the other compared methods (10 frames/sec during inference on TitanX). The proposed high-efficiency framework is believed to be effective in some of the practical applications, especially in the applications of mobile terminals.

Bilinear networks generally only do bilinear pooling operation on the feature vector after the last layer is processed by the convolutional layer of the activation function, and use the fused features for fine-grained image classification. However, this approach ignores the feature information of the middle convolutional layer, which cannot completely describe each semantic part of the image and easily causes the loss of discriminative information of fine-grained categories. In this paper, we propose a fine-grained image classification method based on multilayer feature fusion, in which the ResNet model is selected to replace the two-way VGGNet model in the original bilinear convolutional neural network to improve the feature extraction ability of the network, and the different layer feature outputs extracted from the last three convolutional residual blocks of ResNet are bilinearly pooled, and the three feature vectors are stitched together for multilayer The resulting three feature vectors are stitched together for multilayer feature fusion and then fed to the classifier for classification.

Fine-grained image classification aims to distinguish sub-categories under a general category, characterized by small inter-class differences and large intra-class variations. This paper proposes an improved fine-grained image classification model based on the Squeeze and Excitation attention mechanism, Generalized Mean pooling, and multi-level feature fusion. The SE module enhances the discriminative power of important feature channels, GeM pooling replaces the traditional average pooling to retain more global context, and multi-level feature fusion integrates representations from different depths of the network to improve local detail perception. Experiments conducted on the CUB-200-2011 dataset demonstrate that the proposed method achieves higher accuracy compared with the baseline ResNet50, validating the effectiveness of the combined strategies.