Feedback-based Dynamically Weighted BoF for Image Retrieval

Compact and discriminative representation of Bag-of-Features

The idea of Bag of Features (BoF) is recently often employed for general object recognition. But, as it does not take positional relations of detected features into account, the recognition rate is still not very high for practical use. This paper proposes a method of describing the feature of an object by the BoF representation which considers positional information of the features. Although the original BoF representation is applied to an entire image, the proposed method employs multiple windows on an image. The BoF representation is applied to each of the windows to represent an object in the image interested for recognition. The performance of the proposed method is shown experimentally.

Common visual codebook generation methods used in a bag of visual words model, for example, k-means or Gaussian Mixture Model, use the Euclidean distance to cluster features into visual code words. However, most popular visual descriptors are histograms of image measurements. It has been shown that with histogram features, the Histogram Intersection Kernel (HIK) is more effective than the Euclidean distance in supervised learning tasks. In this paper, we demonstrate that HIK can be used in an unsupervised manner to significantly improve the generation of visual codebooks. We propose a histogram kernel k-means algorithm which is easy to implement and runs almost as fast as the standard k-means. The HIK codebooks have consistently higher recognition accuracy over k-means codebooks by 2-4% in several benchmark object and scene recognition data sets. The algorithm is also generalized to arbitrary additive kernels. Its speed is thousands of times faster than a naive implementation of the kernel k-means algorithm. In addition, we propose a one-class SVM formulation to create more effective visual code words. Finally, we show that the standard k-median clustering method can be used for visual codebook generation and can act as a compromise between the HIK / additive kernel and the k-means approaches.

Local interest points serve as elementary building blocks in many image retrieval algorithms, and most of them exploit the local volume features using a Bag of Feature (BOF) model. However, the model ignores seriously valuable information about the global features in image and the distribution of the interest points. In this paper, we combine the sift feature and a global color feature. Then, we propose an improved strategy based on the BOF model. Finally, we embed the binary of the sift and color feature in the BOF model. Convincing experimental results on several datasets demonstrate that our proposed method approaches to the state-of-the-art level in image retrieval.

Gender classification is challenging. Methods for gender classification need to discriminate subtle differences between male and female. Bag-of-Features (BoF) method with sparse coding has been proven very powerful in image classification. In this paper, we apply BoF method for gender classification. We use two sets of images: training images and testing images. All images are represented by a set of Scale-Invariant Feature Transform (SIFT) descriptors. In training stage, using sparse coding, Visual Words Dictionary (VWD) is constructed from SIFT descriptors extracted from training images. In testing, SIFT descriptors of testing images are approximated by visual words in VWD. The choices of approximating visual words determine the classification decision. We apply our method and another two popular methods on public dataset for gender classification. We achieved promising results.

This paper presents an image representation and matching framework for image categorization in medical image archives. Categorization enables one to determine automatically, based on the image content, the examined body region and imaging modality. It is a basic step in content-based image retrieval (CBIR) systems, the goal of which is to augment text-based search with visual information analysis. CBIR systems are currently being integrated with picture archiving and communication systems for increasing the overall search capabilities and tools available to radiologists. The proposed methodology is comprised of a continuous and probabilistic image representation scheme using Gaussian mixture modeling (GMM) along with information-theoretic image matching via the Kullback-Leibler (KL) measure. The GMM-KL framework is used for matching and categorizing X-ray images by body regions. A multidimensional feature space is used to represent the image input, including intensity, texture, and spatial information. Unsupervised clustering via the GMM is used to extract coherent regions in feature space that are then used in the matching process. A dominant characteristic of the radiological images is their poor contrast and large intensity variations. This presents a challenge to matching among the images, and is handled via an illumination-invariant representation. The GMM-KL framework is evaluated for image categorization and image retrieval on a dataset of 1500 radiological images. A classification rate of 97.5% was achieved. The classification results compare favorably with reported global and local representation schemes. Precision versus recall curves indicate a strong retrieval result as compared with other state-of-the-art retrieval techniques. Finally, category models are learned and results are presented for comparing images to learned category models.

Urban building recognition plays a central role in applications such as urban mapping, heritage documentation, autonomous navigation, and smart city monitoring. Although recent advances have been driven mainly by deep learning approaches, classical visual pipelines remain an attractive alternative in scenarios where datasets are limited, interpretability is required, and computational resources are constrained. In this study, a systematic evaluation of a Bag-of-Features (BoF) representation combined with a Support Vector Machine (SVM) classifier is presented for urban building recognition using the Sheffield Building Image Dataset (SBID). The experimental protocol includes dataset balancing, a reproducible training–testing split, and an extensive investigation of visual vocabulary sizes ranging from 100 to 3000 visual words. The results indicate that increasing the vocabulary size generally improves recognition performance up to a saturation point, with the best trade-off achieved using 2000 visual words. Under this configuration, the proposed approach achieved an overall accuracy of 97.5% while maintaining an average inference time below 25 ms per image, demonstrating competitive performance with low computational cost. A detailed analysis based on confusion matrices and per-class metrics (accuracy, precision, recall, and F1-score) shows that most building categories were recognized with high reliability, while misclassifications were mainly concentrated among visually similar façade types. These findings confirm that BoF representations, when properly tuned, remain highly effective for structured urban recognition tasks. Moreover, the obtained results are consistent with those commonly reported in the literature for the same dataset and problem domain, reinforcing the robustness of the proposed pipeline. Overall, the results highlight the continued relevance of classical computer vision methods in contexts where transparency, reproducibility, and efficiency are essential. Future work will investigate hybrid strategies that combine BoF representations with deep convolutional descriptors, as well as more robust evaluation protocols, aiming to improve generalization across different building datasets and urban environments.

As a rapid developing research topic in the machine vision field, image-based hand posture recognition has the potential to be an efficient and intuitive tool of human-computer interaction. For improving the accuracy of multi-class hand postures and extending the algorithm generalization,a novel hand posture recognition method is proposed by integrating the multiple image features and multiple kernels learning support vector machine(SVM). Firstly, three types of feature descriptors are extracted to describe the characteristics of a hand posture image. Shape context descriptor represents distribution characteristics of the edge points of the hand posture image. Pyramid histogram of oriented gradient describes characteristics of local and global shape effectively. The Bag of Feature(BOF) algorithm describes the surface texture characteristics of the posture image. Secondly, the Chamfer kernel and histogram intersection kernel are rebuilt to obtain the basis kernels of the features. And the combined kernel is constructed by weighting the basis kernels.So the heterogeneous features fusion realizes. Finally, the classification model and optimal fusion weights are calculated by using multiple kernels learning algorithm. The unknown category posture can be recognized by the trained multiple kernels of SVM. Experiments on Jochen Triesch's hand posture dataset demonstrate that the proposed method obtains higher recognition rate than the traditional single-kernel classifier and other recent methods.

Multiple-Instance ranking based deep hashing for multi-Label image retrieval

This paper proposes a new approach for image classification by combining pyramid match kernel(PMK) with spatial pyramid. Unlike the conventional spatial pyramid matching (SPM) approach which only uses a single-resolution feature vector to represent an image, we use a multi-resolution feature vector to represent an image for SPM. We then calculate the match scores at each resolution of SPM representation and finally compute the matching between two images by applying the concept of PMK using the match scores obtained from the multiple resolutions. Our experimental results show that the proposed combined pyramid matching achieves a significant improvement on classification performance.

The airport target recognition method for remote sensing images is generally based on image matching, which is significantly affected by the variations of illumination, viewpoints, scale, and so on. As a well-known semantic model for target recognition, bag-of-features (BoF) performs k-means clustering on enormous local feature descriptors and thus generates the visual words to represent the images. We propose a fast automatic recognition framework for an airport target of a low-resolution remote sensing image under a complicated environment. It can be viewed as a two-phase procedure: detection and then classification. Concretely, it first utilizes a visual attention model for locating the salient region, and then detects possible candidate targets and extracts saliency-constrained scale invariant feature transform descriptors to build a high-level semantics model. Consequently, BoF is applied to mine the high-level semantics of targets. Different from k-means in a traditional BoF, we employ locality preserving indexing (LPI) to obtain the visual words. Because LPI can consider the intrinsic local structure of descriptors and further enhance the ability of words to describe the image content, it can accurately classify the detected candidate targets. Experiments on the dataset of 10 kinds of airport aerial images demonstrate the feasibility and effectiveness of the proposed method.

Part-based Bag-of-Features (BoF) models such as Spatial Pyramid Matching (SPM) play an important role in image classification. Before sending the feature vectors into classifiers for training and testing, it is required to normalize them in order to approximately equalize ranges of the attributes and make them have comparable effects in distance computation. Although some works have been focused on general feature normalization, we do not see any discussion on specialized normalization algorithms for part-based BoF models. In this paper, we fill in the blank with extensive experiments and discussions. Based on solid normalization parameters (power and coefficient), we further study two straightforward part-based properties, i.e., the independent assumption and the hierarchical-contribution assumption, to scale the feature super-vectors separately. Finally, we test our algorithm on challenging image sets, i.e., Caltech 101 and CUB-200-2011, for general and fine-grained classification, and show its efficiency, scalability and adaptability in both scenarios.

Visual phrase considers multiple visual words and captures extra spatial clues among them. Thus, visual phrase shows better discriminative power than single visual word in image retrieval and matching. Not withstanding their success, existing visual phrases still show obvious shortcomings: (1) limited flexibility, i.e., visual phrases are considered for matching only if they contain the same number of visual words; (2) large quantization error and low repeatability, i.e., quantization errors in visual words are aggregated in visual word combinations and visual phrases, making them harder to be matched than single visual words. To avoid these issues, we propose multi-order visual phrase (MVP) which contains two complementary clues: center visual word quantized from the local descriptor of each image keypoint and the visual and spatial clues of multiple nearby keypoints. Two MVPs are flexibly matched by first matching their center visual words, then estimating a match confidence by checking the spatial and visual consistency of their neighbor keypoints. Therefore, center visual word matching equals to traditional visual word matching, but the neighbor spatial and visual clues checking significantly boosts the discriminative power. MVP does not scarify the repeatability of single visual word and is more robust to quantization error than existing visual phrases. We test our approach in three image retrieval tasks on UKbench, Oxford5K, and 1 million distractor images collected from Flickr. Comparisons with recent retrieval approaches and existing visual phrase features clearly demonstrate the competitive accuracy and significantly better efficiency of MVP.

Scene classiflcation is an appealing and challenging problem in image processing and machine vision. Recently, Bag-of-visual-words (BOVW) method using pyramid matching scheme has shown remarkable performance for scene classiflcation. But this method deriving from local keypoints does not contain texture features which are rich in scene images. To further improves the classiflcation accuracy, this paper presents a new method combining Rotation Invariant Local Binary Patterns (RILBP) texture features and BOVW model in spatial pyramid matching framework. First, scene image is subdivided at difierent resolutions for constructing a spatial pyramid. Then based on scale invariant feature transform descriptor and K-means clustering, Pyramid Histogram of visual Words (PHOW) is extracted. And RILBP texture feature is extracted using the mean of a 3*3 neighborhood as threshold. Last we construct a composite kernel of spatial pyramid matching. We regard the keypoint features and texture features as two independent feature channels, and combine them to realize scene classiflcation using one-against-rest SVMs with the composite kernel. Experiments results on the three difierent scene datasets show that our method is efiective.

Content-based image retrieval (CBIR) is a homogeneous search technology based on visual features, and information has been leaked to potential users of pharmaceutical testing, education, and research. However, the CBIR collects images related to acquisition properties, as a result of the inability to predict the similarity of images using the medical part, and the difference between the user’s search destination and object space. In the existing methods, effective points of interest are searched for, and L2 normalization is applied to the spatial arrangement of the original and global features and the RGB channels in the index image by effectively detecting their points of interest. The principal component analysis is used for redundant feature sets and feature sets to provide significant effects. The existing system has certain limitations, such as information loss during the hard task completion coding process and ignores the spatial relationship among the patches in image representation. To overcome the existing method issues, the integrated feature of the images is preferred. In this work, Semantic Weight Based Feed Forward Recurrent Back Propagation (SWFFRBP) is used to retrieve the images, and then based on the semantic weight the correctness of retrieval of images is calculated. The semantic weight determines the accuracy of retrieved images. After that, the recurrent backpropagation calculates the weight error derivatives arising from this entire episode of processing in the network. The existing features compared with the proposed features were Bag of Features (BoF), SURF, and HOG. The retrieval methods taken to compare with proposed retrieval techniques are RFRM and KNN. Thus, medical image retrieval for diagnosis using SWFFRBP, accuracy, False positive rate, relevance, feedback, precision, and recall parameters were tested and proved to be efficient than other methods.