A Color Image Representation Approach for Content-Based Image Retrieval

This paper proposes a novel image compression scheme based on the local feature descriptor - Scale Invariant Feature Transform (SIFT). The SIFT descriptor characterizes an image region invariantly to scale and rotation. It is used widely in image retrieval. By using SIFT descriptors, our compression scheme is able to make use of external image contents to reduce visual redundancy among images. The proposed encoder compresses an input image by SIFT descriptors rather than pixel values. It separates the SIFT descriptors of the image into two groups, a visual description which is a significantly sub sampled image with key SIFT descriptors embedded and a set of differential SIFT descriptors, to reduce the coding bits. The corresponding decoder generates the SIFT descriptors from the visual description and the differential set. The SIFT descriptors are used in our SIFT-based matching to retrieve the candidate predictive patches from a large image dataset. These candidate patches are then integrated into the visual description, presenting the final reconstructed images. Our preliminary but promising results demonstrate the effectiveness of our proposed image coding scheme towards perceptual quality. Our proposed image compression scheme provides a feasible approach to make use of the visual correlation among images.

The Vector of Locally Aggregated Descriptors (VLAD) method, developed from BOW and Fisher Vector, has got great successes in image classification and retrieval. However, the traditional VLAD only assigns local descriptors to the closest visual words in the codebook, which is a hard voting process that leads to a large quantization error. In this paper, we propose an approach to fuse VLAD and locality-constrained linear coding (LLC), compared with the original method, several nearest neighbor centers are considered when assigning local descriptors. We use the reconstruction coefficients of LLC to obtain the weights of several nearest neighbor centers. Due to the excellent representation ability of the reconstruction coefficients for local descriptors, we also combine it with VLAD coding. Experiments were conducted on the 15 Scenes, UIUC Sports Event and Corel 10 datasets to demonstrate that our proposed method has outstanding performance in terms of classification accuracy. Our approach also does not generate much additional computational cost while encoding features.

In this paper, a novel face representation in terms of dense local image descriptors is proposed. Scale Invariant Feature Transform (SIFT) and Gabor, two of the most popular local image descriptors, at dense grid pixels of a face image are used to represent the face. The efficiency of the representation has been investigated in gender recognition. There are four problems when applying the SIFT to facial gender recognition. (1) There may be only a few keypoints that can be found in a face image due to the missing texture and ill-illumined faces; (2) The SIFT descriptors at the keypoints (we called it sparse SIFT) are distinctive whereas alternative descriptors at non-keypoints (e.g. grid) could cause negative impact on the accuracy; (3) Most of the existing methods employ SIFT descriptors matching in which relative larger image size is required in order that enough keypoints can be found to support the matching and (4) The matching is assumed that the faces are well registered. We provide solutions to the above difficulties in this paper and the problem of recognizing gender using the combination of SIFT descriptors and Gabor of face images is studied. The Gabor representations of the face images are fused with the dense SIFT at the feature-level to improve the accuracy. AdaBoost is adopted to select features and form a strong classifier. The experimental results on a large set of faces have shown that the proposed method can achieve high accuracies even for faces that are not aligned.

Local descriptors are very effective features in bag-of-visual-words (BoW) and vector of locally aggregated descriptors (VALD) models for image retrieval. Different kinds of local descriptors represent different visual content. We recognize that spatial contextual information play an important role in image matching, image retrieval and image recognition. Therefore, to explore efficient features, firstly, a new local composite descriptor is proposed, which combines the advantages of SURF and color name (CN) information. Then, VLAD method is used to encode the proposed composite descriptors to a vector. Third, local deep features are extracted and fused with the encoded vector in the image block. Finally, to implement efficient retrieval system, a novel image retrieval framework is organized a novel image retrieval framework is organized based on the proposed feature fusion strategies. The proposed methods areis verified on three benchmark datasets, i.e., Holidays, Oxford5k and Ukbench. Experimental results show that our methods achieves good performance. Eespecially, the mAP and N-S score achieve 0.8281 and 3.5498 on Holidays and Ukbench datasets, respectively.

With the advent of e-commerce, digital services and social media, scammers have changed their way to gain illegal benefits in various forms such as capturing the credit card information or exploiting personal cloud accounts which is termed as phishing. For this reason, against this cyber crime, last two decades have witnessed a variety of combatting methodologies like HTML content based similarity analysis, URL based classification and recently visual similarity based matching since phishing web pages visually mimic to their legitimate counterparts in order to create an illusion to deceive innocent users. To this end, in this study, we propose a computer vision and machine learning based approach in order to classify whether a suspicious web page is phishing and further recognize its original brand name. In this regard, we have utilized and investigated two different local image descriptors namely Scale Invariant Feature Transform (SIFT) and DAISY. Apart from their common properties such as scale invariance, the aforementioned descriptors have apparent differences such that in addition to rotational invariance, SIFT employs key-point based sampling whereas DAISY applies dense sampling by default. Therefore, we first aimed to investigate the feasibility of these two local image descriptors in addition to revealing the effects of sampling strategy and rotational invariance in problem domain. Furthermore, in order to create a discriminative representation of a web page, we followed the bag of visual words (BOVW) approach having different vocabulary sizes such as 50, 100, 200 and 400. In order to evaluate the proposed approach, we have utilized a publicly available phishing dataset including snapshots of webpages sampled from both 14 different highly phished brands and ordinary legitimate web pages yielding a challenging open-set problem. The aforementioned dataset involves 1313 training and 1539 testing image samples in total. The visual features extracted via SIFT and DAISY were first transformed to a BOVW histogram and fed to three different machine learning methods such as SVM, Random Forest and XGBoost. According to the conducted experiments, based on a 400-D visual vocabulary, SIFT descriptor along with XGBoost has been found as the best descriptor-learner configuration having reached up to 89.34% validation accuracy with 0.76% false positive rate. Moreover, SIFT has outperformed DAISY descriptor in all settings. As a result, it has been shown that SIFT descriptors equipped with BOVW representation can be effectively used for brand identification of phishing web pages.

Vector of Locally Aggregated Descriptors (VLAD) method, which aggregates descriptors and produces a compact image representation, has achieved great success in the field of image classification and retrieval. However, the original VLAD method is a hard assignment strategy that only assigns each descriptor to the nearest neighbor visual word in dictionary, which leads to large quantization error. In this paper, improved VLAD based on adaptive bases and saliency weights is proposed to solve the above problem. The new method considers the local density distribution when assigning local descriptors, adaptively selects several nearest neighbor visual words, and takes the coding coefficients obtained by utilizing saliency as the weights of the selected visual words. Experimental results on Corel 10, 15 Scenes and UIUC Sports Event datasets show that the new coding method proposed in this paper achieves better classification performance compared with the existing five VLAD based methods and two commonly used representation methods.

Histogram Intersection Kernel Support Vector Machines (SVM) was used for the image classification problem. Specifically, each image was split into blocks, and each block was represented by the Scale Invariant Feature Transform (SIFT) descriptors; secondly, k-means cluster method was applied to separate the SIFT descriptors into groups, each group represented a visual keywords; thirdly, count the number of the SIFT descriptors in each image, and histogram of each image should be constructed; finally, Histogram Intersection Kernel should be built based on these histograms. In our experimental study, we use Corel-low images to test our method. Compared with typical RBF kernel SVM, the Histogram Intersection kernel SVM performs better than RBF kernel SVM.

There is a growing trend for using content‐based image retrieval (CBIR) systems these days because of the constantly growing interest in digital content. Therefore, the ability of the CBIR to perform the CBIR process will depend on the feature extraction process and its basis, for the retrieval will be done on. Numerous researchers put forward various techniques for feature extraction to enhance the nature of the system. Since features play a very key role in enhancing performance, various features can be used collectively to attain the requisite goal. To retain this in mind, we present in this paper a multifeature fusion system, where three features are integrated and form one feature to improve the situation of retrieval. For this purpose, scale‐invariant feature transform (SIFT), speeded‐up robust features (SURF), and histogram of oriented gradients (HOG) features are adopted. These features are common features that deliver information about the shape of the object and for matching purposes, two techniques of distance matching such as Euclidean and Hausdrauff distance are adopted. To assess the performance of the proposed multifeature‐based CBIR approach, experiments were conducted with the usage of a MATLAB simulator. The Corel‐1000 dataset, consisting of 10,000 images in 100 semantic classes, turned into applied, with each magnificence containing 100 images. A subset of 2500 images across 50 semantic classes was used to train the system. This research aligns with industry, innovation, and infrastructure by contributing to advancements in image processing and retrieval systems. Key characteristic descriptors, along with SIFT, SURF, HOG, texture, and multicharacteristic combinations, were extracted for retrieval functions. The results display that the usage of the Hausdrauff distance as a similarity degree outperforms Euclidean distance, accomplishing retrieval accuracies of 80.02% for HOG, 77.9% for SIFT, 79. 8% for SURF, 77.2% for texture, and 84.2% for multicharacteristic combinations, surpassing Euclidean distance results via 1.7%–3.6% across capabilities. These findings underscore the effectiveness of Hausdrauff distance in enhancing retrieval precision within the CBIR framework.

The MPEG compact descriptors for visual search (CDVS) is a standard toward image matching and retrieval. To achieve high retrieval accuracy over a large scale image/video dataset, recent research efforts have demonstrated that employing extremely high-dimensional descriptors such as the Fisher vector (FV) and the vector of locally aggregated descriptors (VLAD) can yield good performance. Since the FV (or VLAD) possesses high discriminability but small visual vocabulary, it has been adopted by CDVS to construct a global compact descriptor. In this paper, we study the development of global compact descriptors in the completed CDVS standard and the emerging compact descriptors for video analysis (CDVA) standard, in which we formulate the FV (or VLAD) compression as a resource-constrained optimization problem. Accordingly, we propose a codebook-free aggregation method via dual selection to generate a global compact visual descriptor, which supports fast and accurate feature matching free of large visual codebooks, fulfilling the low memory requirement of mobile visual search at significantly reduced latency. Specifically, we investigate both sample-specific Gaussian component redundancy and bit dependency within a binary aggregated descriptor to produce compact binary codes. Our technique contributes to the scalable compressed Fisher vector (SCFV) adopted by the CDVS standard. Moreover, the SCFV descriptor is currently serving as the frame-level hand-crafted video feature, which inspires the inheritance of CDVS descriptors for the emerging CDVA standard. Furthermore, we investigate the positive complementary effect of our standard compliant compact descriptor and deep learning based features extracted from convolutional neural networks with significant mean average precision gains. Extensive evaluation over benchmark databases shows the significant merits of the codebook-free binary codes for scalable visual search.

Image registration is an essential step in many remote-sensing (RS) applications. This article presents a study of a multisource image automatic registration system (MIARS) based on the scale-invariant feature transform (SIFT), which has been demonstrated to be the most robust local invariant feature descriptor for automatically registering various RS images. The SIFT descriptor has two shortcomings: it is unsuitable for extremely large images and has an irregular distribution of feature points. Therefore, three steps are proposed for the MIARS: image division, histogram equalization and the elimination of false point matches by a subregion least squares iteration. Image division makes it possible to use the SIFT descriptor to extract control points from an extremely large RS image. Histogram equalization in prematching improves the contrast sensitivity of RS images. The subregion least squares iteration refines the registration accuracy. Images from multisensor systems, including Quickbird, IRS-P6, Landsat/TM, HJ-CCD, HJ-IRS, light detection and ranging (LiDAR) intensity images and aerial data, were selected to test the reliability of the MIARS. The results indicated that better registration accuracy was achieved, which will be very helpful in the future development of a registration model.

This paper provides a new feature extraction method for object recognition using PCA-KNN algorithm with SIFT descriptor. The proposed method is divided into three steps. The first step is based on feature extraction from the input images using SIFT (Scale Invariant Feature Transform) descriptor. Each of the features is represented using one or more feature descriptors. In medical systems images used as patterns are also represented by feature vectors. In the second step eigen values and eigen vectors are extracted from each image. We apply PCA algorithm after we reduce the number of features by SIFT algorithm. The goal is to extract the important information as a set of new orthogonal variables called principal components. In the final step a nearest neighbor classifier is designed for classifying the images based on the extracted features. The algorithm is experimented in MATLAB and tested with the Caltech 101 database and the experimental results are shown.

In this work, an automated system is designed to identify and classify the modality of medical images. We considered six modalities in this work: X-ray (XR), computed tomography (CT), magnetic resonance imaging (MR), positron emission tomography (PET), ultrasound (US) and photographs (PX). The methodology is based on encoding scale invariant feature transform (SIFT) features using Bag of Visual Words (BoVW), vector of locally aggregated descriptors (VLAD) and Fisher vector (FV). The encoded features are fed to support vector machine (SVM) classifier for training. The classification accuracy of all the classifiers based on three encoding strategies is compared and analyzed. The hybrid model is then implemented by selecting the best performance from each case. The major contribution of this research work is the application of VLAD for modality classification task which has not been tried so far. Combining the best performance of three encoding strategies, the overall classification accuracy obtained with the proposed system is 90.7%. For identification task, the scores from all the three encoding strategies are combined and the recognition rate obtained is 77.7%.

Recognizing human actions in video has gradually attracted much attention in computer vision community, however, it also faces many realistic challenges caused by background clutter, viewpoint changes, variation of actors appearance. These challenges reflect the difficulty of obtaining a clean and discriminative video representation for classification. Recently, VLAD (Vector of Locally Aggregated Descriptors) has shown to be a simple and efficient encoding scheme to obtain discriminative video representations. However, VLAD uses only the nearest visual word in codebook to aggregate each descriptor feature no matter whether it is appropriate or not. Inspired by visual word ambiguity and salience encoding in image classification, we propose Uncertain VLAD (UVLAD) encoding scheme which aggregates each local descriptor feature by considering multiple nearest visual words. The proposed UVLAD scheme ensures each descriptor to be aggregated or discarded appropriately. We evaluate our method on two different benchmark datasets: KTH, and YouTube. Results from experiments show that our encoding scheme outperforms the state-of-arts methods in most cases.

Scale-Invariant Feature Transform (SIFT) is one of the most popular local image features that are widely used in computer vision, image processing and image retrieval. In this paper we study the relation between the SIFT descriptor and its matching accuracy. We propose a method to quantitatively assess the quality of a SIFT feature descriptor in terms of robustness and discriminability. This would enable us to gain a better understanding of the strength and limitations of SIFT in emerging applications of SIFT-based image hash, and also to improve matching accuracy and efficiency in applications such as object search. The experimental results demonstrate the effectiveness of the proposed method.

Recently, the vector of locally aggregated descriptor (VLAD) has shown its great effectiveness in diverse computer vision tasks including image retrieval, Scene classification, and action recognition. Its great success stems from its powerful representation ability and computational efficiency. However, it remains unclear about its theoretical foundation and how it is connected to basic while important algorithms, e.g., the bag-of-words model and match kernels, and how its performance is affected by parameter configurations, e.g., normalization and pooling, which are also widely used in state-of-the-art algorithms based on local features. In this paper, with an aim to achieve the full capacity of VLAD, we conduct a comprehensive and in-depth study from both theoretical analysis and experimental practice perspectives. As a theoretical contribution, we provide a new formulation of VLAD via match kernels, which serves to connect VLAD with existing important encoding methods based on local features. As a contribution to the practical use of VLAD, we comprehensively investigate the roles and effects of the two widely-used operations in local feature encoding: normalization and pooling. To the best of our knowledge, our work provides the first comprehensive study on VLAD, which will not only enable a full understanding of it but also provide an important guidance for state-of-the-art algorithms based on local features. We have conducted extensive experiments on three benchmark datasets: Scene-15, Caltech 101 and PPMI for both image classification and action recognition.