A New Content-Based Image Retrieval System Using Deep Visual Features

In content based image retrieval (CBIR) process, every image has been represented in a compact set of local visual features i.e. color, texture, and/or shape of images. This set of local visual features is known as feature vector. In the CBIR process, feature vectors of images have been used to represent or to identify similar images in adequate way. As a result, feature vector construction has always been considered as an important issue since it must reflect proper image semantics using minimal amount of data. The proposed CBIR scheme is based on the combination of color and texture features. In this work initially, we have converted the given RGB image into HSV color image. Subsequently, we have considered H (hue), S (saturation), and V (intensity) components for extraction of visual image features. The texture features have been extracted from the V component of the image using 2D dual-tree complex wavelet transform (2D DT-CWT) where it analyzes the textural patterns in six different directions i.e. ±15\(^{\circ }\), ±45\(^{\circ }\), and ±75\(^{\circ }\). At the same time, we have computed the probability histograms of H and S components of the image respectively and subsequently those are divided into non-uniform bins based on cumulative probability for extraction of color based features. So, in this work both the color and texture features have been extracted simultaneously. Finally, the obtained features have been concatenated to attain the final feature vector and same is considered in image retrieval process. We have tested the novelty and performance of the proposed work in two Corel, two objects, and, a texture image datasets. The experimental results reveal the acceptable retrieval performances for different types of datasets.

Semi-automatic or automatic image indexation emerged because manual image indexation is slow and tedious. Generally, this first indexation is used as part of a content-based image retrieval system (CBIR). To have a powerful CBIR system, it is necessary to be concerned with three main facets: 1) the choice of the descriptors (based on shape, colour and texture and/or a combination between them); 2) the process of indexation and finally; 3) the retrieval process. In this work, we focus mainly on an indexing based on genetic algorithm and particle swarm optimisation (PSO) algorithm. We chose an optimal combination of colour, shape and texture (PCM: powerful combination method) descriptors. The fruit of our research work is implemented in a system called image search engine (ISE) which showed a very promising performance. In fact, the performance evaluation of the PCM method of our descriptors combination showed upgrades of the average precision metric from 66.6% to 89.30% for the 'food' category colour histogram, from 77.7% to 100% concerning CCV for the 'flower' category, and from 44.4% to 87.65% concerning the co-occurrence matrix for the 'building' category using the Corel dataset. Likewise, our ISE system showed much more interesting performance compared to what was shown in previous works.

In the field of computer vision and image processing CBIR (Content based image retrieval) system become an active research area. The CBIR technique involves feature extraction based on image features (like color, shape and texture) and defining rules for comparing image. A classifier is also used by authors for image classification. For this purpose an image database taken from James Z. Wang research group website is used by many authors. Due to enormous increase in the size of database traditional methods of image retrieval by using keywords does not provide accurate result. Thus the concept of CBIR system is emerged. In last couple of years many method of feature extraction is developed but since now researcher are not able to develop a system which has gained high accuracy of human visual perception in distinguishing pictures. Therefore earlier researchers have concentrated to increase the retrieval accuracy of CBIR system by using different combination of features. The objective of this paper is to present review on a CBIR technique which combined four kinds of features set including every kind of feature for developing feature vector. Color based features (histogram), statistical features (mean, median and standard deviation), spatial features (Gabor feature and wavelet features) and texture features (GLCM) is used for forming feature vector. A SVM classifier is used with CBIR system to increase retrieval accuracy. SVM (support vector machine) is a machine learning classifier which provides good classification accuracy. Keywords: CBIR, SVM, NN, GLCM, HSV Color space, Euclidean distance matrix.

As the use of digital media increases, effective retrieval and management techniques become more important. In order to extract useful information from this huge amount of data, many content based image retrieval (CBIR) systems have been developed. A typical CBIR system captures image features such as color, texture, or shape of objects in the query image and tries to retrieve images from the database with similar features. The main advantage of CBIR systems with relevance feedback is that these systems take into account the gap between the high-level concepts and low-level features and subjectivity of human perception of visual content. In this paper, a CBIR system based on color histogram has been presented and a new approach for image storage and retrieval called association-based image retrieval (ABIR) has been studied. The results of both systems have been shown.

Different types of search engines are used to search image and text contents. Two types of image search methods are available in the Internet. They are query keyword based model and content based image retrieval models. Text query strings are used in the textual image retrieval model. Content based image retrieval (CBIR) model uses the visual information of the images. Image search methods use the text annotation and image visual features. Google image search and Bing image search engines are used to fetch images from the web. Image query string is used to search image on Internet. One click query image selection method is used to submit user intention for image retrieval. Content based image re-ranking is performed with visual and textual similarity metrics. Adaptive Weight Schema is used for the similarity analysis. Feature weight learning algorithm is applied to estimate feature weights for the images and its category. Query is expanded with keyword and visual information. Rank boost framework algorithm is enhanced to rank images with photographic quality. Content similarity and visual quality factors are used for the re-ranking process. In this paper, we propose an image indexing and retrieval using speech annotations based on a predefined structured syntax. To improve the retrieval effectively, N-best lists for index generation is used .so, a query expansion technique is explored to enhance the query terms. All this process is automatic, without extra effort from the user. This is critically important in web-based image search engine for any commercial, where the user interface has to be extremely simple.

Content Based Image Retrieval (CBIR) is a process in which for a given query image similar images will be retrieved based on the image content similarity. Image content refers to its visual features, which are mathematical representations of a digital image. The image retrieval task primarily depends on image feature extraction and similarity measurement between the feature vectors. The performance of CBIR process not only decided by the optimum features extracted from the image but also on the proper choice of the similarity/dissimilarity measures (distance metrics). As the image features broadly diversified in terms of color, texture and shape based, using the same distance metric may not work well for all of these features. In this paper, first we presented overview of geometric and statistical distance metrics used in CBIR along with the comparative analysis of these measures on color and texture features. Color features extracted by computing color histograms in HSV space and texture features by wavelet decompositions. Geometrical distances such as Manhattan, Chebyshev, Euclidean, statistical distance metrics such as Cosine Similarity, Chi-square, KullbackLeibler, Jeffrey and cumulative statistical distance metrics of Kolmogorov-Smirnov, Cramer von Mises and Earthmover's distances were analyzed for feature similarity. We gave certain conclusions on the performance of all these distance metrics in terms of Mean Average Precision (MAP) and Recall rates with color and texture features respectively.

Image retrieval or content-based image retrieval (CBIR) can be transformed into the calculation of the distance between image feature vectors. The closer the vectors are, the higher the image similarity will be. In the image retrieval system for large-scale dataset, the approximate nearest-neighbor (ANN) search can quickly obtain the top k images closest to the query image, which is the Top-k problem in the field of information retrieval. With the traditional ANN algorithms, such as KD-Tree, R-Tree, and M-Tree, when the dimension of the image feature vector increases, the computing time will increase exponentially due to the curse of dimensionality. In order to reduce the calculation time and improve the efficiency of image retrieval, we propose an ANN search algorithm based on the Product Quantization Table (PQTable). After quantizing and compressing the image feature vectors by the product quantization algorithm, we can construct the image index structure of the PQTable, which speeds up image retrieval. We also propose a multi-PQTable query strategy for ANN search. Besides, we generate several nearest-neighbor vectors for each sub-compressed vector of the query vector to reduce the failure rate and improve the recall in image retrieval. Through theoretical analysis and experimental verification, it is proved that the multi-PQTable query strategy and the generation of several nearest-neighbor vectors are greatly correct and efficient.

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.

To support content based image retrieval, MPEG-7 is developed to define the content interfaces for images. In MPEG-7, Dominant Color Descriptor (DCD) is considered as the most important feature, and is widely used to describe the color features of an image. To support semantic queries from users, we proposed a color feature semantic mapping method in this work, which can translate the DCD values into semantic color names. The semantic mapping method is realized by constructing a mapping table between the DCD values and the semantic color names. To validate the effectiveness of our mapping method, an image retrieval experiment is conducted. From the comparison with the manually indexed description, the proposed mapping method is proved to be effective by the experiment results. Our work is very important to automatically generate the semantic description of an image and then support the users’ semantic retrieval queries.

Content-based image retrieval (CBIR) systems utilize low level query image feature as identifying similarity between a query image and the image database.Image contents are plays significant role for image retrieval.There are three fundamental bases for content-based image retrieval, i.e. visual feature extraction, multidimensional indexing, and retrieval system design.Each image has three contents such as: color, texture and shape features.Color and texture both plays important image visual features used in Content-Based Image Retrieval to improve results.Color histogram and texture features have potential to retrieve similar images on the basis of their properties.As the feature extracted from a query is low level, it is extremely difficult for user to provide an appropriate example in based query.To overcome these problems and reach higher accuracy in CBIR system, providing user with relevance feedback is famous for provide promising solution.

As we all know, the content-based image retrieval (CBIR) is very time-consuming due to the extraction and matching of high dimensional and complex features. The traditional CBIR systems could not respond to a very large number of retrieval requests at mean time, which are submitted from the Internet. In this paper, we propose a content-based parallel image retrieval system to achieve high responding ability. Our system is developed on cluster architectures. It has several retrieval servers to supply the service of content-based image retrieval. Our system adopts the Browser/Server (B/S) mode. The users could visit our system though web pages. Our system uses the Symmetrical Color-Spatial Features (SCSF)[25] to represent the content of an image. The SCSF is effective and efficient for image matching because it is independent of image distortion such as rotation and flip as well as it increases the matching accuracy. The SCSF was organized by M-tree[16], which could speedup the searching procedure. Our experiments show that the image matching is quickly and efficiently with the use of SCSF. And with the support of several retrieval servers, the system could respond to many users at mean time. (Abstract)

Color coherence vectors (CCV) are commonly used for image retrieval in content-based image retrieval (CBIR). It is a more sophisticated form of histogram refinement, in which histogram buckets are partitioned based on spatial coherence, but a single CCV is not enough for efficient and robust image retrieval system. In this paper, a method for image retrieval using the multi-CCV mixing location is proposed. In this method, we separate the image into core region and minor region, adopt hash table to store the pixel into the appropriate color of our color set, and mixing location information with the traditional method to bring multi-CCV of the image. The multi-CCV mixing location gives much better retrieval efficiency, but with higher computations and appear to perform better than the traditional CCV.

In content-based image retrieval (CBIR), one of the most challenging and ambiguous tasks are to correctly understand the human query intention and measure its semantic relevance with images in the database. Due to the impressive capability of visual saliency in predicting human visual attention that is closely related to the query intention, this paper attempts to explicitly discover the essential effect of visual saliency in CBIR via qualitative and quantitative experiments. Toward this end, we first generate the fixation density maps of images from a widely used CBIR dataset by using an eye-tracking apparatus. These ground-truth saliency maps are then used to measure the influence of visual saliency to the task of CBIR by exploring several probable ways of incorporating such saliency cues into the retrieval process. We find that visual saliency is indeed beneficial to the CBIR task, and the best saliency involving scheme is possibly different for different image retrieval models. Inspired by the findings, this paper presents two-stream attentive CNNs with saliency embedded inside for CBIR. The proposed network has two streams that simultaneously handle two tasks. The main stream focuses on extracting discriminative visual features that are tightly related to semantic attributes. Meanwhile, the auxiliary stream aims to facilitate the main stream by redirecting the feature extraction to the salient image content that human may pay attention to. By fusing these two streams into the Main and Auxiliary CNNs (MAC), image similarity can be computed as the human being does by reserving conspicuous content and suppressing irrelevant regions. Extensive experiments show that the proposed model achieves impressive performance in image retrieval on four public datasets.

In a content-based image retrieval (CBIR) system, indexing feature vectors and the similarity measure between feature vectors are two key factors for retrieval performance. We present a new CBIR system with statistical-model based image feature extraction in the wavelet domain and a Kullback divergence based similarity measure. A two component Gaussian mixture model (GMM) in the wavelet domain is employed and the model parameters are used to form features for image indexing. A new Kullback divergence based similarity measure is then presented for image retrieval. The experimental results demonstrate that the similarity measure based on the Kullback divergence is more effective than conventional similarity measures, such as the city-block distance and the Euclidean distance. It is shown that the new CBIR system, with the combination of the GMM and the new Kullback divergence based similarity measure, outperforms most other methods in retrieval performance for texture images, while keeping a comparable level of computational complexity.

Effective image retrieval using dominant color descriptor and fuzzy support vector machine