Efficient Image Retrieval with Vision Transformer (ViT) and FAISS

While general object recognition is difficult, it is relatively easy to capture various primitive properties such as color distributions, prominent regions and their topological features from an image and use them to narrow down the search space when attempts to retrieving images by contents from an image database are made. In this paper, we present an image database in which images are indexed and retrieved based on color histograms. We first address the problems inherent in color histograms created by the conventional method, and then propose a new method to create histograms which are compact in size and insensitive to minor illumination variations such as highlight, shape, and etc. A powerful indexing scheme where each histogram of an image is encoded into a numerical key, and stored in a two-layered tree structure is introduced. This approach turns the problem of histogram matching, which is computation intensive, into index key search, so as to realize quick data access in a large scale image database. Two types of user interfaces, Query by user provided sample images, and Query by combination of the system provided templates, are provided to meet various user requests. Various experimental evaluations exhibit the effectiveness of the image database system.

Image retrieval with relevant feedback on large and high-dimensional image databases is a challenging task. In this paper, we propose an image retrieval method, called BCFIR (Binary Codes for Fast Image Retrieval). BCFIR utilizes sparse discriminant analysis to select the most important original feature set, and solve the small class problem in the relevance feedback. Besides, to increase the retrieval performance on large-scale image databases, in addition to BCFIR mapping real-valued features to short binary codes, it also applies a bagging learning strategy to improve the ability general capabilities of autoencoders. In addition, our proposed method also takes advantage of both labeled and unlabeled samples to improve the retrieval precision. The experimental results on three databases demonstrate that the proposed method obtains competitive precision compared with other state-of-the-art image retrieval methods.

Online image repositories such as Flickr contain hundreds of millions of images and are growing quickly. Along with that the needs for supporting indexing, searching and browsing is becoming more and more pressing. In this work we will employ the image content as a source of information to retrieve images. We study the representation of images by Latent Dirichlet Allocation (LDA) models for content-based image retrieval. Image representations are learned in an unsupervised fashion, and each image is modeled as the mixture of topics/object parts depicted in the image. This allows us to put images into subspaces for higher-level reasoning which in turn can be used to find similar images. Different similarity measures based on the described image representation are studied. The presented approach is evaluated on a real world image database consisting of more than 246,000 images and compared to image models based on probabilistic Latent Semantic Analysis (pLSA). Results show the suitability of the approach for large-scale databases. Finally we incorporate active learning with user relevance feedback in our framework, which further boosts the retrieval performance.

Content-based techniques enable retrieval of remotely sensed data based on low-level features. However, the deep gap between low-level features and high-level semantics concepts is a major obstacle to more effective image retrieval. Therefore, a semantics-based retrieval approach was implemented. The semantics classifiers are trained using heterogeneous features from a group of satellite images. The proposed approach is mainly composed of two steps. The first step is to form hierarchical semantics classifiers based on the low-level features of the training images. In the second step, unknown satellite images are classified into a certain semantics class if their feature vectors are located in the corresponding feature space. To achieve an effective and at the same time efficient identification of the multiple semantics classes within the satellite scenes, an approximation approach was developed. At the query time, the system retrieves the satellite images based on semantics classes extracted from the query image or provided directly by the users

The use of deep neural networks is a recent research area in content-based image retrieval that has enhanced results on many datasets and outperformed handcrafted for fine-tuning of the network. The image feature is computed by combining different feature descriptors like shape with color and texture, so that it describes the image more effectively. In this paper, image retrieval on database images is done to get the top-k most similar database images using kNN on the image embeddings with cosine similarity as the distance metric. Two unsupervised methods are performed by generating image embeddings using a pre-trained network such as simple autoencoder, convolution autoencoder, and transfer learning such as VGG19, VGG16, and ResNET152. This is done by removing its last few layers and performing inference on our image vectors for the generation of flattened embeddings. No training is needed throughout this entire processing, only the loading of the pre-trained weights is needed. We train both a simple autoencoder and a convolutional autoencoder on our database images with the objective of minimizing reconstruction loss. After sufficient training, we extract the encoder part of the autoencoder and use it during inference to generate flattened embeddings. This work is using transfer learning and autoencoder-based retrieval system with classification. Experiments are carried out on flower and fruit datasets. Results are recorded using GPU on Google Colaboratory platform. Visualization is shown using “t-distributed stochastic neighbor embedding (t-SNE)”.

Automatic image classification is the first step toward semantic understanding of an object in the computer vision area. The key challenge of problem for accurate object recognition is the ability to extract the robust features from various viewpoint images and rapidly calculate similarity between features in the image database or video stream. In order to solve these problems, an effective and rapid image classification method was presented for the object recognition based on the video learning technique. The optical-flow and RANSAC algorithm were used to acquire scene images from each video sequence. After the selection of scene images, the local maximum points on corner of object around local area were found using the Harris corner detection algorithm and the several attributes from local block around each feature point were calculated by using scale invariant feature transform (SIFT) for extracting local descriptor. Finally, the extracted local descriptor was learned to the three-dimensional pyramid match kernel. Experimental results show that our method can extract features in various multi-viewpoint images from query video and calculate a similarity between a query image and images in the database.

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.

Content-Based Image Retrieval (CBIR) is an essential tool for arranging and acquiring visual content from large-scale image databases. This research presents a robust hybrid CBIR structure that combines transformer-based deep feature extraction with Genetic Algorithm (GA) optimization to significantly improve retrieval accuracy and efficiency. The proposed system introduces Vision Transformers (ViT) to efficiently capture intricate, global visual figures over the distinctive image categories, supporting both single and multi-object image retrieval scenarios. By influencing the long-range dependency modelling abilities of transformers, the system extracts highly different feature representations. These elements are further optimized with the help of Genetic Algorithm, a powerful adaptive technique that efficiently enhances feature selection and matching through iterative selection, crossover, and mutation processes. Comprehensive experiments were performed on the Corel 1K benchmark dataset illustrates the proposed hybrid model surpasses conventional CBIR model in terms of precision, recall, accuracy, and F1-score. The system achieves a retrieval accuracy of 99.38%, an F1-score of 95.12%, and a reduced error rate of 0.62%, showcasing its superior retrieval performance and computational efficiency. The results highlight the potential of integrating transformer-based deep learning with evolutionary optimization in advancing modern CBIR systems.

Research in content-based image retrieval today is a lively discipline, expanding in breadth. Discovering new and innovative processes of locating a desired image from an expanding collection of images has been a major area of interest for many professional fields. This has led to the increasing use of a technique called Content Based Image Retrieval which is used to retrieve images based on low-level features which are automatically driven such as color, shape and texture. CBIR draws many of its methods from the field of image processing and computer vision and can be regarded as subset of that field. Using this technique, the entire database may be searched to find the most closely matching image. In this paper, color and texture features of an image have been used to retrieve all the similar images from the image database. For this purpose, Block Color Histogram as color feature and Gray-level Co-occurrence Matrix (GLCM) and Gabor wavelets as texture features have been used. The proposed approach is very efficient since it uses the prominent and distinct features of an image useful for effective image retrieval using fuzzy heuristics according to human visual system.

Content-based image retrieval is a method of retrieving images based on its semantic features. Classical CBIR models in the literature uses low-level features like color, shape, and texture to retrieve similar images from the database. Low-level features work well when the dataset is small; however, it fails to reduce the semantic gap when the image database becomes large and diverse. In the recent past, many researchers have exploited the usage of pretrained neural network as a feature extractor capable of giving a high-level feature representation for images. In this chapter, we use the output vectors of intermediate layers of an Inception Resnet deep learning model as the feature representation for image retrieval. The high-level feature vectors obtained from the deep layers of Inception Resnet are clustered using K-means clustering algorithm to make the retrieval process faster. The second part of the chapter explores various distance metrics to find most similar images corresponding to the query image. Based on the analysis on various distance measures, it is found that Hamming distance performs better than Euclidean distance in retrieving similar images when using deep features owing to the sparse nature of the feature vectors produced by the intermediate deep layers of Inception Resnet.KeywordsContent-based image retrieval Deep learning Hamming distance K-means clustering Deep features for CBIR Image retrieval

Sketch based image retrieval using grid approach on large scale database

Image retrieval is an important problem in multimedia systems. It is determined as the process of searching and fetching images from a dataset. Content-based Image Retrieval (CBIR) is a significant and challenging field of research in digital image processing. The CBIR system's essential requirement is to retrieve the relevant information following a query image with higher system output from a large image database. Unfortunately, not all the methods are suitable to be used to get high accuracy of retrieval. Therefore, this research aims to classify the data of query image with the data of image database to get a similar image retrieval using Support Vector Machine (SVM) and validate its accuracy based on the classification for the performance evaluation using the precision-recall measure. The critical point of SVM is to get an optimal hyperplane that separates the data points into two classes. This method was applied to different image databases because the classified-based proposed scheme proved better performance than various existing methods. This project assessed experimental results toward 500 images of the Caltech-256 image dataset to demonstrate the proposed method. For retrieving a similar image following the query image, 20 images from five classes were successfully retrieved by using this method. It will show that the SVM method's average accuracy from all image classes is 94.79%.

Content-based image retrieval is an active and fast advancing research area since the 1990s as a result of advances in the Internet and new digital image sensor technologies. However, many challenging research problems continue to attract researchers from multiple disciplines. Content-based image retrieval uses the visual contents of an image as features to represent and index the image to be searched from large scale image databases. The quality of the selected features relies mainly on the degree of the invariance property that is ensured under acceptable manipulations. This paper proposes an efficient method for compactly representing color and texture features and combining them for image retrieval. The performance of retrieval based on these compact descriptors obtained by the proposed techniques is analyzed and tested on wang database images yielding satisfactory accuracy rates. A comparative study demonstrated that the developed feature extraction scheme outperformed the other schemes being compared with.

Content Based Image Retrieval (CBIR) plays a significant role in identifying the similarity of images with large datasets. It is identified based on the size, colour, and texture features of the image. But in such conditions, it is complex to determine the features of query images in large datasets and does not show accurate similarity when compared with every image in the retrieval process. In order to perform an efficient similarity of images, a novel Machine Learning (ML) approach Kernelized Radial Basis Auto-Encoder Function Neural Network (Ker_RadBAEFNN) technique is proposed that performs the individual image classification in the retrieval process. Moreover, the neural networks are optimised based on the reinforcement process and perform the extraction process regarding individual images. Further,reinforcement-based optimisation estimates the images in neural networks for undertaking an automatic feature extraction of query images. The performance of the classification process is validated based on MNIST, METU, and COCO datasets that determined the efficiency of the recognition and classification process of image retrieval. The experimental analysis is carried out based on various measures such as accuracy, precision, recall, F1-score, RMSE, and MAPE for the proposed and existing GLCM-ABC, PSO-ANN, IRB-CNN, FAGWO, and OCAM methods. The analysis shows that the performance of the proposed attained better effectiveness with attained accuracy by 98% and diminished for state-of-the-art techniques as 92%, 95%, 94%, 96.8%, as well as 96%, respectively. Compared to existing methods, the accuracy rate of the proposed method is maximised by 1.3%.

The increasing number of mobile phones equipped with powerful cameras leads to huge collections of user-generated images. To utilize the information of the images on site, image retrieval systems are becoming more and more popular to search for similar objects in an own image database. As the computational performance and the memory capacity of mobile devices are constantly increasing, this search can often be performed on the device itself. This is feasible, for example, if the images are represented with global image features or if the search is done using EXIF or textual metadata. However, for larger image databases, if multiple users are meant to contribute to a growing image database or if powerful content-based image retrieval methods with local features are required, a server-based image retrieval backend is needed. In this work, we present a content-based image retrieval system with a client server architecture working with local features. On the server side, the scalability to large image databases is addressed with the popular bag-of-word model with state-of-the-art extensions. The client end of the system focuses on a lightweight user interface presenting the most similar images of the database highlighting the visual information which is common with the query image. Additionally, new images can be added to the database making it a powerful and interactive tool for mobile contentbased image retrieval.