A FLEXIBLE SUB-BLOCK IN REGION BASED IMAGE RETRIEVAL BASED ON TRANSITION REGION

A new indexing system for multiresolution image retrieval has been developed. The system analyzes not only whole images but also--for the first time--local features present in the images. This capability is important in managing large-scale image systems such as video databases and medical image databases. Key to the method is the use of an innovative wavelet, called the W-transform. This transform is characterized by its ability to handle signals of arbitrary size. Equally important, the W-transform is not constrained by assumptions about periodicity. That is, the transformed signal in some location is related only to the signals at its neighboring location. These properties make the W-transform a natural tool for analyzing local image features. A brief discussion is given of how the system works. The feature-oriented image indexing and retrieval method offers two major advantages over traditional multiresolution histogram comparison methods. The response time for retrieving image coarse components is considerably faster than that for retrieving whole images--thereby dramatically speeding image retrieval over the Ethernet. Moreover, the new method can be used to search images that are significantly different from the query image as a whole but contain local features identical with or similar to those in the query image.

The increasing number of mobile phones equipped with powerful cameras leads to huge collections of user-generated images. To utilize the information of the images on site, image retrieval systems are becoming more and more popular to search for similar objects in an own image database. As the computational performance and the memory capacity of mobile devices are constantly increasing, this search can often be performed on the device itself. This is feasible, for example, if the images are represented with global image features or if the search is done using EXIF or textual metadata. However, for larger image databases, if multiple users are meant to contribute to a growing image database or if powerful content-based image retrieval methods with local features are required, a server-based image retrieval backend is needed. In this work, we present a content-based image retrieval system with a client server architecture working with local features. On the server side, the scalability to large image databases is addressed with the popular bag-of-word model with state-of-the-art extensions. The client end of the system focuses on a lightweight user interface presenting the most similar images of the database highlighting the visual information which is common with the query image. Additionally, new images can be added to the database making it a powerful and interactive tool for mobile contentbased image retrieval.

This paper presents a new approach for the challenging problem of geo-localization using image matching in a structured database of city-wide reference images with known GPS coordinates. We cast the geo-localization as a clustering problem of local image features. Akin to existing approaches to the problem, our framework builds on low-level features which allow local matching between images. For each local feature in the query image, we find its approximate nearest neighbors in the reference set. Next, we cluster the features from reference images using Dominant Set clustering, which affords several advantages over existing approaches. First, it permits variable number of nodes in the cluster, which we use to dynamically select the number of nearest neighbors for each query feature based on its discrimination value. Second, this approach is several orders of magnitude faster than existing approaches. Thus, we obtain multiple clusters (different local maximizers) and obtain a robust final solution to the problem using multiple weak solutions through constrained Dominant Set clustering on global image features, where we enforce the constraint that the query image must be included in the cluster. This second level of clustering also bypasses heuristic approaches to voting and selecting the reference image that matches to the query. We evaluate the proposed framework on an existing dataset of 102k street view images as well as a new larger dataset of 300k images, and show that it outperforms the state-of-the-art by 20 and 7 percent, respectively, on the two datasets.

In person re-identification, extracting image features is an important step when retrieving pedestrian images. Most of the current methods only extract global features or local features of pedestrian images. Some inconspicuous details are easily ignored when learning image features, which is not efficient or robust to for scenarios with large differences. In this paper, we propose a Multi-level Feature Fusion model that combines both global features and local features of images through deep learning networks to generate more discriminative pedestrian descriptors. Specifically, we extract local features from different depths of network by the Part-based Multi-level Net to fuse low-to-high level local features of pedestrian images. Global-Local Branches are used to extract the local features and global features at the highest level. The experiments have proved that our deep learning model based on multi-level feature fusion works well in person re-identification. The overall results outperform the state of the art with considerable margins on three widely-used datasets. For instance, we achieve 96% Rank-1 accuracy on the Market-1501 dataset and 76.1% mAP on the DukeMTMC-reID dataset, outperforming the existing works by a large margin (more than 6%).

A key task in computer vision is image retrieval, which has wide-ranging applications across multiple domains. Using query picture features, this abstract proposes an image retrieval approach that focuses on the widely used Scale-Invariant Feature Transform (SIFT) algorithm for feature extraction and distance calculation. The suggested method starts by extracting SIFT features from a set of photos, building a keypoints and descriptor database. By capturing the unique qualities of nearby image regions, these features enable reliable matching and retrieval. The security of private cloud data, which includes query queries, the search tree, and outsourced photographs, is another major issue. Use a feature extraction method first for integrated picture features, which are composed of fundamental components like colour and shape. In particular, because the proposed method uses a balancing index tree, it can achieve logarithmic search time. Second, the picture and query feature are encrypted using the secure inner product. Include a system for determining duplicate picture material as well. When given a query image, SIFT feature extraction is applied to it, producing keypoints and descriptors that correspond to its visual characteristics. Next, a distance computation method like Euclidean distance is used to compare the features of the database photos with the query image features. The similarity between the query image and the database images is measured through this comparison. The retrieved photos are ranked in order of similarity to the query image based on the calculated distances. Search results with lesser distances between images are deemed more similar and are displayed first. To locate visually related images in huge databases, the image retrieval system that uses SIFT feature extraction and distance calculation provides a reliable and effective solution. It contributes to developments in multimedia retrieval, visual analytics, and picture understanding by enabling applications like content-based image search, image recommendation systems, and image clustering.

Illumination distortion due to uncontrolled lighting can severely degrade the color appearance of a photo. Frequently, the desired colors for objects in a newly taken query image are found in a previously stored database image. Then, the goal is to change the colors in the query image to match the colors in the database image. This paper presents a color restoration system that automatically retrieves a database image which matches the query image, even if the two images are taken from different viewpoints and under different illuminations. Robust features enable both accurate retrieval from the database and efficient sampling of the color differences between the query and database images. A spatially varying color mismatch model is generated, and the colors of the query image are effectively restored.

Multimodal biometrics-based systems aim to improve the recognition accuracy of human beings using more than one physical and/or behavioral characteristics of a person. In this paper, different fusion schemes at matching score level and feature level are employed to obtain a robust recognition system using several standard feature extractors. The proposed method involves the consideration of a face–iris multimodal biometric system using score level and feature level fusion. Principal Component Analysis (PCA), subspace Linear Discriminant Analysis (LDA), subpattern-based PCA, modular PCA and Local Binary Patterns (LBP) are global and local feature extraction methods applied on face and iris images. In fact, different feature sets obtained from five local and global feature extraction methods for unimodal iris biometric system are concatenated at feature level fusion called iris feature vector fusion (iris-FVF), while for unimodal face biometric system, LBP is used to achieve efficient texture descriptors. Feature selection is performed using Particle Swarm Optimization (PSO) at feature level fusion step to reduce the dimension of feature vectors for improving the recognition performance. Our proposed method is validated by forming three datasets using ORL, BANCA, FERET face databases and CASIA, UBIRIS iris databases. The results based on recognition performance and ROC analysis demonstrate that the proposed matching score level fusion scheme using Weighted Sum rule, tanh normalization, iris-FVF and facial features extracted by LBP achieves a significant improvement over unimodal and multimodal methods. Support Vector Machine (SVM) and t-norm normalization are also used to improve the recognition performance of the proposed method.

Any city-scale visual localization system has to overcome long-term appearance changes, such as varying illumination conditions or seasonal changes between query and database images. Since semantic content is more robust to such changes, we exploit semantic information to improve visual localization. In our scenario, the database consists of gnomonic views generated from panoramic images (e.g. Google Street View) and query images are collected with a standard field-of-view camera at a different time. To improve localization, we check the semantic similarity between query and database images, which is not trivial since the position and viewpoint of the cameras do not exactly match. To learn similarity, we propose training a CNN in a self-supervised fashion with contrastive learning on a dataset of semantically segmented images. With experiments we showed that this semantic similarity estimation approach works better than measuring the similarity at pixel-level. Finally, we used the semantic similarity scores to verify the retrievals obtained by a state-of-the-art visual localization method and observed that contrastive learning-based pose verification increases top-1 recall value to 0.90 which corresponds to a 2% improvement.

Bag of Features (BoF) has been successfully exploited in content-based image retrieval for several years. Due to its performance and popularity, several extensions have been proposed that involve feature description, dictionary building, feature encoding and post-query process, etc. This paper proposes a dynamically weighting scheme for BoF-based image retrieval based on feedback. It involves two contributions: (i) analyzing the statistical distribution characteristic of similar BoF representations and (ii) computing weights dynamically based on the feedback obtained from different initial query results. We quantitatively evaluate the proposed method on two different databases. Experiments confirm that the proposed weighting scheme has better performance than the baseline of BoF-based image retrieval systems. Meanwhile, the results demonstrate the effectiveness of the weighting scheme in terms of the precision of top-N returned images.

Histogram refinement for texture descriptor based image retrieval

Conv-Swinformer: Integration of CNN and shift window attention for Alzheimer’s disease classification

AEA-Net:Affinity-supervised entanglement attentive network for person re-identification

Interpreting endomicroscopic images is still a significant challenge, especially since one single still image may not always contain enough information to make a robust diagnosis. To aid the physicians, we investigated some local feature-based retrieval methods that provide, given a query image, similar annotated images from a database of endomicroscopic images combined with high-level diagnosis represented as textual information. Local feature-based methods may be limited by the small field of view (FOV) of endomicroscopy and the fact that they do not take into account the spatial relationship between the local features, and the time relationship between successive images of the video sequences. To extract discriminative information over the entire image field, our proposed method collects local features in a dense manner instead of using a standard salient region detector. After the retrieval process, we introduce a verification step driven by the textual information in the database and in which spatial relationship between the local features is used. A spatial criterion is built from the co-occurence matrix of local features and used to remove outliers by thresholding on this criterion. To overcome the small-FOV problem and take advantage of the video sequence, we propose to combine image retrieval and mosaicing. Mosaicing essentially projects the temporal dimension onto a large field of view image. In this framework, videos, represented by mosaics, and single images can be retrieved with the same tools. With a leave-n-out cross-validation, our results show that taking into account the spatial relationship between local features and the temporal information of endomicroscopic videos by image mosaicing improves the retrieval accuracy.

This paper develops a general framework of image retrieval, named A3, by introducing an auxiliary set of samples (object references), each of which is annotated with semantic attributes (tags). Given a query image (without tags), we first map it into the references by a non-convex sparse coding formulation, which jointly optimizes appearance reconstruction of the query and semantics consistency among references. A novel algorithm, namely semantic coherent coding (SCC), is presented to solve this optimization. As a result, a subset of references is selected to augment the query for retrieval. Then we can rank images to be retrieved by intuitively matching them together with the query and the constructed subset. The advantages of our approach are validated on the public datasets (Caltech-256 and ImageNet), and summarized as follows: (1) The auxiliary set of references gives rise to an alternating way for query augmentation while utilizing semantics. (2) The semantics of references can also be propagated into the query and database images.

Retrieval of similar images from large dataset of brain images across patients would help the experts in the decision diagnosis process of diseases. Generally used feature extraction methods are color, texture and shape. In medical images texture and shape features are most efficient. Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP) are good descriptor for brain MR image retrieval. But there are many challenges facing in medical application. An empirical study of the impact of increasing bins number in the HOG descriptor concluded that larger the number is more accurate the descriptor is. In fact this is due to the reduction of orientations range that each bin covers. Despite the efficiency of augmenting the bins number, this technique has limited spatial support as the augmentation of the number of bins used leads to increase the histogram dimension. So here proposed a method called Histogram of Fuzzy Oriented Gradients (HFOG), in which a pixel can belong several bins with different degrees. The Local Binary Patterns feature extraction method is widely used for texture analysis; however, the original LBP is based on hard thresholding the neighborhood of each pixel. Therefore, texture representation with LBP is very sensitive to noise and cannot distinguish between a strong and a weak pattern. In this study, Fuzzy Local Binary Patterns was introduced to improve the original LBP.