AIR-HLoc: Adaptive Retrieved Images Selection for Efficient Visual Localisation

In the Content Based Image Retrieval, accurate image retrieval using different methods and descriptors is an important task. The single descriptor, local or global, gives different results for the same query image, as the local descriptor works on a part of an image and global descriptor works on whole image and to overcome this issue of single descriptor, the fusion technique is introduced. The fusion technique combines the local and global descriptor to improve the performance and it extracts multi-features which are more informative about local and global features for the query image. The proposed method, reduces and manages multi-features for relevant output. The experimentation is done on sample of Holidays dataset and the comparison between fusion, local and global descriptor method is presented. The proposed method outperforms, in terms of precision and recall rate.

As a fundamental yet challenging problem in intelligent transportation systems, point cloud registration attracts vast attention and has been attained with various deep learning-based algorithms. The unsupervised registration algorithms take advantage of deep neural network-enabled novel representation learning while requiring no human annotations, making them applicable to industrial applications. However, unsupervised methods mainly depend on global descriptors, which ignore the high-level representations of local geometries. In this paper, we propose to jointly use both global and local descriptors to register point clouds in a self-supervised manner, which is motivated by a critical observation that all local geometries of point clouds are transformed consistently under the same transformation. Therefore, local geometries can be employed to enhance the representation ability of the feature extraction module. Moreover, the proposed local descriptor is flexible and can be integrated into most existing registration methods and improve their performance. Besides, we also utilize point cloud reconstruction and normal estimation to enhance the transformation awareness of global and local descriptors. Lastly, extensive experimental results on one synthetic and three real-world datasets demonstrate that our method outperforms existing state-of-art unsupervised registration methods and even surpasses supervised ones in some cases. Robustness and computational efficiency evaluations also indicate that the proposed method applies to intelligent vehicles.

The affinities of 177 nonameric peptides binding to the HLA-A*0201 molecule were measured using a FACS-based MHC stabilisation assay and analysed using chemometrics. Their structures were described by global and local descriptors, QSAR models were derived by genetic algorithm, stepwise regression and PLS. The global molecular descriptors included molecular connectivity chi indices, kappa shape indices, E-state indices, molecular properties like molecular weight and log P, and three-dimensional descriptors like polarizability, surface area and volume. The local descriptors were of two types. The first used a binary string to indicate the presence of each amino acid type at each position of the peptide. The second was also position-dependent but used five z-scales to describe the main physicochemical properties of the amino acids forming the peptides. The models were developed using a representative training set of 131 peptides and validated using an independent test set of 46 peptides. It was found that the global descriptors could not explain the variance in the training set nor predict the affinities of the test set accurately. Both types of local descriptors gave QSAR models with better explained variance and predictive ability. The results suggest that, in their interactions with the MHC molecule, the peptide acts as a complicated ensemble of multiple amino acids mutually potentiating each other.

During the continual transitions from lab research to real-world applications of vision-based algorithms, there are significant challenges, e.g. the robustness to adapt to complex environments and the high demands of multi-task, multi-modal learning models. More specifically on visual simultaneous localization and mapping (visual SLAM) for mobile robots, two notable limitations are: (i) the drift issue during pose estimation, especially in dynamic environments, makes the positioning system unstable; (ii) the additional loop detection takes many computational resources for image retrieval and global feature geometry checking. This thesis explores the approaches of visual localization on unmanned ground vehicles by solving those limitations. The conventional feature extraction and matching pipeline for vision-based tasks involve three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. The first contribution of the thesis is adding a feature selection and anticipation stage to reduce tracking drift. We explore the noise model of image features to select a subset of all the observed image features with the best ”contribution” during data association and pose estimation across multiple frames. Conventional SLAM algorithms take a strong assumption of scene rigidity, which limits the application under challenging environments. The second part of the thesis addresses the tough issue in dynamic environments with moving objects. We presented GMC, namely the motion clustering approach, a lightweight dynamic object filtering method. It can distinguish moving objects from static landmarks. Based on the theory of motion coherence within a particular image area, GMC could segment dynamic objects in 3D space. We can provide an efficient and robust correspondence algorithm that can extract dynamic objects from a static background with the method. In this way, we propose a dynamic SLAM system that is real-time and free from expensive GPU processors. In contrast to GMC, the thesis’s third part turns to an object-aware learning-based model for more general dynamic scenarios. We use object detection and tracking as points, lines, planes, etc. We utilize semantic information and extract sparse image features simultaneously to keep track of dynamic objects. The static background and different dynamic objects are jointly optimized in a newly developed bundle adjustment sliding window. The estimated 3D bounding boxes can provide more robust camera tracking and better scene understanding, and better map merging. The fourth part of the thesis leverages the emerging feature learning framework. It proposes a unified self-supervised model called LGDNet to generate both global and local image feature descriptors end-to-end. Global feature descriptors embed the whole image into a compact representation, leading to easier scene comparison. On the other hand, local features focus more on the local region similarities of some salient parts for structure from motion. Our proposed method can directly extract features together with descriptors that encode both local maximum responses and global context information, avoiding duplicate calculations based on different feature extraction criteria.

SfM (Structure from Motion) has been extensively used for UAV (Unmanned Aerial Vehicle) image orientation. Its efficiency is directly influenced by feature matching. Although image retrieval has been extensively used for match pair selection, high computational costs are consumed due to a large number of local features and the large size of the used codebook. Thus, this paper proposes an efficient match pair retrieval method and implements an integrated workflow for parallel SfM reconstruction. First, an individual codebook is trained online by considering the redundancy of UAV images and local features, which avoids the ambiguity of training codebooks from other datasets. Second, local features of each image are aggregated into a single high-dimension global descriptor through the VLAD (Vector of Locally Aggregated Descriptors) aggregation by using the trained codebook, which remarkably reduces the number of features and the burden of nearest neighbor searching in image indexing. Third, the global descriptors are indexed via the HNSW (Hierarchical Navigable Small World) based graph structure for the nearest neighbor searching. Match pairs are then retrieved by using an adaptive threshold selection strategy and utilized to create a view graph for divide-and-conquer based parallel SfM reconstruction. Finally, the performance of the proposed solution has been verified using three large-scale UAV datasets. The test results demonstrate that the proposed solution accelerates match pair retrieval with a speedup ratio ranging from 36 to 108 and improves the efficiency of SfM reconstruction with competitive accuracy in both relative and absolute orientation.

The first step in an image retrieval pipeline consists of comparing global descriptors from a large database to find a short list of candidate matching images. The more compact the global descriptor, the faster the descriptors can be compared for matching. State-of-the-art global descriptors based on Fisher Vectors are represented with tens of thousands of floating point numbers. While there is significant work on compression of local descriptors, there is relatively little work on compression of high dimensional Fisher Vectors. We study the problem of global descriptor compression in the context of image retrieval, focusing on extremely compact binary representations: 64-1024 bits. Motivated by the remarkable success of deep neural networks in recent literature, we propose a compression scheme based on deeply stacked Restricted Boltzmann Machines (SRBM), which learn lower dimensional non-linear subspaces on which the data lie. We provide a thorough evaluation of several state-of-the-art compression schemes based on PCA, Locality Sensitive Hashing, Product Quantization and greedy bit selection, and show that the proposed compression scheme outperforms all existing schemes.

In recent trends, Content Based image retrieval (CBIR) is a vast as well as an attractive research area, where various active researchers are doing their respective work. It is the widely used technique of image retrieval, which enables the system to retrieve the relevant images from wide range of image database following the user's query image. In designing of good CBIR system, to select an appropriate feature descriptor which can represent the image content efficiently is not an easy task. In this proposed work, an efficient CBIR system has been suggested for image retrieval which mainly focused on the classification of image database as well as appropriate image representation. Here, we used HSV histogram, discrete wavelet transform and local as well as global edge histogram descriptor with SVM classifier based on RBF kernel function. In the scheme of feature evaluation, first extracted quantized HSV histogram to extract the color feature of image, applied discrete wavelet transform on the each component (Hue, Saturation and value components) of the HSV image to extract the complex texture pattern of image and for evaluation of geometric as well as spatial information of image applied local as well as global edge histogram descriptor on the Value component of HSV Image. After combining the all feature vectors, SVM Classifier applied on the image feature database to classify the images appropriately. For experimental analysis, this proposed methodology applied on the Wang image database of 1-k images with 10 categories. On the basis of Precision, recall and accuracy, it is found that this system, based on classification, is performing well in comparison to other existing proposed schemes.

Bag of Features (BoF) has been successfully exploited in content-based image retrieval for several years. Due to its performance and popularity, several extensions have been proposed that involve feature description, dictionary building, feature encoding and post-query process, etc. This paper proposes a dynamically weighting scheme for BoF-based image retrieval based on feedback. It involves two contributions: (i) analyzing the statistical distribution characteristic of similar BoF representations and (ii) computing weights dynamically based on the feedback obtained from different initial query results. We quantitatively evaluate the proposed method on two different databases. Experiments confirm that the proposed weighting scheme has better performance than the baseline of BoF-based image retrieval systems. Meanwhile, the results demonstrate the effectiveness of the weighting scheme in terms of the precision of top-N returned images.

Highly Hazardous Pesticides (HHPs) pose severe risks to human health and the environment, making it essential to understand their molecular stability and degradation pathways. In this study, the Quantum Theory of Atoms in Molecules (QTAIM) was applied to four representative organophosphate pesticides, allowing the identification of electronically weak bonds as intrinsic sites of lability. These findings are consistent with reported hydrolytic, oxidative, enzymatic, and microbial degradation routes. Importantly, QTAIM descriptors proved largely insensitive to solvation, confirming their intrinsic character within the molecular electronic structure. To complement QTAIM, conceptual DFT (Density Functional Theory) reactivity indices were analyzed, revealing that solvent effects induce more noticeable variations in global and local descriptors than in topological parameters. In addition, a Topological Analysis of the Fukui Function (TAFF) was performed, which mapped nucleophilic, electrophilic, and radical susceptibilities directly onto QTAIM basins. The TAFF analysis confirmed that bonds identified as weak by QTAIM (notably P–O, P–S, and P–N linkages) also coincide with the most reactive sites, thereby reinforcing their mechanistic role in degradation pathways. This integrated framework highlights the robustness of QTAIM, the sensitivity of global and local reactivity descriptors to solvation revealed by conceptual DFT, and the complementary insights provided by TAFF, contributing to risk assessment, remediation strategies, and the rational design of safer pesticides.

The estimation of the number of people present in an image has many applications such as intelligent transportation, urban planning and crowd surveillance. Rather than conventional counting by detection or regression/machine-learning methods, we propose an image retrieval approach, which uses an image descriptor to estimate the people count. We review the performance of several image descriptors. In addition, we propose a straightforward global image descriptor for image retrieval based on compressed sensing theory. Extensive evaluations on existing crowd analysis benchmark datasets demonstrate the effectiveness of our image retrieval-based approach compared to state-of-the-art regression-based people counting methods.

This paper presents novel approaches for combining re-ranking and rank aggregation methods aiming at improving the effectiveness of Content-Based Image Retrieval (CBIR) systems. Given a query image as input, CBIR systems retrieve the most similar images in a collection by taking into account image visual properties. In this scenario, accurately ranking collection images is of great relevance. Aiming at improving the effectiveness of CBIR systems, re-ranking and rank aggregation algorithms have been proposed. However, different re-ranking and rank aggregation approaches, applied to different image descriptors, may produce different and complementary image rankings. In this paper, we present four novel approaches for combining these rankings aiming at obtaining more effective results. Several experiments were conducted involving shape, color, and texture descriptors. The proposed approaches are also evaluated on multimodal retrieval tasks, considering visual and textual descriptors. Experimental results demonstrate that our approaches can improve significantly the effectiveness of image retrieval systems.

Many computer vision applications, including image classification, matching, and retrieval use global image representations, such as the Fisher vector, to encode a set of local image patches. To describe these patches, many local descriptors have been designed to be robust against lighting changes and noise. However, local image descriptors are unstable when the underlying image signal is low. Such low-signal patches are sensitive to small image perturbations, which might come e.g., from camera noise or lighting effects. In this paper, we first quantify the relation between the signal strength of a patch and the instability of that patch, and second, we extend the standard Fisher vector framework to explicitly take the descriptor instabilities into account. In comparison to common approaches to dealing with descriptor instabilities, our results show that modeling local descriptor instability is beneficial for object matching, image retrieval, and classification.

Recent attempts on the Convolutional Neural Network (CNN) based image retrieval usually adopt the output of a specific convolutional or fully connected layer as feature representation. Though superior representation capability has yielded better retrieval performance, the scale variation and clutter distracting remain to be two challenging problems in CNN based image retrieval. In this work, we propose a Multi-Scale Context Attention Network (MSCAN) to generate global descriptors, which is able to selectively focus on the informative regions with the assistance of multi-scale context information. We model the multi-scale context information by an improved Long Short-Term Memory (LSTM) network across different layers. As such, the proposed global descriptor is equipped with the scale aware attention capability. Experimental results show that our proposed method can effectively capture the informative regions in images and retain reliable attention responses when encountering scale variation and clutter distracting. Moreover, we compare the performance of the proposed scheme with the state-of-the-art global descriptors, and extensive results verify that the proposed MSCAN can achieve superior performance on several image retrieval benchmarks.

In a complex process like face recognition, the type of the features embedded in human face plays a dominant role in the recognition process. The global features describe the whole face image, while the local features are used to describe the local areas in the face. Therefore, combining these features together in the same feature vectors leads to provide attractive features, which have the ability to distinguishing between similar face images. In this paper, we propose a combined local and global face recognition technique. The local features are captured using radon transform descriptor, while the global features are provided using Chebyshev–Fourier moments. Extensive experiments have been performed on four different face databases, namely JAFFE, FERET, YALE, and UMIST, to evaluate the accuracy of the suggested approach under different face variations. Results of these experiments indicate that the presented technique achieved high recognition accuracy in the presence of illumination, pose, and facial expression variations.

With the rapid advancement of digital imaging technologies, and the use of large volume image databases in various applications, it becomes imperative to build an automatic and an efficient image retrieval system. Content Based Image Retrieval (CBIR) is most emerging and vivid research area in computer vision, in which unknown query image assigns to the closest possible similar images available in the database. Current systems mainly use colour, texture, and shape information for image retrieval using similarity measures between query and database images features. Here this work, proposed a classification system that allows recognizing and recovering the class of a query image based on its visual content. This successful categorization of images greatly enhances the performance of retrieval by filtering out irrelevant classes. In this way we have done the comparative analysis of various features as an individual or in combinations, with direct similarity measure and proposed framework. Experimentson benchmark Wang database show that the proposed classification & retrieval framework performs significantly better than the common framework of distances.