Semantic Ambiguity Modeling and Propagation for Fine-Grained Visual Cross View Geo-Localization

Change detection, or anomaly detection, from street-view images acquired by an autonomous robot at multiple different times, is a major problem in robotic mapping and autonomous driving. Formulation as an image comparison task, which operates on a given pair of query and reference images is common to many existing approaches to this problem. Unfortunately, providing relevant reference images is not straightforward. In this paper, we propose a novel formulation for change detection, termed compressive change retrieval, which can operate on a query image and similar reference images retrieved from the web. Compared to previous formulations, there are two sources of difficulty. First, the retrieved reference images may frequently contain non-relevant reference images, because even state-of-the-art place-recognition techniques suffer from retrieval noise. Second, image comparison needs to be conducted in a compressed domain to minimize the storage cost of large collections of street-view images. To address the above issues, we also present a practical change detection algorithm that uses compressed bag-of-words (BoW) image representation as a scalable solution. The results of experiments conducted on a practical change detection task, "moving object detection (MOD)," using the publicly available Malaga dataset validate the effectiveness of the proposed approach.

Motivated by the recent success of supervised and weakly supervised common object discovery, in this paper, we move forward one step further to tackle common object discovery in a fully unsupervised way. Generally, object co-localization aims at simultaneously localizing objects of the same class across a group of images. Traditional object localization/detection usually trains specific object detectors which require bounding box annotations of object instances, or at least image-level labels to indicate the presence/absence of objects in an image. Given a collection of images without any annotations, our proposed fully unsupervised method is to simultaneously discover images that contain common objects and also localize common objects in corresponding images. Without requiring to know the total number of common objects, we formulate this unsupervised object discovery as a sub-graph mining problem from a weighted graph of object proposals, where nodes correspond to object proposals, and edges represent the similarities between neighbouring proposals. The positive images and common objects are jointly discovered by finding sub-graphs of strongly connected nodes, with each sub-graph capturing one object pattern. The optimization problem can be efficiently solved by our proposed maximal-flow-based algorithm. Instead of assuming that each image contains only one common object, our proposed solution can better address wild images where each image may contain multiple common objects or even no common object. Moreover, our proposed method can be easily tailored to the task of image retrieval in which the nodes correspond to the similarity between query and reference images. Extensive experiments on PASCAL VOC 2007 and Object Discovery data sets demonstrate that even without any supervision, our approach can discover/localize common objects of various classes in the presence of scale, view point, appearance variation, and partial occlusions. We also conduct broad experiments on image retrieval benchmarks, Holidays and Oxford5k data sets, to show that our proposed method, which considers both the similarity between query and reference images and also similarities among reference images, can help to improve the retrieval results significantly.

This paper presents a new approach for the challenging problem of geo-localization using image matching in a structured database of city-wide reference images with known GPS coordinates. We cast the geo-localization as a clustering problem of local image features. Akin to existing approaches to the problem, our framework builds on low-level features which allow local matching between images. For each local feature in the query image, we find its approximate nearest neighbors in the reference set. Next, we cluster the features from reference images using Dominant Set clustering, which affords several advantages over existing approaches. First, it permits variable number of nodes in the cluster, which we use to dynamically select the number of nearest neighbors for each query feature based on its discrimination value. Second, this approach is several orders of magnitude faster than existing approaches. Thus, we obtain multiple clusters (different local maximizers) and obtain a robust final solution to the problem using multiple weak solutions through constrained Dominant Set clustering on global image features, where we enforce the constraint that the query image must be included in the cluster. This second level of clustering also bypasses heuristic approaches to voting and selecting the reference image that matches to the query. We evaluate the proposed framework on an existing dataset of 102k street view images as well as a new larger dataset of 300k images, and show that it outperforms the state-of-the-art by 20 and 7 percent, respectively, on the two datasets.

Exploiting the complementary strengths of multi-layer CNN features for image retrieval

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions (DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.

Finding an image's exact GPS location is a challenging computer vision problem that has many real-world applications. In this paper, we address the problem of finding the GPS location of images with an accuracy which is comparable to hand-held GPS devices. We leverage a structured data set of about 100,000 images build from Google Maps Street View as the reference images. We propose a localization method in which the SIFT descriptors of the detected SIFT interest points in the reference images are indexed using a tree. In order to localize a query image, the tree is queried using the detected SIFT descriptors in the query image. A novel GPS-tag-based pruning method removes the less reliable descriptors. Then, a smoothing step with an associated voting scheme is utilized; this allows each query descriptor to vote for the location its nearest neighbor belongs to, in order to accurately localize the query image. A parameter called Confidence of Localization which is based on the Kurtosis of the distribution of votes is defined to determine how reliable the localization of a particular image is. In addition, we propose a novel approach to localize groups of images accurately in a hierarchical manner. First, each image is localized individually; then, the rest of the images in the group are matched against images in the neighboring area of the found first match. The final location is determined based on the Confidence of Localization parameter. The proposed image group localization method can deal with very unclear queries which are not capable of being geolocated individually.

It is known that physical coupling between two subjects may be advantageous in joint tasks. However, little is known about how two people mutually exchange information to exploit the coupling. Therefore, we adopted a reversed, novel perspective to the standard one that focuses on the ability of physically coupled subjects to adapt to cooperative contexts that require negotiating a common plan: we investigated how training in pairs on a novel task affects the development of motor skills of each of the interacting partners. The task involved reaching movements in an unstable dynamic environment using a bilateral non-linear elastic tool that could be used bimanually or dyadically. The main result is that training with an expert leads to the greatest performance in the joint task. However, the performance in the individual test is strongly affected by the initial skill level of the partner. Moreover, practicing with a peer rather than an expert appears to be more advantageous for a naive; and motor skills can be transferred to a bimanual context, after training with an expert, only if the non-expert subject had prior experience of the dynamics of the novel task.

This paper presented a novel approach for document clustering applying cloud technologies for the system's performance issue. It is critical for application service in cloud computing to provide precise information. Ontology servicing is one of the methods to deal with semantic ambiguity and information overload efficiently through appropriate semantic models and semantic technology. This system is the advanced and extended version of the system we have been published before. The experiments reveal that even the testing documents increased; the system may actually be able to produce useful result for text document clustering. In this paper, we propose a cloud service that exploits a novel ontology-based technique for identifying cloud service to improve the accuracy of cloud services searching. Our approach has the capability to perform cloud service concepts from cloud service sources. The main idea behind our method is cloud services using an ontology-based technique.

In recent years video surveillance has gained significant attention due to its versatile applications in security and traffic flow management. Vehicle Re-Identification (Re-ID) is a complicated task in Intelligent Transportation Systems (ITS) where vehicles are tracked by multiple cameras in non-overlapping views. In re-identification the targeted vehicle is matched with the query image from the gallery of vehicle images. There are many uncertainties involved in Re-ID due to inter- class and intra-class similarity because of the viewpoint changes. Large number of studies have explored the deep learning techniques for vehicle detection but classification of vehicle images and re-identifying them is still a field of exploration. In this paper we have analyzed the issue of fine grained vehicle classification and thereby re-identifying them using different CNN models. Vehicle images are pre-processed by different data augmentation techniques and then transfer learning is applied for testing the Stanford cars dataset on three models namely ResNet152, ResNet50 and DenseNet201. The accuracy is obtained for all the models and ResNet152 produces a highest accuracy of 89% among the three models.

This paper tackles the novel challenging problem of 3D object phenotype recognition from a single 2D silhouette. To bridge the large pose (articulation or deformation) and camera viewpoint changes between the gallery images and query image, we propose a novel probabilistic inference algorithm based on 3D shape priors. Our approach combines both generative and discriminative learning. We use latent probabilistic generative models to capture 3D shape and pose variations from a set of 3D mesh models. Based on these 3D shape priors, we generate a large number of projections for different phenotype classes, poses, and camera viewpoints, and implement Random Forests to efficiently solve the shape and pose inference problems. By model selection in terms of the silhouette coherency between the query and the projections of 3D shapes synthesized using the galleries, we achieve the phenotype recognition result as well as a fast approximate 3D reconstruction of the query. To verify the efficacy of the proposed approach, we present new datasets which contain over 500 images of various human and shark phenotypes and motions. The experimental results clearly show the benefits of using the 3D priors in the proposed method over previous 2D-based methods.

The problem of object or scene recognition is often addressed by seeking geometric image properties that are invariant under changes in viewing conditions. An alternative, non-geometric, ratio method was described by Funt and Finlayson (IEEE Trans. Pattern Anal. Mach. Intell. 17,522, 1995) in which histograms of spatial ratios of colour RGB triplets from neighbouring image regions were used to recognise objects under changes in viewpoint and illumination. In this study, ratio indexing was extended from RGB images to hyperspectral images with a variable number of sensor channels distributed over 400-720 nm. Fifty natural scenes were used to generate test and reference images. For each number of sensors, independent random samples were drawn from each test image of a scene under either a daylight of correlated colour temperature of 25000 K or of 4000 K and matched against independent random samples drawn from each reference image of the scenes under a daylight of correlated colour temperature 6500 K. Matching was based on the intersection of multi-dimensional histograms of ratios of sensor signals in these samples Differences between match hit and false-alarm rates provided a measure of recognition performance. Results suggest that for small samples, indexing with five sensor channels has advantages over indexing with three sensor channels for the recognition of natural scenes.

Interest points have traditionally been favoured over dense features for image retrieval tasks, where the goal is to retrieve images similar to a query image from an image corpus. While interest points are invariant to scale and rotation, their coverage of the image is not adequate for sub-image retrieval problems, where the query image occupies a small part of the corpus image. On the other hand, dense features provide excellent coverage but lack invariance, as they are computed at a fixed scale and orientation. Recently, we proposed a novel technique of combining dense features with interest points (Bhattacharya and Gavrilova, Vis Comput 29(6---8):491---499, 2013) to leverage the benefits of both worlds. This allows dense features to be scale invariant but not rotation invariant. In this paper, we build on this framework by incorporating rotation invariance for dense features and introducing several improvements in the voting and match score computation stages. Our method can produce high-quality recognition results that outperform bag of words even with geometric verification and several state-of-art methods that have considered spatial information. We achieve significant improvements in terms of both search speed and accuracy over (Bhattacharya and Gavrilova, Vis Comput 29(6---8):491---499, 2013). Experiments on Oxford Buildings, Holidays and UKbench datasets reveal that our method is not only robust to viewpoint and scale changes that occur in real-world photographs but also to geometric transformations.

This chapter puts forward a new approach to address a general purpose Content-Based Image Retrieval(CBIR) task. Six spatial color moments extracted from visually significant locations of an image are used as features to characterize an image. It utilizes information given by a set of queries, as opposed to a single image query. A set of similar images is posed as a query and Mahalanobis distance is used to evaluate the similarity between query images and target images of the database. Given a query set, the mean and covariance for computing Mahalanobis distance is obtained from the same. Unlike conventional CBIR methods in which images are retrieved based on considering similarities between the query image and the database images through a sequential search, a clustering technique using K-means algorithm is first used to create meaningful groups in the database. As clusters are created by considering similarities between images in the database, the image retrieval search space is reduced if clusters near to the query are searched. The effectiveness of the proposed algorithm is demonstrated with increased accuracy and reduced retrieval time.KeywordsImage matchingMahalanobis distanceclusteringprecisionrecall

CLIP and BiomedCLIP are examples of vision-language foundation models and offer strong cross-modal embeddings; however, they are not optimized for fine-grained medical retrieval tasks, such as retrieving clinically relevant radiology reports using chest X-ray (CXR) image queries. To address this shortcoming, we propose a multi-task learning framework to fine-tune BiomedCLIP and evaluate improvements to CXR image-text retrieval. Using BiomedCLIP as the backbone, we incorporate a lightweight MLP projector head trained with a multi-task composite loss function that includes: (1) a binary cross-entropy loss to distinguish normal from abnormal CXR studies, (2) a supervised contrastive loss to reinforce intra-class consistency, and (3) a CLIP loss to maintain cross-modal alignment. Experimental results demonstrate that the fine-tuned model achieves more balanced and clinically meaningful performance across both image-to-text and text-to-image retrieval tasks compared to the pretrained BiomedCLIP and general-purpose CLIP models. Furthermore, t-SNE visualizations reveal clearer semantic clustering of normal and abnormal cases, demonstrating the model's enhanced diagnostic sensitivity. These findings highlight the value of domain-adaptive, multi-task learning for advancing cross-modal retrieval in biomedical applications.

On-line translation of natural language questions into artificial language queries