Compressive change retrieval for moving object detection

This paper presents a new approach for the challenging problem of geo-localization using image matching in a structured database of city-wide reference images with known GPS coordinates. We cast the geo-localization as a clustering problem of local image features. Akin to existing approaches to the problem, our framework builds on low-level features which allow local matching between images. For each local feature in the query image, we find its approximate nearest neighbors in the reference set. Next, we cluster the features from reference images using Dominant Set clustering, which affords several advantages over existing approaches. First, it permits variable number of nodes in the cluster, which we use to dynamically select the number of nearest neighbors for each query feature based on its discrimination value. Second, this approach is several orders of magnitude faster than existing approaches. Thus, we obtain multiple clusters (different local maximizers) and obtain a robust final solution to the problem using multiple weak solutions through constrained Dominant Set clustering on global image features, where we enforce the constraint that the query image must be included in the cluster. This second level of clustering also bypasses heuristic approaches to voting and selecting the reference image that matches to the query. We evaluate the proposed framework on an existing dataset of 102k street view images as well as a new larger dataset of 300k images, and show that it outperforms the state-of-the-art by 20 and 7 percent, respectively, on the two datasets.

Motivated by the recent success of supervised and weakly supervised common object discovery, in this paper, we move forward one step further to tackle common object discovery in a fully unsupervised way. Generally, object co-localization aims at simultaneously localizing objects of the same class across a group of images. Traditional object localization/detection usually trains specific object detectors which require bounding box annotations of object instances, or at least image-level labels to indicate the presence/absence of objects in an image. Given a collection of images without any annotations, our proposed fully unsupervised method is to simultaneously discover images that contain common objects and also localize common objects in corresponding images. Without requiring to know the total number of common objects, we formulate this unsupervised object discovery as a sub-graph mining problem from a weighted graph of object proposals, where nodes correspond to object proposals, and edges represent the similarities between neighbouring proposals. The positive images and common objects are jointly discovered by finding sub-graphs of strongly connected nodes, with each sub-graph capturing one object pattern. The optimization problem can be efficiently solved by our proposed maximal-flow-based algorithm. Instead of assuming that each image contains only one common object, our proposed solution can better address wild images where each image may contain multiple common objects or even no common object. Moreover, our proposed method can be easily tailored to the task of image retrieval in which the nodes correspond to the similarity between query and reference images. Extensive experiments on PASCAL VOC 2007 and Object Discovery data sets demonstrate that even without any supervision, our approach can discover/localize common objects of various classes in the presence of scale, view point, appearance variation, and partial occlusions. We also conduct broad experiments on image retrieval benchmarks, Holidays and Oxford5k data sets, to show that our proposed method, which considers both the similarity between query and reference images and also similarities among reference images, can help to improve the retrieval results significantly.

Place recognition and visual localization are particularly challenging in wide baseline configurations. In this letter, we contribute with the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Danish Airs and Grounds</i> (DAG) dataset, a large collection of street-level and aerial images targeting such cases. Its main challenge lies in the extreme viewing-angle difference between query and reference images with consequent changes in illumination and perspective. The dataset is larger and more diverse than current publicly available data, including more than 50 km of roads in urban, suburban and rural areas. All images are associated with accurate 6-DoF metadata that allows the benchmarking of visual localization methods. Additionally, we validate our data by presenting the results of a simple map-to-image re-localization baseline. that first estimates a dense 3D reconstruction from the aerial images and then matches query street-level images to street-level renderings of the 3D model. The dataset can be downloaded at:.

This paper addresses the problem of image alignment based on random measurements. Image alignment consists of estimating the relative transformation between a query image and a reference image. We consider the specific problem where the query image is provided in compressed form in terms of linear measurements captured by a vision sensor. We cast the alignment problem as a manifold distance minimization problem in the linear subspace defined by the measurements. The transformation manifold that represents synthesis of shift, rotation, and isotropic scaling of the reference image can be given in closed form when the reference pattern is sparsely represented over a parametric dictionary. We show that the objective function can then be decomposed as the difference of two convex functions (DC) in the particular case where the dictionary is built on Gaussian functions. Thus, the optimization problem becomes a DC program, which in turn can be solved globally by a cutting plane method. The quality of the solution is typically affected by the number of random measurements and the condition number of the manifold that describes the transformations of the reference image. We show that the curvature, which is closely related to the condition number, remains bounded in our image alignment problem, which means that the relative transformation between two images can be determined optimally in a reduced subspace.

The ability to accurately predict natural backgrounds over a wide range of potential image acquisition conditions is key to wide-area change detection. The lack of robustness in pure image-based techniques has led to the development of site modeling and model-supported exploitation approaches. Instead of using multiple images to build a site model, we consider the possibility of using them directly to estimate the background of a new image for change detection. Our method uses a nonlinear mean-squared estimation technique to compute image backgrounds from multiple reference images. In comparing linear and nonlinear estimators, we find that although the performance of both improves as the number of reference images increases, the nonlinear estimator yields significantly better background estimates and is much less sensitive to registration errors between images than the linear estimator. We show that as the number of references increases, the pixels within a region become physically significant; i.e., they correspond to parts of the scene with similar physical properties. Synthetic aperture radar (SAR) background modeling and change detection results including a crosssensor change detection example in which SAR reference images are used to predict the background of an electro-optical (EO) image are presented. Significant performance gains over linear change detection (up to two orders of magnitude reduction in the false alarm rate) are demonstrated in each case over a 60-deg range of SAR sensor aspect angles.

Finding an image's exact GPS location is a challenging computer vision problem that has many real-world applications. In this paper, we address the problem of finding the GPS location of images with an accuracy which is comparable to hand-held GPS devices. We leverage a structured data set of about 100,000 images build from Google Maps Street View as the reference images. We propose a localization method in which the SIFT descriptors of the detected SIFT interest points in the reference images are indexed using a tree. In order to localize a query image, the tree is queried using the detected SIFT descriptors in the query image. A novel GPS-tag-based pruning method removes the less reliable descriptors. Then, a smoothing step with an associated voting scheme is utilized; this allows each query descriptor to vote for the location its nearest neighbor belongs to, in order to accurately localize the query image. A parameter called Confidence of Localization which is based on the Kurtosis of the distribution of votes is defined to determine how reliable the localization of a particular image is. In addition, we propose a novel approach to localize groups of images accurately in a hierarchical manner. First, each image is localized individually; then, the rest of the images in the group are matched against images in the neighboring area of the found first match. The final location is determined based on the Confidence of Localization parameter. The proposed image group localization method can deal with very unclear queries which are not capable of being geolocated individually.

We present a novel approach to change detection between two brain MRI scans (reference and target.) The proposed method uses a single modality to find subtle changes; and does not require prior knowledge (learning) of the type of changes to be sought. The method is based on the computation of a local kernel from the reference image, which measures the likeness of a pixel to its surroundings. This kernel is then used as a feature and compared against analogous features from the target image. This comparison is made using cosine similarity. The overall algorithm yields a scalar dissimilarity map (DM), indicating the local statistical likelihood of dissimilarity between the reference and target images. DM values exceeding a threshold then identify meaningful and relevant changes. The proposed method is robust to various challenging conditions including unequal signal strength.

Few-shot semantic segmentation (FSS) aims to segment unseen objects in a query image using a few pixel-wise annotated support images, thus expanding the capabilities of semantic segmentation. The main challenge lies in extracting sufficient information from the limited support images to guide the segmentation process. Conventional methods typically address this problem by generating single or multiple prototypes from the support images and calculating their cosine similarity to the query image. However, these methods often fail to capture meaningful information for modeling the de facto joint distribution of pixel and category. Consequently, they result in incomplete segmentation of foreground objects and mis-segmentation of the complex background. To overcome this issue, we propose the Cross Gaussian Mixture Generative Model (CGMGM), a novel Gaussian Mixture Models~(GMMs)-based FSS method, which establishes the joint distribution of pixel and category in both the support and query images. Specifically, our method initially matches the feature representations of the query image with those of the support images to generate and refine an initial segmentation mask. It then employs GMMs to accurately model the joint distribution of foreground and background using the support masks and the initial segmentation mask. Subsequently, a parametric decoder utilizes the posterior probability of pixels in the query image, by applying the Bayesian theorem, to the joint distribution, to generate the final segmentation mask. Experimental results on PASCAL-5i and COCO-20i datasets demonstrate our CGMGM's effectiveness and superior performance compared to the state-of-the-art methods.

Visual cross view geo-localization is generally approached within a joint retrieval-and-calibration framework. However, existing methods overlook semantic ambiguities arising from query and reference images characterized by low overlap, dynamic foregrounds, viewpoint changes, and perceptual aliasing. This makes it challenging to automatically control the relative importance of the two tasks, potentially compromising the retrieval task in favor of the offset regression. Consequently, the model may encounter conflicting dominating gradients during joint training. To address this, we propose to model the semantic ambiguity during the offset regression process by integrating associated uncertainty scores, represented as 2D Gaussian distributions, to mitigate negative transfer effects within the joint tasks. We further introduce an uncertainty-aware similarity metric to enhance similarity assessment between query and reference images, accounting for their semantic ambiguities. This metric propagates uncertainty scores into the retrieval task, focusing on certain samples and learning discriminative feature embeddings, allowing the model to adaptively handle conflicting dominating gradients during joint training. Extensive experiments demonstrate that our method improves the overall performance of the joint tasks, achieving state-of-the-art results on the VIGOR and CVACT datasets.

In mountainous area,many factors such as the high spatial heterogeneity in land cover conditions and the topographic effect on remote sensing data acquirement significantly constraint the change detection by using remote sensing method. It has become a great challenge to develop an effective method to accurately map the land cover changes for complexterrain areas. In the traditional change detection methods,the spectral features are the major information. Due to the lack of the consideration of other information from structure, topography and other parameters, many false changes will be introduced in the final detection results with a big uncertainty. To avoid the false change and overcome the influence from terrain shadow and phenology differences, an object-oriented change detection method by combining multi-scale segmentation and decision tree algorithm for complex terrain condition is proposed in this study. In the new method,firstly,a series of spectrum information( i.e. reflectance,change vector intensity,NDVI),shape information( i. e. area,lengthwidth ratio,shape index) and terrain parameters( i.e. elevation,slope,aspect) are calculated for change detection rules construction. Secondly,the unchanged surfaces which are usually detected as the false changes due to the influence of terrain shadow and phenology difference are extracted as training sample. Finally,the change detection rules are built automatically by C5.0 decision tree algorithm. the proposed method is applied to Landsat-TM images acquired in 1989 and2009 at Panxi region,the detection results indicate:( 1) Over the mountainous area,the method can effectively reduce the influence from topographic effect,and has a big improvement on detection accuracy with the overall accuracy of 93.57% and Kappa coefcient of 0.8706 by validating ground reference samples and stratified random samples.( 2) The C5.0 decision tree algorithm performs well in terms of robustness and adaptability,which serves well in land cover classification with types only labeled as "changed "and "unchanged "and highly heterogeneous training samples. Considering the false changes result from topographic and phenological as "unchanged "training sample,the accuracy of change detection is improved effectively.( 3) Spectral features are the predominant information in analyzing TM images,and the NDVI differences between the reference image and detected image is an effective indicator for change detection of land cover pattern in vegetation coverage region.( 4) The changes of land cover pattern in Panxi region from 1989 to 2009 were closely associated with human activities. The typical driving forces of pattern change include returning farmland to forest policy,water conservancy project,mining,urban sprawl and so on. The changed area amounted to 740.2 km2,accounting for 2.49% of the total detected area.

Identifying the manufacturer and the type of cardiac implantable electronic devices (CIEDs) is important in emergent clinical settings. Recent studies have illustrated that artificial neural network models can successfully recognize CIEDs from chest X-ray images. However, all existing methods require a vast amount of medical data to train the classification model. Here, we have proposed a novel method to retrieve an identical CIED image from an image database by employing the feature point matching algorithm. A total of 653 unique X-ray images from 456 patients who visited our pacemaker clinic between April 2012 and August 2020 were collected. The device images were manually square-shaped, and was thereafter resized to 224 × 224 pixels. A scale-invariant feature transform (SIFT) algorithm was used to extract the keypoints from the query image and reference images. Paired feature points were selected via brute-force matching, and the average Euclidean distance was calculated. The image with the shortest average distance was defined as the most similar image. The classification performance was indicated by accuracy, precision, recall, and F1-score for detecting the manufacturers and model groups, respectively. The average accuracy, precision, recall, and F-1 score for the manufacturer classification were 97.0%, 0.97, 0.96, and 0.96, respectively. For the model classification task, the average accuracy, precision, recall, and F-1 score were 93.2%, 0.94, 0.92, and 0.93, respectively, all of which were higher than those of the previously reported machine learning models. Feature point matching is useful for identifying CIEDs from X-ray images.

Abstract. In this paper, we present our approach for robust long-term visual localization in large scale urban environments exploiting street level imagery. Our approach consists of a 2D-image based localization using image retrieval (NetVLAD) to select reference images. This is followed by a 3D-structure based localization with a robust image matcher (DenseSfM) for accurate pose estimation. This visual localization approach is evaluated by means of the ‘Sun’ subset of the RobotCar seasons dataset, which is part of the Visual Localization benchmark. As the results on the RobotCar benchmark dataset are nearly on par with the top ranked approaches, we focused our investigations on reproducibility and performance with own data. For this purpose, we created a dataset with street-level imagery. In order to have independent reference and query images, we used a road-based and a tram-based mapping campaign with a time difference of four years. The approximately 90% successfully oriented images of both datasets are a good indicator for the robustness of our approach. With about 50% success rate, every second image could be localized with a position accuracy better than 0.25 m and a rotation accuracy better than 2°.

To quickly synthesize complex scenes, digital artists often collage together visual elements from multiple sources: for example, mountains from New Zealand behind a Scottish castle with wisps of Saharan sand in front. In this paper, we propose to use a similar process in order to parse a scene. We model a scene as a collage of warped, layered objects sampled from labeled, reference images. Each object is related to the rest by a set of support constraints. Scene parsing is achieved through analysis-by-synthesis. Starting with a dataset of labeled exemplar scenes, we retrieve a dictionary of candidate object segments that match a query image. We then combine elements of this set into a "scene collage" that explains the query image. Beyond just assigning object labels to pixels, scene collaging produces a lot more information such as the number of each type of object in the scene, how they support one another, the ordinal depth of each object, and, to some degree, occluded content. We exploit this representation for several applications: image editing, random scene synthesis, and image-to-anaglyph.

Visual Place Recognition (VPR) is process of properly identifying a formerly visited place under varying viewpoints conditions. VPR becomes a challenging task due to the variations in lighting conditions (day time or night time), shadows, weather conditions, view points, or even different seasons. Therefore, it is needed to develop VPR algorithms which can handle the variation in visual appearances. This article designs a visual place recognition using a novel region of interest (RoI) extraction with deep learning based (VPR-ROIDL) technique on changing environments. The proposed VPR-ROIDL technique initially enables the extraction of RoIs using saliency map then features of RoIs using local diagonal extrema patterns (LDEP) respectively. Besides, the extracted features of RoIs are passed into the convolutional neural network based residual network (ResNet) model and the computed deep features are stored in the database. Upon providing a query image (QI), the RoIs and features of RoIs are extracted as same as reference images. Moreover, the deep features from the ResNet model are generated and Euclidean distance based similarity measurement is used to finally recognize the places. By the use of RoI and descriptors matching process, the VPR-ROIDL model has the ability to recognize places irrespective of changing illuminations, seasons, and viewpoints. The simulation results pointed out the supremacy of the ROIDL-VPR technique over the recent state of art approaches.KeywordsVisual Place recognitionDeep learningVisual appearancesRegion of InterestSaliency mapSimilarity Measurement

These days lot of people spend long hours at a computer, so they face lot of health issues because of wrong sitting postures. Good posture is crucial for human well-being. A vision-based system is proposed to minimize the poor ergonomic practices among computer users. It can be installed in the corresponding user computer, and it runs with low computation burden. The proposed system is based on image comparison with a reference image. The reference image is an image, that the user utilizing their webcam or computer camera to capture themselves in the correct seated posture. Initially it has been a converted into grayscale and applied a Gaussian filter before store in the memory. After making the reference image, the proposed system is stared to capture user’s image at a free defined frequency. The captured user’s image is filtered and converted to a binary image using a predefined threshold. After that, two parallel processes are applied to same resultant image to identify the moment of left and right side and bending of forward and backward. The decision of both processes has been estimated by comparing the calculated black- to-white pixel ratio with a predefined threshold. The notification has been generated via a pop-up message on corresponding computer screen. To test the effectiveness, the proposed system has been implemented on a laptop computer in a Python environment with the support of OpenCV library. The test has been conducted of four different bad postures such as over leaning to the right angle, left angle, forward and backward postures with 150 attempts. The test results shows that it has high potential to identify the bad postures of computer uses. Keywords: Poor ergonomic practices; Image comparison; Gaussian filter; Binary image; Black-to- white pixel ratio