Learning and Calibrating Per-Location Classifiers for Visual Place Recognition

Object recognition is of fundamental importance in computer vision. In a few years, pedestrian detection, car detection, and more generally scene recognition will likely be reliable enough to allow fully-automated car navigation, and the human driver will be relegated to the back seat to sip his coffee. In this thesis we are interested in recognizing individual objects and categories. In order to reduce the volume of information one has to process, images are characterized by sets of features. These features, also called interest points, are targeted at image locations with high local information content. Various systems for detecting interest points and for describing the local image appearance near these points, have been proposed in the last two decades. We investigate which combinations from this plethora of detectors and descriptors, are most suited for object recognition tasks. On to the problem of object recognition, we are first interested in recognizing individual objects. In a few years, one can imagine that customers in shops, will take with their cell phone a picture of a product that looks interesting, send it to a remote server with a huge database of individual objects, and get back information about that specific product. We propose a system for individual object recognition, inspired from previous work on coarse-to-fine recognition. All steps of the recognition process are translated into principled probabilistic terms, which allows us to outperform a state-of-the-art commercial system for individual recognition. Regarding categories, faces are probably the category that has received the most attention in computer vision literature. Here we propose a system to recognize images of the same individual in large databases of images. This can be of high interest when looking for images of a given person over the internet. Our method's advantage is that it works on real-world images, as opposed to the face databases from the literature, collected in laboratories with controlled lighting, pose and background conditions. Finally, we are interested in recognition of object categories in general. Using support vector machines for the classification task, we propose a features-based kernel that improves recognition performance on object categories.

Establishing structural or functional relationship between sequences, for instance to infer the structural class of an unannotated protein, is a key task in biological analysis. Protein sequences undergo complex transformations such as mutation, insertion and deletion during the evolutionary process and typically share low sequence similarity on the superfamily level, making the task for remote homology detection based on primary sequence only very challenging. Based on previous studies stating that knowledge based on only a subset of critical positions and the preferred symbols on such positions are sufficient for remote homology detection, we present a series of works, each enforcing different notion of sparsity, to recover such critical positions. We first start with a generative model and present the sparse profile hidden Markov models. Such generative approach recovers some critical patterns and motivates the need for discriminative learning. In our second study, we present a discriminative approach to recover such critical positions and the preferred symbols. In our third study, we address the issue of very few positive training examples, accompanied by a large number of negative training examples, which is typical in many remote homology detection task. Such issue motivates the need for semi-supervised learning. However, though containing abundant useful and critical information, large uncurated sequence databases also contain a lot of noise, which may compromise the quality of the classifiers. As a result, we present a systematic and biologically motivated framework for semi-supervised learning with large uncurated sequence databases. Combined with a very fast string kernel, our method not only realizes rapid and accurate remote homology detection and show state-of-the-art performance, but also recovers some critical patterns conserved in superfamilies.

BackgroundThe rapid growth of biomedical literature requires accurate text analysis and text processing tools. Detecting abbreviations and identifying their definitions is an important component of such tools. Most existing approaches for the abbreviation definition identification task employ rule-based methods. While achieving high precision, rule-based methods are limited to the rules defined and fail to capture many uncommon definition patterns. Supervised learning techniques, which offer more flexibility in detecting abbreviation definitions, have also been applied to the problem. However, they require manually labeled training data.MethodsIn this work, we develop a machine learning algorithm for abbreviation definition identification in text which makes use of what we term naturally labeled data. Positive training examples are naturally occurring potential abbreviation-definition pairs in text. Negative training examples are generated by randomly mixing potential abbreviations with unrelated potential definitions. The machine learner is trained to distinguish between these two sets of examples. Then, the learned feature weights are used to identify the abbreviation full form. This approach does not require manually labeled training data.ResultsWe evaluate the performance of our algorithm on the Ab3P, BIOADI and Medstract corpora. Our system demonstrated results that compare favourably to the existing Ab3P and BIOADI systems. We achieve an F-measure of 91.36% on Ab3P corpus, and an F-measure of 87.13% on BIOADI corpus which are superior to the results reported by Ab3P and BIOADI systems. Moreover, we outperform these systems in terms of recall, which is one of our goals.

Place recognition is an essential capability for robotic autonomy. While ground robots observe the world from generally similar viewpoints over repeated visits, other robots, such as small aircraft, experience far more different viewpoints, requiring place recognition for images captured from very wide baselines. While traditional feature-based methods fail dramatically under extreme viewpoint changes, deep learning approaches demand heavy runtime processing. Driven by the need for cheaper alternatives able to run on computationally restricted platforms, such as small aircraft, this letter proposes a novel real-time pipeline employing depth-completion on sparse feature maps that are anyway computed during robot localization and mapping, to enable place recognition at extreme viewpoint changes. The proposed approach demonstrates unprecedented precision-recall rates on challenging benchmarking and own synthetic and real datasets with up to 45° difference in viewpoints. In particular, our synthetic datasets are, to the best of our knowledge, the first to isolate the challenge of viewpoint changes for place recognition, addressing a crucial gap in the literature. All of the new datasets are publicly available to aid benchmarking.

Visual place recognition is a mature field in mobile robotics research. Recognizing places in datasets covering traversals of hundreds or thousands of kilometres and accurate localization in small and medium size environments have been successfully demonstrated. However, for real world long term operation, visual place recognition has to face severe environmental appearance changes due to day-night cycles, seasonal or weather changes. Existing approaches for recognizing places in such changing environments provide solutions for matching images from the exact same viewpoint using powerful holistic descriptors, using less sophisticated holistic descriptors in combinations with images sequences, and/or pose strong requirements on training data to learn systematic appearance changes. In this paper, we present a novel, training free, single image matching procedure that builds upon local region detectors for powerful Convolutional Neural Network (CNN) based descriptors. It can be used with a broad range of local region detectors including keypoints, segmentation based approaches and object proposals. We propose a novel evaluation criterion for selection of an appropriate local region detector for changing environments and compare several available detectors. The scale space extrema detector known from the SIFT keypoint detector in combination with appropriate magnification factors performs best. We present preliminary results of the proposed image matching procedure with several region detectors on the challenging Nordland dataset on place recognition between different seasons and a dataset including severe viewpoint changes. The proposed method outperforms the best existing holistic method for place recognition in such changing environments and can additionally handle severe viewpoint changes. Additionally, the combination of the best performing detectors with superpixel based spatial image support shows promising results.

This paper presents a novel method to retrieve images by learning the embedded visual concept from a set of given examples. Through a user's relevance feedback, the visual concept can be effectively learned to classify images which contain common visual entities. The learning process is started by providing a set of either positive or negative training examples and is then interactively adjusted according to the user's relevance feedback. In contrast to traditional methods, the proposed method utilizes a novel way to overcome the under-training problem which is frequently suffered in the learning process. Since no time-consuming optimization process is involved, the proposed method learns the visual concepts extremely fast. Therefore, the target concept can be learned on-line and is user-adaptable for effective retrieval of image contents. Experimental results are provided to prove the superiority of the proposed method.

For hyper-rectangles in $$\mathbb{R}^{d}$$ Auer (1997) proved a PAC bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ , where $$\varepsilon$$ and $$\delta$$ are the accuracy and confidence parameters. It is still an open question whether one can obtain the same bound for intersection-closed concept classes of VC-dimension $$d$$ in general. We present a step towards a solution of this problem showing on one hand a new PAC bound of $$O(\frac{1}{\varepsilon}(d\log d + \log \frac{1}{\delta}))$$ for arbitrary intersection-closed concept classes, complementing the well-known bounds $$O(\frac{1}{\varepsilon}(\log \frac{1}{\delta}+d\log \frac{1}{\varepsilon}))$$ and $$O(\frac{d}{\varepsilon}\log \frac{1}{\delta})$$ of Blumer et al. and (1989) and Haussler, Littlestone and Warmuth (1994). Our bound is established using the closure algorithm, that generates as its hypothesis the intersection of all concepts that are consistent with the positive training examples. On the other hand, we show that many intersection-closed concept classes including e.g. maximum intersection-closed classes satisfy an additional combinatorial property that allows a proof of the optimal bound of $$O(\frac{1}{\varepsilon}(d+\log \frac{1}{\delta}))$$ . For such improved bounds the choice of the learning algorithm is crucial, as there are consistent learning algorithms that need $$\Omega(\frac{1}{\varepsilon}(d\log\frac{1}{\varepsilon} +\log\frac{1}{\delta}))$$ examples to learn some particular maximum intersection-closed concept classes.

An object detection method is developed based on combining top-down recognition with bottom-up image segmentation. There are three main steps in this method: a shape learning step, a hypothesis generation step and a verification step. In the shape learning step, training images are used to building codebook dictionary, and matching template is also constructed based on codebook dictionary. In the top-down hypothesis generation step, the Probability Shape Context (PSC) is used to generate a set of hypotheses of object locations. In the verification step, the hypotheses of object locations are segmented based on Partial Differential Equation (PDE). Based on segmented outlines and matching template, the inference is used to prune out false positives. Experimental results demonstrate that our proposed approach is able to accurately detect objects with only a few positive training examples.

Hierarchical learning strategy in semantic relation extraction

Density-aware generative algorithms learning from positive examples have verified high recall for visual object detection, but such generative methods suffer from excessive false positives which leads to low precision. Inspired by the recent success of detection-recognition pipeline with deep neural networks, this paper proposes a two-step framework by training a generative detector with positive samples first and then utilising a discriminative model to get rid of false positives in those detected bounding box candidates by the generative detector. Evidently, the discriminative model can be viewed as a post-processing step which improves the robustness by distinguishing true positives from false positives that confuse the generative detector. We exemplify the proposed approach on public ImageNet classes to demonstrate the significant improvement on precision while using only positive examples in training.

We seek to recognize the place depicted in a query image using a database of “street side” images annotated with geolocation information. This is a challenging task due to changes in scale, viewpoint and lighting between the query and the images in the database. One of the key problems in place recognition is the presence of objects such as trees or road markings, which frequently occur in the database and hence cause significant confusion between different places. As the main contribution, we show how to avoid features leading to confusion of particular places by using geotags attached to database images as a form of supervision. We develop a method for automatic detection of image-specific and spatially-localized groups of confusing features, and demonstrate that suppressing them significantly improves place recognition performance while reducing the database size. We show the method combines well with the state of the art bag-of-features model including query expansion, and demonstrate place recognition that generalizes over wide range of viewpoints and lighting conditions. Results are shown on a geotagged database of over 17K images of Paris downloaded from Google Street View.KeywordsVisual WordQuery ImageQuery ExpansionObject RetrievalPlace RecognitionThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Visual place recognition is the problem of camera-based localization of a robot given a database of images of known places, potentially under severe appearance changes (e.g., different weather or illumination). Typically, the current query images and the database images are sequences of consecutive images. In previous work, we proposed to adapt hierarchical temporal memory, a biologically plausible model of sequence processing in the human neocortex to address this task. The previous work described the algorithmic steps and showed synthetic experimental results from simulation. This letter extends the approach to application on real-world data based on a novel encoder for state-of-the-art image processing front ends. The neurologically inspired approach is compared with several state-of-the-art algorithms on a variety of datasets and shows preferable performance. Beyond the place recognition performance, the neurological roots of the algorithm result in appealing properties like potentially very energy efficient implementation due to the usage of sparse distributed representations and natural extendability like the integration of motion estimates similar to entorhinal grid cells. Finally, we underline its practical applicability by online, soft real-time application on a mobile robot.

Robust model-based scene interpretation by multilayered context information

Appearance-invariant place recognition by discriminatively training a convolutional neural network

A common obstacle in effective learning of visual concept classifiers is the scarcity of positive training examples due to expensive labeling cost. This paper explores the sampling of weakly tagged web images for concept learning without human assistance. In particular, ontology knowledge is incorporated for semantic pooling of positive examples from ontologically neighboring concepts. This effectively widens the coverage of the positive samples with visually more diversified content, which is important for learning a good concept classifier. We experiment with two learning strategies: aggregate and incremental. The former strategy re-trains a new classifier by combining existing and newly collected examples, while the latter updates the existing model using the new samples incrementally. Extensive experiments on NUS-WIDE and VOC 2010 datasets show very encouraging results, even when comparing with classifiers learnt using expert labeled training examples.