Robust Visual Place Recognition in Changing Environments Using Improved DTW

Visual place recognition is a mature field in mobile robotics research. Recognizing places in datasets covering traversals of hundreds or thousands of kilometres and accurate localization in small and medium size environments have been successfully demonstrated. However, for real world long term operation, visual place recognition has to face severe environmental appearance changes due to day-night cycles, seasonal or weather changes. Existing approaches for recognizing places in such changing environments provide solutions for matching images from the exact same viewpoint using powerful holistic descriptors, using less sophisticated holistic descriptors in combinations with images sequences, and/or pose strong requirements on training data to learn systematic appearance changes. In this paper, we present a novel, training free, single image matching procedure that builds upon local region detectors for powerful Convolutional Neural Network (CNN) based descriptors. It can be used with a broad range of local region detectors including keypoints, segmentation based approaches and object proposals. We propose a novel evaluation criterion for selection of an appropriate local region detector for changing environments and compare several available detectors. The scale space extrema detector known from the SIFT keypoint detector in combination with appropriate magnification factors performs best. We present preliminary results of the proposed image matching procedure with several region detectors on the challenging Nordland dataset on place recognition between different seasons and a dataset including severe viewpoint changes. The proposed method outperforms the best existing holistic method for place recognition in such changing environments and can additionally handle severe viewpoint changes. Additionally, the combination of the best performing detectors with superpixel based spatial image support shows promising results.

Appearance-invariant place recognition by discriminatively training a convolutional neural network

Distilled representation using patch-based local-to-global similarity strategy for visual place recognition

This thesis explores the problem of visual place recognition and localization for a mobile robot, particularly dealing with the challenges of simultaneous variations in scene appearance and camera viewpoint. The proposed methods draw inspiration from humans and make use of semantic cues to represent places. This approach enables effective place recognition from similar or opposing viewpoints, despite variations in scene appearance caused by different times of day or seasons. The research contributions presented in the thesis advance visual place recognition techniques, making them more useful for deployment in a wide range of robotic and autonomous vehicle scenarios.

Visual place recognition: A survey from deep learning perspective

Large scale optical character recognition (OCR) refers to or means the computerization of large amounts of documents such as news papers. Despite the diversity of commercial OCR products, this task still remains too far from the mature especially if the input documents are insufficient quality or cursive writing such as the Arabic documents (Vinciarelli, 2002). Indeed, in their project (Holley, 2009), the national library of Australia reports that existing OCR systems are, commonly, weak. Moreover, their conducted experiments on historical newspapers show that the corresponding accuracy raw varied from 71% to 98.02%. This is surely due to the weakness of the approaches and techniques used in these systems. Printed cursive written documents such as the Arabic one presents, in addition, other difficulties which are behind the weaknesses of the existing commercialized systems especially when the quality of the input binary image of the document is not good enough. The first difficulty encountered for such writing is the segmentation of any given input word or sub-word into isolated characters given that the size of each ofwhich is variable. In practice, if the segmentation process is conducted successfully, then it eases the recognition step to a large extent. That is why Latin printedOCR systems are, commonly, more powerful compared to those devoted to the cursive writing documents. Dynamic Time Warp (DTW) algorithm is a well known procedure especially in pattern recognition (Alves et al., 2002; Khemakhem et al., 2005; Philip, 1992; Vuori et al., 2001), (Khemakhem et al., 2009; Kumar et al., 2006; Tapia et al., 2007). The DTW algorithm is the result of the adaptation of dynamic programming to the field of pattern recognition. Printed cursive writing OCR by the DTW algorithm provides very interesting recognition rates without prior character segmentation (such as: the Arabic, Persian, Urdu, latin connected characters,...), (Khemakhem et al., 2005). The purpose of the DTW algorithm is to perform optimal time alignment between a reference pattern and an unknown pattern and evaluate their difference. Intensive experiments show that the recognition rate of the DTW algorithm remains acceptable compared to the existing commercialized systems even when the quality of the input documents is not good enough. Intensive tests on more than 100.000 connected characters (most of them are Arabic cursive and including some important noise) show that the segmentation average rate is greater than 98% and the recognition average rate is 5

Visual Place Recognition (VPR) is a fundamental yet challenging task in Visual Simultaneous Localization and Mapping (V-SLAM) problems. The VPR works as a subsystem of the V-SLAM. VPR is the task of retrieving images upon revisiting the same place in different conditions. The problem is even more difficult for agricultural and all-terrain autonomous mobile robots that work in different scenarios and weather conditions. Over the last few years, many state-of-the-art methods have been proposed to solve the limitations of existing VPR techniques. VPR using bag-of-words obtained from local features works well for a large-scale image retrieval problem. However, the aggregation of local features arbitrarily produces a large bag-of-words vector database, limits the capability of efficient feature learning, and aggregation and querying of candidate images. Moreover, aggregating arbitrary features is inefficient as not all local features equally contribute to long-term place recognition tasks. Therefore, a novel VPR architecture is proposed suitable for efficient place recognition with semantically meaningful local features and their 3D geometrical verifications. The proposed end-to-end architecture is fueled by a deep neural network, a bag-of-words database, and 3D geometrical verification for place recognition. This method is aware of meaningful and informative features of images for better scene understanding. Later, 3D geometrical information from the corresponding meaningful features is computed and utilised for verifying correct place recognition. The proposed method is tested on four well-known public datasets, and Micro Aerial Vehicle (MAV) recorded dataset for experimental validation from Victoria Park, Adelaide, Australia. The extensive experimental results considering standard evaluation metrics for VPR show that the proposed method produces superior performance than the available state-of-the-art methods.

A time series analysis method for spacecraft telemetry data is presented in this paper. For spacecraft testing and on-orbit flight, this method can monitor the changes of telemetry data automatically and identify the failure modes of spacecraft. Using dynamic time warping (DTW) algorithm, combining historical data samples as well as fault cases with this method analyzes the similarity of telemetry data transformed into time series. By comparing the results of analysis with the results of DTW distance calculation, the relative deviation of data is measured and the abnormal data in fault mode is identified. The results show that the telemetry data analysis method based on DTW algorithm can effectively detect data anomalies and realize fault identification, which has a certain application prospect.

CSPFormer: A cross-spatial pyramid transformer for visual place recognition

Partial matching and search space reduction for QbE-STD

Visual Place Recognition (VPR), the task of identifying the place where an image has been taken from, is at the core of important robotic problems as relocalization, loop-closure detection or topological navigation. Even for indoors, the focus of this work, VPR is challenging for a number of reasons, including real-time performance when dealing with large image databases ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim10^{4}$</tex-math></inline-formula> ) (probably captured by different robots), or the avoidance of Perceptual Aliasing in environments with repetitive structures and scenes. In this letter, we tackle these issues by proposing an off-line mapping technique that abstracts a dense database of georeferenced images without particular order into a Multivariate Gaussian Mixture Model, by creating soft clusters in terms of their similarity in both pose and appearance. This abstract representation is obtained through an Expectation-Maximization algorithm and plays the role of a simplified map. Since querying this map yields a probability of being in a cluster, we exploit this “belief” within a Bayesian filter that regards previous query images and a topological map between clusters to perform more robust VPR. We evaluate our proposal in two different indoor datasets, demonstrating comparable VPR precision to querying the full database while incurring in shorter query times and handling Perceptual Aliasing for sequential navigation.

This paper presents an approach for semantic place categorization using data obtained from RGB cameras. Previous studies on visual place recognition and classification have shown that, by considering features derived from pre-trained Convolutional Neural Networks (CNNs) in combination with part-based classification models, high recognition accuracy can be achieved, even in presence of occlusions and severe viewpoint changes. Inspired by these works, we propose to exploit local deep representations, representing images as set of regions applying a Na\"{i}ve Bayes Nearest Neighbor (NBNN) model for image classification. As opposed to previous methods where CNNs are merely used as feature extractors, our approach seamlessly integrates the NBNN model into a fully-convolutional neural network. Experimental results show that the proposed algorithm outperforms previous methods based on pre-trained CNN models and that, when employed in challenging robot place recognition tasks, it is robust to occlusions, environmental and sensor changes.

Visual place recognition from a 3D laser LiDAR is one of the most active research areas in robotics. Especially, learning and recognition of scene descriptors, such as scan context descriptors that map 3D point clouds to 2D point clouds, is one of the promising research directions. Although the scan-context descriptor has a sufficiently high recognition performance, it is still expensive image data and cannot be handled with low-capacity non-deep models. In this paper, we explore the task of compressing the scan context descriptor model while maintaining its recognition performance. To this end, the proposed approach slightly modifies the off-the-shelf classifier model of convolutional neural networks (CNN) from its basis, by replacing the SoftMax part with a support vector machine (SVM). Experiments with publicly available NCLT dataset validate the effectiveness of the proposed approach.

Insects play significant role in the human life. And insects pollinate major food crops consumed in the world. Insect pests consume and destroy major crops in the world. Hence to have control over the disease and pests, researches are going on in the area of entomology using chemical, biological and mechanical approaches. The data relevant to the flying insects often changes over time, and classification of such data is a central issue. And such time series mining tasks along with classification is critical nowadays. Most time series data mining algorithms use similarity search and hence time taken for similarity search is the bottleneck and it does not produce accurate results and also produces very poor performance. In this paper, a novel classification method that is based on the dynamic time warping (DTW) algorithm is proposed. The dynamic time warping algorithm is deterministic and lacks in modeling stochastic signals. The dynamic time warping (DTW) algorithm is improved by implementing a nonlinear median filtering (NMF). Recognition accuracy of conventional DTW algorithms is less than that of the hidden Markov model (HMM) by same voice activity detection (VAD) and noise-reduction. With running spectrum filtering (RSF) and dynamic range adjustment (DRA). NMF seek the median distance of every reference of time series data and the recognition accuracy is much improved. In this research work, optical sensors are used to record the sound of insect flight, with invariance to interference from ambient sounds. The implementation of our tool includes two parts, an optical sensor to record the "sound" of insect flight, and a software that leverages on the sensor information, to automatically detect and identify flying insects.

Visual place recognition is the problem of camera-based localization of a robot given a database of images of known places, potentially under severe appearance changes (e.g., different weather or illumination). Typically, the current query images and the database images are sequences of consecutive images. In previous work, we proposed to adapt hierarchical temporal memory, a biologically plausible model of sequence processing in the human neocortex to address this task. The previous work described the algorithmic steps and showed synthetic experimental results from simulation. This letter extends the approach to application on real-world data based on a novel encoder for state-of-the-art image processing front ends. The neurologically inspired approach is compared with several state-of-the-art algorithms on a variety of datasets and shows preferable performance. Beyond the place recognition performance, the neurological roots of the algorithm result in appealing properties like potentially very energy efficient implementation due to the usage of sparse distributed representations and natural extendability like the integration of motion estimates similar to entorhinal grid cells. Finally, we underline its practical applicability by online, soft real-time application on a mobile robot.