Robust tracking and event detection using receding horizon incremental locally linear embeddings

Multiple Object Tracking (MOT) has attracted increasing attention due to its academic and commercial potential. However, it still remains challenging due to factors like data association ambiguity and abrupt appearance changes. To address these issues, we extended the multiple hypothesis tracking (MHT) by incorporating the appearance features which extracted from faster region convolutional neural network (Faster-RCNN). Additionally, a principal component analysis method was introduced to update discriminative appearance features. Thus, the data association ambiguity can be reduced surprisingly by exploiting Faster-RCNN, which means fewer hypothesis for MHT. Meanwhile, the issue of abrupt appearance changes can be addressed by modeling discriminative appearance feature based-on MHT. Many experiments on MOT benchmark show competitive performance of the proposed approach in comparison with other state-of-art tracking methods.

Distilled representation using patch-based local-to-global similarity strategy for visual place recognition

Multiple object tracking: A literature review

The inverse synthetic aperture lidar (ISAL) have attracted increasing attention for its merits including small visual tracking which is considered as one of the important research topics in the field of computer vision due to its key role in versatile applications, such as precision guidance, intelligent video surveillance, human-computer interaction, robot navigation and public safety. The basic idea for implementing visual tracking is composed of finding the target object in a video or sequence of images, then determining its exact position in the next successive frames and finally generating the corresponding trajectory of this object. Visual tracking, however, is still a challenging problem in practice while taking into account the abrupt appearance changes of the target objects induced by their non-rigid transformation, the sophisticated lighting variation, the obstruction by the block or similar objects in the background and the camera jitter. Motivated by the successful applications in target detection and recognition in recent years, plenty of deep learning models have been integrated in the visual tracking and better performance over traditional methods was achieved in a series of data evaluations, which opens a new door in the field of visual tracking. In this paper, the overview and progress on visual tracking were summarized. The current challenges and corresponding solving approaches in this field are introduced firstly and in particular, several novel and mainstream visual tracking algorithms based on the deep learning are specially described and analyzed in details, including their basic ideas, advantages and disadvantages and future prospect.

Changing environments pose a serious problem to current robotic systems aiming at long term operation. While place recognition systems perform reasonably well in static or low-dynamic environments, severe appearance changes that occur between day and night, between different seasons or different local weather conditions remain a challenge. In this paper we propose to learn to predict the changes in an environment. Our key insight is that the occurring appearance changes are in part systematic, repeatable and therefore predictable. The goal of our work is to support existing approaches to place recognition by learning how the visual appearance of an environment changes over time and by using this learned knowledge to predict its appearance under different environmental conditions. We describe the general idea of appearance change prediction (ACP) and a novel implementation based on vocabularies of superpixels (SP-ACP). Despite its simplicity, we can further show that the proposed approach can improve the performance of SeqSLAM and BRIEF-Gist for place recognition on a large-scale dataset that traverses an environment under extremely different conditions in winter and summer.

Objective. Accurate target localization is essential for effective radiation therapy, yet respiratory motion introduces considerable uncertainty. Ultrasound-based motion tracking offers a non-invasive solution, but the presence of similar anatomical structures and significant changes in appearance severely hinders robust and accurate tracking. Robust tracking is essential to ensure accuracy and improve treatment outcomes.Approach. We propose SiamAT, a tracking framework comprising a Siamese-like feature extraction network with a Swin-Transformer backbone, an attention-based target-aware module, and multi-task prediction heads. The target-aware module adaptively integrate template and search features to generate target-aware representations for precise localization.Main results. The proposed SiamAT is evaluated on a public dataset, i.e. MICCAI 2015 challenge on liver US tracking, and our clinical dataset provided by the Chinese People's Liberation Army General Hospital. Experimental results demonstrate that the method achieves accurate and robust tracking(0.60±0.30 mm and 0.47±0.31 mm tacking errors in two datasets) and outperforms existing methods. The proposed method runs at approximatively 36 fps on GPU.Significance. The proposed SiamAT provides clinicians with an effective tool for analyzing 2D motion in US images in clinical settings, assisting in the planning of optimal treatment strategies while reducing diagnostic and therapeutic risks.

Person re-identification is a challenging task because of severe appearance changes of a person due to diverse camera viewpoints and person poses. To alleviate the impact of different poses, more and more studies have been done in pose estimation. In this work, we present three task-specific neural networks (TNN) algorithm designed to address the problem of pose estimation for re-identification in both single-shot and multi-shot matching. In order to recognize the human pose as one of the four classes (front, back, left, right), a PoseNet-A is first required to estimate the pose as class front-back or class left-right. Based on the results, we select the appropriate network (PoseNet-B1, PoseNet-B2) to obtain the final pose. According to the results, our method achieves very good results on a large data set (CUHK03-Pose). One thing that needs to be pointed out is that we build the dataset CUHK03-Pose which is based on the person re-identification dataset CUHK03.

Inferring ego position by recognizing previously seen places in the world is an essential capability for autonomous mobile systems. Recent advances have addressed increasingly challenging recognition problems, e.g. long-term vision-based localization despite severe appearance changes induced by changing illumination, weather or season. Since robots typically move continuously through an environment, there is high correlation within consecutive sensory inputs and across similar trajectories. Exploiting this sequential information is a key element of some of the most successful approaches for place recognition in changing environments. We present a novel, neurally inspired approach that uses sequences for mobile robot localization. It builds upon Hierarchical Temporal Memory (HTM), an established neuroscientific model of working principles of the human neocortex. HTM features two properties that are interesting for place recognition applications: (1) It relies on sparse distributed representations, which are known to have high representational capacity and high robustness towards noise. (2) It heavily exploits the sequential structure of incoming sensory data. In this paper, we discuss the importance of sequence information for mobile robot localization, we provide an introduction to HTM, and discuss theoretical analogies between the problem of place recognition and HTM. We then present a novel approach, applying a modified version of HTM’s higher order sequence memory to mobile robot localization. Finally we demonstrate the capabilities of the proposed approach on a set of simulation-based experiments.

Recently, the methods based on Convolutional Neural Networks (CNNs) have achieved state-of-the-art performance in visual place recognition. CNN is a class of multilayer perceptrons, but unlike common multilayer perceptrons that it is not usually fully connected networks. It can acquire more general image features and make the image processing computationally manageable through filtering the connections by proximity. In this paper, we utilize the deep features generated by CNNs and the dynamic time warping (DTW) algorithm for image sequence place recognition. We propose a novel image similarity measurement, which is derived from cosine distance and can better distinguish match and mismatch. Meanwhile, we improve the DTW algorithm to design a local matching method that can reduce time complexity from O(n3) to O(n). To test the proposed method, four datasets (Nordland, Gardens Point, St. Lucia, and UoA datasets) are used as benchmarks; using two traverses in each dataset with one for reference and the other for testing. The results show high precision-recall characteristics of our method in the cases of severe appearance changes. Besides, our method achieves substantial improvements over the methods using the deep feature representations of a single image for recognition, which reflects that the spatiotemporal information contained in the image sequence is significant for the task of visual place recognition. Moreover, the proposed method also shows to outperform the classical sequence-based method SeqSLAM.

Objects with non-Lambertian reflecting surfaces suffer severe appearance changes according to the change of observer's angle of view. Thus it is difficult to obtain a consistent object model. This paper addresses the problem of bin picking for such objects without using explicit object models. Due to the above-mentioned reasons, traditional pattern matching approaches are difficult to apply. Furthermore, 3D sensors often fail to capture these objects correctly, which also disables the approaches based on 3D object models. Considering these limitations, this paper proposes a novel algorithm that tries to find a reasonable pickup point by investigating the status of the pile in the bin instead of trying to perfectly isolate single object from the pile and measure full pose information. For the purpose, feature point matching between stereo cameras is adopted to retrieve approximate 3D information of the pile. Additionally, by associating these corresponding point pairs with the binary patches around the point, corresponding binary patch pairs are detected, which is evaluated by a proposed objective function to select the most appropriate candidate for the pickup operation.

Visual place recognition is the problem of camera-based localization of a robot given a database of images of known places, potentially under severe appearance changes (e.g., different weather or illumination). Typically, the current query images and the database images are sequences of consecutive images. In previous work, we proposed to adapt hierarchical temporal memory, a biologically plausible model of sequence processing in the human neocortex to address this task. The previous work described the algorithmic steps and showed synthetic experimental results from simulation. This letter extends the approach to application on real-world data based on a novel encoder for state-of-the-art image processing front ends. The neurologically inspired approach is compared with several state-of-the-art algorithms on a variety of datasets and shows preferable performance. Beyond the place recognition performance, the neurological roots of the algorithm result in appealing properties like potentially very energy efficient implementation due to the usage of sparse distributed representations and natural extendability like the integration of motion estimates similar to entorhinal grid cells. Finally, we underline its practical applicability by online, soft real-time application on a mobile robot.

Appearance-invariant place recognition by discriminatively training a convolutional neural network

Person re-identification is the problem of recognizing people across images or videos from non-overlapping views. Although there has been much progress in person re-identification for the last decade, it still remains a challenging task because of severe appearance changes of a person due to diverse camera viewpoints and person poses. In this paper, we propose a novel framework for person reidentification by analyzing camera viewpoints and person poses, so-called Pose-aware Multi-shot Matching (PaMM), which robustly estimates target poses and efficiently conducts multi-shot matching based on the target pose information. Experimental results using public person reidentification datasets show that the proposed methods are promising for person re-identification under diverse viewpoints and pose variances.

Visual place recognition is a mature field in mobile robotics research. Recognizing places in datasets covering traversals of hundreds or thousands of kilometres and accurate localization in small and medium size environments have been successfully demonstrated. However, for real world long term operation, visual place recognition has to face severe environmental appearance changes due to day-night cycles, seasonal or weather changes. Existing approaches for recognizing places in such changing environments provide solutions for matching images from the exact same viewpoint using powerful holistic descriptors, using less sophisticated holistic descriptors in combinations with images sequences, and/or pose strong requirements on training data to learn systematic appearance changes. In this paper, we present a novel, training free, single image matching procedure that builds upon local region detectors for powerful Convolutional Neural Network (CNN) based descriptors. It can be used with a broad range of local region detectors including keypoints, segmentation based approaches and object proposals. We propose a novel evaluation criterion for selection of an appropriate local region detector for changing environments and compare several available detectors. The scale space extrema detector known from the SIFT keypoint detector in combination with appropriate magnification factors performs best. We present preliminary results of the proposed image matching procedure with several region detectors on the challenging Nordland dataset on place recognition between different seasons and a dataset including severe viewpoint changes. The proposed method outperforms the best existing holistic method for place recognition in such changing environments and can additionally handle severe viewpoint changes. Additionally, the combination of the best performing detectors with superpixel based spatial image support shows promising results.

Robust object tracking using semi-supervised appearance dictionary learning