3D Object Global Localization Using Particle Filter with Back Projection-based Sampling on Saliency

The next generation of service robots is to offer services relying heavily on visually guided manipulation, besides navigation, such services as errand, logistics, appliance, home keeping, etc. For the successful introduction of these services, it is critical to establish the consumer-level dependability in 3D object recognition and pose estimation in a natural setting where a large variation of environment, e.g., perspective, texture, form factor, illumination, occlusion, etc., is common. To address this problem, we propose an approach of the two-layered particle filter to the dependability in 3D object recognition and pose estimation. In the upper layer, a set of object pose candidates is identified and maintained in the search space as a set of super-particles, each of which is assigned a probability of the true pose and evolved in time along with the accumulation of further evidences. To define the object pose candidates, first, we acquire initially weak evidences quickly and interpret them in terms of possible object poses in space. These interpretations serve as the region of interest for detailed investigation by which the pose probabilities are computed for individual interpretations based on the likelihood and unlikelihood of various features available in the corresponding regions of interest. During the process of probability computation, we select the object pose candidates to be used as super-particles in the upper layer. In the lower level, the pose uncertainties associated with the individual candidates are represented as particles that are subject to the propagation in time. Finally, the experimental results support the strength of the proposed approach in the real environment in terms of its dependability in 3D object recognition and pose estimation.

We present two data-driven importance distributions for particle filter-based articulated tracking; one based on background subtraction, another on depth information. In order to keep the algorithms efficient, we represent human poses in terms of spatial joint positions. To ensure constant bone lengths, the joint positions are confined to a non-linear representation manifold embedded in a high-dimensional Euclidean space. We define the importance distributions in the embedding space and project them onto the representation manifold. The resulting importance distributions are used in a particle filter, where they improve both accuracy and efficiency of the tracker. In fact, they triple the effective number of samples compared to the most commonly used importance distribution at little extra computational cost.

Informative path planning is an important and challenging problem in robotics that remains to be solved in a manner that allows for wide-spread implementation and real-world practical adoption. Among various reasons for this, one is the lack of approaches that allow for informative path planning in high-dimensional spaces and non-trivial sensor constraints. In this work we present a sampling-based approach that allows us to tackle the challenges of large and high-dimensional search spaces. This is done by performing informed sampling in the high-dimensional continuous space and incorporating potential information gain along edges in the reward estimation. This method rapidly generates a global path that maximizes information gain for the given path budget constraints. We discuss the details of our implementation for an example use case of searching for multiple objects of interest in a large search space using a fixed-wing UAV with a forward-facing camera. We compare our approach to a sampling-based planner baseline and demonstrate how our contributions allow our approach to consistently out-perform the baseline by 18.0%. With this we thus present a practical and generalizable informative path planning framework that can be used for very large environments, limited budgets, and high dimensional search spaces, such as robots with motion constraints or high-dimensional configuration spaces. [Code] <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</sup> Codebase: https://github.com/castacks/tigris [Video] <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sup> Video: https://youtu.be/bMw5nUGL5GQ

A multiple object tracking approach that combines colour and depth information using a confidence measure

Electrocution accidents caused by operation and maintenance personnel and high-voltage live equipment frequently occur in substations. Although many cameras have been installed for the surveillance of critical energized equipment, they cannot complete the monitoring of safe distance. In addition, the current research based on vision or LiDAR suffers from computationally intensive, lack of real-time or expensive equipment. In order to meet the low cost, easy maintenance and real-time requirements of substation safe distance monitoring, a monocular vision method based on 2D-3D fusion is proposed in this study. Specifically, instance segmentation, depth estimation, depth reconstruction, and back projection transformation are used to predict the 3D distance of objects in 2D images. The work is mainly implemented in two aspects: a) To obtain the high-quality masks of Power Transformers and Person, SOLOv2 is optimized on three aspects: feature extraction, feature fusion and ability to cope with object geometric transformation. b) Pseudo-LiDAR output from RGB images via depth estimation and back projection techniques. Subsequently, combined with the point cloud of the object and the actual-to-virtual conversion ratio, the actual distance between Power Transformer and Person can be calculated. Experimental results show that the maximum error of ranging is within 9%, with an average error rate of 5.34% on the Power Transformer-Person dataset. It can be seen that the proposed method has achieved good ranging effects, and can realize the automatic measurement of the safe distance of Power Transformer.

Conventional evolutionary algorithms operate in a fixed search space with limiting parameter range, which is often predefined via a priori knowledge or trial and error in order to `guess' a suitable region comprising the global optimal solution. This requirement is hard, if not impossible, to fulfil in many real-world optimization problems since there is often no clue of where the desired solutions are located in these problems. Thus, this paper proposes an inductive---deductive learning approach for single- and multi-objective evolutionary optimization. The method is capable of directing evolution towards more promising search regions even if these regions are outside the initial predefined space. For problems where the global optimum is included in the initial search space, it is capable of shrinking the search space dynamically for better resolution in genetic representation to facilitate the evolutionary search towards more accurate optimal solutions. Validation results based on benchmark optimization problems show that the proposed inductive---deductive learning is capable of handling different fitness landscapes as well as distributing nondominated solutions uniformly along the final trade-offs in multi-objective optimization, even if there exist many local optima in a high-dimensional search space or the global optimum is outside the predefined search region.

Over the past few years, particle filters have been applied with great success to a variety of state estimation problems. In this paper we present a statistical approach to increasing the efficiency of particle filters by adapting the size of sample sets during the estimation process. The key idea of the KLD-sampling method is to bound the approximation error introduced by the sample-based representation of the particle filter. The name KLD-sampling is due to the fact that we measure the approximation error using the Kullback-Leibler distance. Our adaptation approach chooses a small number of samples if the density is focused on a small part of the state space, and it chooses a large number of samples if the state uncertainty is high. Both the implementation and computation overhead of this approach are small. Extensive experiments using mobile robot localization as a test application show that our approach yields drastic improvements over particle filters with fixed sample set sizes and over a previously introduced adaptation technique.

Chaotic particle filter for visual object tracking

Effectively recognizing human activities requires at least 32 joint related degrees of freedom to be estimated so as to reliably track the human body in 3D. The particle filter is robust to distracting clutter by maintaining multiple hypotheses for each of these joint angles. Real-time tracking is difficult however with the computational overhead of such a large search space. This paper optimizes this search space utilizing feedback from a continuous human activity recognition (CHAR) system and improves the robustness and efficiency of each particle calculation using a novel body model. The joint angles are estimated for the next frame using a particle filter with forward smoothing. A new paradigm enables the temporal segmentation of continuous motion into dynemes. Using HMM, the CHAR system attempts to infer the human movement skill that could have produced the observed sequence of dynemes. Hundreds of movement skills, from gait to saltos, are successfully tracked and recognized.

We propose a family of methods for biomedical image time series registration based on Particle filtering. The first method applies an intensity-based information-theoretic approach to calculate importance weights. An effective second group of methods use landmark-based approaches for the same purpose by automatically detecting intensity maxima or SIFT interest points from image time series. A brute-force search for the best alignment usually produces good results with proper cost functions, but becomes computationally expensive if the whole search space is explored. Hill climbing optimizations seek local optima. Particle filtering avoids local solutions by introducing randomness and sequentially updating the posterior distribution representing probable solutions. Thus, it can be more robust for the registration of image time series. We show promising preliminary results on dendrite image time series.

Code optimization involves the application of rules and algorithms to program code, with the goal of making it faster, smaller, more efficient, and so on. Applying the right compiler optimizations to a particular program can have a significant impact on program performance. The effectiveness of compiler optimizations is determined by the combination of target architectures, target application, and the compilation environment, which is defined by the setting of the compiler optimizations and compiler heuristics. Finding a compiler setting which is optimal requires a delicate tradeoff between these factors. Due to the non-linear interaction of compiler optimizations, however, determining the best setting is nontrivial. The trivial solution of trying all combinations of techniques would be infeasible, as it is of complexity O (2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> ) even for "n" on-off optimizations. There have been several proposed techniques that search the space of compiler options to find good solutions; however such approaches can be expensive. In current compilers, through command line arguments, the user must decide which optimizations are to be applied in a given compilation run. Clearly, this is not a long-term solution. As compiler optimizations get increasingly numerous and complex, this problem must find an automated solution. In this paper, a new technique is suggested, which prunes the large search space using branch and bound technique, so that only the area in search space which is most beneficial is given higher priority for further exploration whereas the least promising regions are straightaway pruned off, thus saving time by not exploring those regions which give a handful or negligible benefits. Also further probes of the search space tree are halted, once it is determined that the relative improvement in the time is not considerable as compared to the cost incurred for further probes. The time complexity of proposed method under worst case is found to be of O (n <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and under best case it can be even O (n).

To become robust, a tracking algorithm must be able to support uncertainty and ambiguity often inherently present in the data in form of occlusion and clutter. This comes usually at the price of more demanding computations. Sampling methods, such as the popular particle filter, accommodate this capability and provide a means of controlling the computational trade-off by adapting their resolution. This paper presents a method for adapting resolution on-the-fly to current demands. The key idea is to select the number of samples necessary to populate the high probability regions with a predefined density. The scheme then allocates more particles when uncertainty is high while saving resources otherwise. The resulting tracker propagates compact while consistent representations and enables for reliable real time operation otherwise compromised.

To localize mobile target timely and improve the accuracy of the target state estimation in wireless sensor networks, this paper presents a novel mobile target localization algorithm. By using the mean shift algorithm to construct the proposal distribution to make use of the current measurements, and weighting particles according to the virtual hamming distances in the particle filter, this proposed algorithm improves the accuracy of the target state estimation and reduces the necessary number of samples. It has lower computation and communication cost than the conventional particle filter because of more accurate state estimate of samples. Extensive simulation results confirm that this localization scheme outperforms conventional particle filter and its localization accuracy is comparable to the unscented particle filter, but its computation cost is 50% less than that of UPF.

A multiple color-filter aperture (MCA) camera can provide depth information as well as color and intensity in the single-camera framework, where the MCA generates misalignment between color channels depending on the distance of a region-of-interest. In this paper, we present a simultaneous object tracking and depth estimation approach based on the color shifting property of an MCA camera. An object region is first extracted by using Markov Chain Monte Carlo (MCMC) sampling-based particle filter. The extracted object's region has color misalignment among RGB planes depending on the distance of the object. We estimate color shifting vectors (CSVs) between green-and-red (G-R) and green-and-blue (G-B) channels using a simplified elastic registration algorithm. Using the color shifting property of the MCA camera, the depth of the object's region is estimated from CSVs. From experimental results, we can show the MCA camera-based surveillance system can estimate depth information as well as track object.

SIFT (scale invariant feature transform) is one of the most robust and the widely used image matching algorithms based on local features. It has been used in the aspect of the cameras target matching. For its high complexity, this paper proposes a target matching method for multiple cameras by combining SIFT and the rotation-invariant LBP (local binary patterns). The object detection is obtained by improved Back-ground subtraction technique, and particle filter is used for region-based object tracking. The descriptor combining SIFT and the rotation-invariant LBP is calculated for every object region. The object matching is obtained by comparing the distance of feature matrix. Experimental results show that the proposed algorithm is more rapid than the standard SIFT algorithm while the performance is favorably compared to the standard SIFT algorithm, and it is robust in various scenes.