Robust robot localization with visually adaptive consensus filters in dynamic corridor environments

The state-of-art simultaneous localization and mapping (SLAM) methods have been demonstrated of satisfactory results in static environments. However, most of the existing methods cannot be directly applied in dynamic environments to account for moving objects. To solve the practical problem of SLAM in dynamic environments, an RGB-D SLAM method is developed based on moving objects removal and dense map reconstruction. Specifically, deep learning-based instance segmentation is performed to obtain the semantic information which is furthermore combined with multiview geometry theory to detect possible moving objects. Kalman filter and feature fusion algorithms are further used for moving objects tracking to improve the detection accuracy. Then the camera pose is estimated using the static feature points after motion removal to realize reliable localization in dynamic environments. Out of concern for the infeasibility of navigation and perception with sparse feature maps, each-single-view of dense map is constructed using the RGB image and the depth map collected by the RGB-D camera after motion removal. The dense environmental map is finally reconstructed through the registration of point cloud of selected key image frames. Experiments are conducted on public datasets and real scene data to evaluate the performance of the proposed method. Compared with the state-of-art method ORB-SLAM2, the absolute trajectory error and the relative pose error obtained by the proposed method are significantly reduced by at least 91%, and the localization accuracy reaches 0.001 m.

Visual localization has been well studied in recent decades and applied in many fields as a fundamental capability in robotics. However, the success of the state of the arts usually builds on the assumption that the environment is static. In dynamic scenarios where moving objects are present, the performance of the existing visual localization systems degrades a lot due to the disturbance of the dynamic factors. To address this problem, we propose a novel sparse motion removal (SMR) model that detects the dynamic and static regions for an input frame based on a Bayesian framework. The similarity between the consecutive frames and the difference between the current frame and the reference frame are both considered to reduce the detection uncertainty. After the detection process is finished, the dynamic regions are eliminated while the static ones are fed into a feature-based visual simultaneous localization and mapping (SLAM) system for further visual localization. To verify the proposed method, both qualitative and quantitative experiments are performed and the experimental results have demonstrated that the proposed model can significantly improve the accuracy and robustness for visual localization in dynamic environments.<;/p><;p><;i>Note to Practitioners<;/i>-This article was motivated by the visual localization problem in dynamic environments. Visual localization is well applied in many robotic fields such as path planning and exploration as the basic capability for a mobile robot. In the GPS-denied environments, one robot needs to localize itself through perceiving the unknown environment based on a visual sensor. In real-world scenes, the existence of the moving objects will significantly degrade the localization accuracy, which makes the robot implementation unreliable. In this article, an SMR model is designed to handle this problem. Once receiving a frame, the proposed model divides it into dynamic and static regions through a Bayesian framework. The dynamic regions are eliminated, while the static ones are maintained and fed into a feature-based visual SLAM system for further visual localization. The proposed method greatly improves the localization accuracy in dynamic environments and guarantees the robustness for robotic implementation.

Current simultaneous localization and mapping (SLAM) algorithms perform well in static environments but easily fail in dynamic environments. Recent works introduce deep learning-based semantic information to SLAM systems to reduce the influence of dynamic objects. However, it is still challenging to apply a robust localization in dynamic environments for resource-restricted robots. This paper proposes a real-time RGB-D inertial odometry system for resource-restricted robots in dynamic environments named Dynamic-VINS. Three main threads run in parallel: object detection, feature tracking, and state optimization. The proposed Dynamic-VINS combines object detection and depth information for dynamic feature recognition and achieves performance comparable to semantic segmentation. Dynamic-VINS adopts grid-based feature detection and proposes a fast and efficient method to extract high-quality FAST feature points. IMU is applied to predict motion for feature tracking and moving consistency check. The proposed method is evaluated on both public datasets and real-world applications and shows competitive localization accuracy and robustness in dynamic environments. Yet, to the best of our knowledge, it is the best-performance real-time RGB-D inertial odometry for resource-restricted platforms in dynamic environments for now. The proposed system is open source at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/HITSZ-NRSL/Dynamic-VINS.git</uri>

Camera localization from monocular images has been a long-standing problem, but its robustness in dynamic environments is still not adequately addressed. Compared with classic geometric approaches, modern CNN-based methods (e.g. PoseNet) have manifested the reliability against illumination or viewpoint variations, but they still have the following limitations. First, foreground moving objects are not explicitly handled, which results in poor performance and instability in dynamic environments. Second, the output for each image is a point estimate without uncertainty quantification. In this paper, we propose a framework which can be generally applied to existing CNN-based pose regressors to improve their robustness in dynamic environments. The key idea is a prior guided dropout module coupled with a self-attention module which can guide CNNs to ignore foreground objects during both training and inference. Additionally, the dropout module enables the pose regressor to output multiple hypotheses from which the uncertainty of pose estimates can be quantified and leveraged in the following uncertainty-aware pose-graph optimization to improve the robustness further. We achieve an average accuracy of 9.98m/3.63° on RobotCar dataset, which outperforms the state-of-the-art method by 62.97%/47.08%. The source code of our implementation is available at https://github.com/zju3dv/RVL-dynamic.

Mobile robots, in modern technology, demand a more robust localization in a complex environment. Currently, the most commonly used 2D LiDAR localization system for mobile robots requires maps that are constructed by 2D SLAM. Such systems do not cope well with dynamic environments and also have high deployment costs when moving robots to a new environment setting as they require the reconstruction of a map for each new place. In modern days, a floor plan is indispensable for an indoor environment. It typically represents essential structures such as walls, corners, pillars, etc. for humans to navigate in the environment. This information turns out to be crucial for robot localization. In this paper, we propose an approach for 2D LiDAR localization in an architectural floor plan. We use partial simultaneous localization and mapping (PSLAM) algorithm to generate a map while we concurrently aligned it to the floor plan using Monte Carlo Localization (MCL) method. Real-world experiments have been conducted with our proposed method which results in robust robot localization, the algorithm is even evaluated on a large discrepancies floor plan (discrepancies between the floor plan and real-world). Our algorithm demonstrates that its capabilities of localizing in real-time applications.

Accurate object detection is important for the safe navigation of autonomous vehicles (AVs) in dynamic environments. Conventional AVs rely on high-resolution but bulky, topmounted 3D Light Detection And Ranging (LiDAR) sensors, which are impractical for scalable production and computationally expensive to process. To address this, we utilize quasi-2D LiDAR sensors, which offer a compact and scalable solution for AVs. However, point clouds generated by these sensors have a narrow vertical field of view and contain a limited number of points compared to 3D LiDARs, which makes object detection significantly more difficult. To overcome this challenge, we transform the quasi-2D point clouds into Bird's-Eye-View (BEV) images and apply four different input encoding strategies. These encodings enhance the spatial and temporal context of the data, making them suitable for efficient deep-learning-based 2D object detection models to perform real-time detection with minimal computational overhead. In contrast to other approaches for 2D LiDARs, we are able to detect not only vehicles but also pedestrians and cyclists. We use a large real-world dataset to compare our proposed input encodings combined with three different state-of-the-art 2D object detection models. Our results demonstrate that these input encodings enhance object detection performance across all models, significantly reducing false positives, improving heading angle prediction, and boosting the detection accuracy of smaller traffic participants.

Indoor localization service is an indispensable part of modern intelligent life, among which Wi-Fi based fingerprint localization system is popular in indoor positioning researches due to its advantages of low cost and widely deployment. However, Wi-Fi based localization system is susceptible to dynamic environment, and fingerprint collection and updating are time-consuming and labor-intensive. To address this problem, we propose a novel positioning framework based on multiple transfer learning fusion using Generalized Policy Iteration (GPI). Firstly, a 1-Dimension Convolutional Autoencoder (1-D CAE) is designed to extract features from one-dimensional fingerprint data; similar to Convolutional Neural Network (CNN), it can not only pay more attention to the information of different dimensions of fingerprints, but also compress redundant information and reduce noise. After that, Domain Adversarial Neural Network (DANN) and Passive Aggressive (PA) algorithm are fused to train localization model based on unlabeled fingerprint of target domain using the theory of GPI in offline stage. Finally, the model is fine-tuned with unlabeled fingerprints and few labeled fingerprints in daily online predictions to improve the performance of the localization system. Various evaluations in five typical scenarios validate the effectiveness of proposed algorithm in dynamic environment, with low tendency, easy recalibration, long-term stabilization high accuracy and so on.

With the advancement of intelligent perception in multirobot systems, cooperative localization in dynamic environments has become a critical component. However, existing multirobot cooperative localization systems still fall short in meeting high-precision localization requirements in Global Navigation Satellite System (GNSS)-denied environments. In this paper, we propose a factor-graph-based resilient cooperative localization (FG-RCL) algorithm for multirobot systems. This algorithm integrates measurements from visual sensors and Ultra-WideBand (UWB) to achieve accurate cooperative state estimation—overcoming the visibility issues of visual sensors within limited fields of view. We utilize the Joint Probabilistic Data Association (JPDA) algorithm to calculate the corresponding probabilities of multiple visual detection measurements between robots and assign them to their respective edges in the factor graph, thereby addressing the data association challenges in visual detection measurements. Finally, simulation results demonstrate that the proposed algorithm significantly reduces the influence of visual detection measurement interference on the performance of cooperative localization. Experimental results indicate that the proposed algorithm outperforms UWB-based and vision-based methods in terms of localization accuracy. The system is implemented using a factor-graph-based optimization approach, and it exhibits scalability and enables plug-and-play for sensors. Furthermore, it demonstrates resilience in abnormal situations.

Accurate localization in dynamic environments is a critical challenge for autonomous systems, particularly in GPS-denied settings and under sensor noise or failure conditions. This review paper explores state-of-the-art sensor fusion and localization techniques, including Kalman filters, particle filters, and machine learning-based approaches. The paper identifies key challenges such as operating in GPS-denied environments, managing sensor noise and failure, and ensuring scalability and real-time processing in complex scenarios. To address these issues, the paper proposes enhanced sensor fusion methods, advanced localization algorithms, and hybrid approaches that integrate traditional techniques with machine learning. These innovations are designed to improve autonomous systems' accuracy, reliability, and adaptability in increasingly complex and unpredictable environments. The paper also outlines validation strategies to ensure the effectiveness of these new methodologies, paving the way for future advancements in the field.

Received signal strength (RSS) fingerprint map is one of the most widely-used indoor localization approaches, but it often relies on multiple access points (AP) for data collection and suffers from frequent data updates due to dynamic wireless environments. In this work, we implement a reconfigurable-intelligent-surface (RIS) assisted indoor localization system named Meta2Locate to tackle the above issues using only one AP. In the proposed system, we deploy our self-designed RIS at 5.5GHz in an indoor environment, which can customize the propagation channels between the AP and the target. For the changing propagation environment, we design a mean maximum discrepancy weighted meta-learning approach to train a model that maps the RSS fingerprint to the location of the user, and it only needs a few data for the model update.

In the past, there has been a tremendous amount of progress in the area of autonomous robot navigation, and a large variety of robots have been developed that demonstrated robust navigation capabilities indoors, in nonurban outdoor environments, or on roads; relatively few approaches have focused on navigation in urban environments such as city centers. Urban areas, however, introduce numerous challenges for autonomous robots as they are rather unstructured and dynamic. In this paper, we present a navigation system for mobile robots designed to operate in crowded city environments and pedestrian zones. We describe the different components of this system, including a simultaneous localization and mapping module for dealing with huge maps of city centers, a planning component for inferring feasible paths, taking into account the traversability and type of terrain, a module for accurate localization in dynamic environments, and the means for calibrating and monitoring the platform. Our navigation system has been implemented and tested in several large‐scale field tests, in which a real robot autonomously navigated over several kilometers in a complex urban environment. This also included a public demonstration, during which the robot autonomously traveled along a more than 3‐km‐long route through the city center of Freiburg, Germany.

Sound source localization in dynamic environments with multiple moving speakers presents significant challenges due to reverberation, noise, and unknown source counts. To address these issues, this paper proposes an integrated deep-learning framework combining spatial spectrum estimation with blind source detection. The method employs a causal convolution–recurrent network (SRP-DPCRN) to extract robust spatial features from multichannel audio signals under adverse acoustic conditions. Subsequently, an iterative attention-based detection network (IASDNet) automatically identifies active sources from the estimated spatial spectrum without requiring prior knowledge of source quantity. Evaluated on both simulated datasets and the real-recorded LOCATA benchmark, the proposed system demonstrates superior performance in multi-source tracking scenarios, achieving an average detection accuracy of 96% with mean angular error below 3.5 degrees. The results confirm that joint optimization of feature learning and source counting provides an effective solution for blind localization in practical applications, significantly outperforming conventional and deep-learning baselines.

Odometry estimation is a fundamental technique for many robotics and autonomous driving applications. Recent advances using light detection and ranging (LiDAR) and inertial measurement unit (IMU) sensor-fusion methods show promising potential in odometry estimation by combining the precise perception of LiDAR and the high-frequency motion estimation from IMU sensors. In this article, we propose a novel uncertainty-aware LiDAR-inertial odometry (LIO) algorithm designed for autonomous vehicles operating in urban driving environments. Unlike previous approaches that use point-to-point or point-to-plane methods for updates, in this article, we employ a distribution-to-distribution approach for updates. Each point sampled on a surface is modeled as a Gaussian distribution, and the covariance is estimated from the decomposed eigenvalues by considering the correlation between the current point and its surrounding points. The estimated covariance makes the update module aware of match quality, allowing it to ignore poorly matched points and focus on well-matched ones, thereby improving odometry accuracy. In addition, it eliminates Z-axis drift in long-term odometry estimation by using ground plane information and linearly adjusts pose uncertainty based on optimized pose values. This unified approach to managing uncertainty is essential for the system’s long-term stability and accuracy. We thoroughly evaluated our method using multiple publicly available datasets. The experimental results show that our method is accurate and reliable in dynamic urban environments and achieves state-of-the-art LIO performance with fast speed and strong generalization ability. We will release the code of our method here: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Gatsby23/UA-LIO.git</uri>.

To address the degradation of visual simultaneous localization and mapping (VSLAM) pose estimation accuracy caused by extensive moving objects and the resulting contamination of feature points in indoor dynamic environments, this study proposes a dynamic feature point rejection strategy that integrates a You Only Look Once (YOLO) family detector, YOLO11, with the Lucas–Kanade (LK) optical flow method within the ORB_SLAM3 framework. First, YOLO11 is employed to detect potentially dynamic targets such as pedestrians in the image, and ORB features extracted within the detection bounding boxes are labeled as high-risk dynamic points. Subsequently, based on pyramidal LK optical flow, the motion consistency of feature points is analyzed across multiple frames, and a secondary discrimination is performed on features located at the edges and in the vicinity of the detection boxes. In this way, feature points that conform to a static motion model are maximally preserved, thereby alleviating the excessive removal and misclassification of static points caused by oversized detection boxes. On this basis, a new map is reconstructed using the filtered static feature points, and the pose estimation process is further refined. Experiments on multiple indoor dynamic scenarios demonstrate that, compared with the original ORB_SLAM3, the proposed method effectively suppresses dynamic interference, enhances the stability of feature points, and significantly improves pose estimation accuracy and trajectory continuity, thus providing a valuable reference for robust VSLAM applications in dynamic environments.

During the continual transitions from lab research to real-world applications of vision-based algorithms, there are significant challenges, e.g. the robustness to adapt to complex environments and the high demands of multi-task, multi-modal learning models. More specifically on visual simultaneous localization and mapping (visual SLAM) for mobile robots, two notable limitations are: (i) the drift issue during pose estimation, especially in dynamic environments, makes the positioning system unstable; (ii) the additional loop detection takes many computational resources for image retrieval and global feature geometry checking. This thesis explores the approaches of visual localization on unmanned ground vehicles by solving those limitations. The conventional feature extraction and matching pipeline for vision-based tasks involve three steps: (i) local feature detection and description, (ii) feature matching, and (iii) outlier rejection. The first contribution of the thesis is adding a feature selection and anticipation stage to reduce tracking drift. We explore the noise model of image features to select a subset of all the observed image features with the best ”contribution” during data association and pose estimation across multiple frames. Conventional SLAM algorithms take a strong assumption of scene rigidity, which limits the application under challenging environments. The second part of the thesis addresses the tough issue in dynamic environments with moving objects. We presented GMC, namely the motion clustering approach, a lightweight dynamic object filtering method. It can distinguish moving objects from static landmarks. Based on the theory of motion coherence within a particular image area, GMC could segment dynamic objects in 3D space. We can provide an efficient and robust correspondence algorithm that can extract dynamic objects from a static background with the method. In this way, we propose a dynamic SLAM system that is real-time and free from expensive GPU processors. In contrast to GMC, the thesis’s third part turns to an object-aware learning-based model for more general dynamic scenarios. We use object detection and tracking as points, lines, planes, etc. We utilize semantic information and extract sparse image features simultaneously to keep track of dynamic objects. The static background and different dynamic objects are jointly optimized in a newly developed bundle adjustment sliding window. The estimated 3D bounding boxes can provide more robust camera tracking and better scene understanding, and better map merging. The fourth part of the thesis leverages the emerging feature learning framework. It proposes a unified self-supervised model called LGDNet to generate both global and local image feature descriptors end-to-end. Global feature descriptors embed the whole image into a compact representation, leading to easier scene comparison. On the other hand, local features focus more on the local region similarities of some salient parts for structure from motion. Our proposed method can directly extract features together with descriptors that encode both local maximum responses and global context information, avoiding duplicate calculations based on different feature extraction criteria.