Dynamic to Static Lidar Scan Reconstruction Using Adversarially Trained Auto Encoder

MOVES: Movable and moving LiDAR scene segmentation in label-free settings using static reconstruction

In recent years, the utilization of visual SLAM with a camera as a sensor has become increasingly widespread, particularly in the context of rapidly developing artificial intelligence such as mobile robots, VR, and AR. This approach is favored due to its affordability, lightweight design, ability to capture comprehensive information, and other advantages. The traditional slam technology has achieved a very mature effect with the static scene as the assumption condition, and the classic representatives are LSD-SLAM and ORB-SLAM which will be introduced in the following. However, since dynamic scenarios are unavoidable in the real world, overcoming the influence of dynamic objects becomes a challenge if researchers want to move forward with more applications. At present, under the dynamic environment of the visual slam algorithm faces positioning problems of low accuracy and poor robustness. To address the challenges posed by dynamic objects in a scene, many researchers are incorporating deep learning techniques and exploring the use of reference semantic information to collaboratively resolve the issue. This paper reviews this and summarizes the development process and important algorithms.

The early applications of Visual Simultaneous Localization and Mapping (VSLAM) technology were primarily focused on static environments, relying on the static nature of the environment for map construction and localization. However, in practical applications, we often encounter various dynamic environments, such as city streets, where moving objects are present. These dynamic objects can make it challenging for robots to accurately understand their own position. This paper proposes a real-time localization and mapping method tailored for dynamic environments to effectively deal with the interference of moving objects in such settings. Firstly, depth images are clustered, and they are subdivided into sub-point clouds to obtain clearer local information. Secondly, when processing regular frames, we fully exploit the structural invariance of static sub-point clouds and their relative relationships. Among these, the concept of the sub-point cloud is introduced as novel idea in this paper. By utilizing the results computed based on sub-poses, we can effectively quantify the disparities between regular frames and reference frames. This enables us to accurately detect dynamic areas within the regular frames. Furthermore, by refining the dynamic areas of keyframes using historical observation data, the robustness of the system is further enhanced. We conducted comprehensive experimental evaluations on challenging dynamic sequences from the TUM dataset and compared our approach with state-of-the-art dynamic VSLAM systems. The experimental results demonstrate that our method significantly enhances the accuracy and robustness of pose estimation. Additionally, we validated the effectiveness of the system in dynamic environments through real-world scenario tests.

We present a simultaneous localization and mapping algorithm for a mobile robot in 3D. It allows to construct a map of the environment without artifacts caused by dynamic objects such as trees or people. Experimental results confirm the effectiveness of the algorithm in real world scenarios. We also compare the developed algorithm with previously realized analogs.

In unsupervised domain adaptation (UDA), a classifier for a target domain is trained with labeled source data and unlabeled target data. Existing UDA methods assume that the source data come from the same source domain (i.e., single-source scenario) or from multiple source domains, whose feature spaces have the same dimension ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">homogeneous</i> ) but different distributions (i.e., multihomogeneous-source scenario). However, in the real world, for a specific target domain, we probably have multiple different-dimension ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">heterogeneous</i> ) source domains, which do not satisfy the assumption of existing UDA methods. To remove this assumption and move forward to a realistic UDA problem, this article presents a shared-fuzzy-equivalence-relation neural network (SFERNN) for addressing the multisource heterogeneous UDA problem. The SFERNN is a five-layer neural network containing <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> source branches and one target branch. The network structure of the SFERNN is first confirmed by a novel fuzzy relation called multisource shared fuzzy equivalence relation. Then, we optimize parameters of the SFERNN via minimizing cross-entropy loss on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c$</tex-math></inline-formula> source branches and the distributional discrepancy between each source branch and the target branch. Experiments distributed across eight real-world datasets are conducted to validate the SFERNN. This testing regime demonstrates that the SFERNN outperforms the existing single-source heterogeneous UDA methods, especially when the target domain contains few data.

This paper reviews the latest research progress of visual SLAM in dynamic scenes. The improved algorithms dependent on deep learning and multi-sensor fusion are discussed. First of all, the challenges of traditional visual SLAM in dynamic environments are stated, especially the interference of high dynamic objects on system feature extraction, pose estimation and mapping. Then, the application of several object detection technologies such as YOLOv5 and YOLOv7 in dynamic visual SLAM was analyzed in detail. These methods significantly improve the positioning precision and robustness of the system by identifying and eliminating dynamic feature points in the scene. In addition, this paper explores the visual SLAM system combining semantic segmentation and geometric constraint optimization, and finally introduces the multi-sensor SLAM method combining IMU data, point cloud data and visual information. Experimental results show that these improved algorithms have a significant improvement in positioning accuracy and system robustness compared with traditional visual SLAM algorithms under TUM and other dynamic data sets.

This paper proposes a visual-inertial navigation system (VINS) combined with a dynamic object detection (DOD) algorithm, to improve the localization and state estimate accuracy of autonomous driving vehicles (ADV) in dynamic environments. Firstly, based on the YOLOv5 network, we train the proposed DOD model to detect dynamic objects in the road environment. Secondly, by removing the feature points in the dynamic object regions, we track the remaining feature points to eliminate the influence of the dynamic object. Furthermore, we model the global positioning system (GPS) measurement as a general factor and introduce its residual factor into the cost function to eliminate the cumulative error. Finally, we validate the performance of the proposed method on public datasets and real-world experiments. The results show that the proposed method can effectively eliminate the influence of the dynamic object and eliminate the cumulative error. It provides theoretical guidance for ADV navigation in dynamic or large-scale outdoor environments.

Existing visual SLAM (vSLAM) systems fail to perform well in dynamic environments as they cannot effectively ignore moving objects during pose estimation and mapping. We propose a lightweight approach to improve the robustness of existing feature based RGB-D and stereo vSLAM by accurately removing dynamic outliers in the scene that contribute to failures in pose estimation and mapping. First, a novel motion state detection algorithm using the depth and feature flow information is presented to identify regions in the scene with high moving probability. This information is then fused with semantic cues via a probability framework to enable accurate and robust moving object extraction to retain the useful features for pose estimation and mapping. To reduce the computational complexity of extracting semantic information in every frame, we propose to extract semantics only on keyframes with significant changes in image content. Semantic propagation is used to compensate for the changes in the intermediate frames (i.e., non-keyframes). This is achieved by computing the dense transformation map using the available feature flow vectors. The proposed techniques can be integrated into existing vSLAM systems to increase their robustness in dynamic environments without incurring much computation cost. Our work highlights the importance of distinguishing between motion states of potential moving objects for vSLAM in highly dynamic environments. We provide extensive experimental results on four well-known RGB-D and stereo datasets to show that the proposed technique outperforms existing vSLAM methods in indoor and outdoor environments under various dynamic scenarios including crowded scenes. We also perform our experiments on a low-cost embedded platform, i.e., Jetson TX1, to demonstrate the computational efficiency of our method.

Lidar has been widely used in military and civilian applications. This passage mainly introduces a type of lidar used in autonomous vehicle: TOF lidar. The working principle of TOF lidar was introduced in detail. Then based on the TOF lidar, there are two types of lidar to be introduced: scanning lidar and non-scanning lidar. Scanning lidar includes single-line scanning lidar and multi-line scanning lidar. 3D-flash lidar was taken as example to introduce non-scanning lidar. The contents of each type of lidar are system structure, working principle, the development around the world and some problems existing currently. Introduction of lidar development direction is made, including MEMS technology and ZOL technology.

Visual Simultaneous Localization and Mapping (VSLAM) has developed as the basic ability of robots in past few decades. There are a lot of open-sourced and impressive SLAM systems. However, the majority of the theories and approaches of SLAM systems at present are based on the static scene assumption, which is usually not practical in reality because moving objects are ubiquitous and inevitable under most circumstances. In this paper the DDL-SLAM (Dynamic Deep Learning SLAM) is proposed, a robust RGB-D SLAM system for dynamic scenarios that, based on ORB-SLAM2, adds the abilities of dynamic object segmentation and background inpainting. We are able to detect moving objects utilizing both semantic segmentation and multi-view geometry. Having a static scene map allows inpainting background of the frame which has been obscured by moving objects, therefore the localization accuracy is greatly improved in the dynamic environment. Experiment with a public RGB-D benchmark dataset, the results clarify that DDL-SLAM can significantly enhance the robustness and stability of the RGB-D SLAM system in the highly-dynamic environment.

Simultaneous Localization and Mapping (SLAM) is a crucial technology for intelligent mobile robots to operate successfully in unknown environments. While many excellent SLAM systems have been developed in recent years, most assume that the environment is static, resulting in poor performance in dynamic environments. To address this limitation, we propose multiview stereo (MVS)‐SLAM (MVS‐SLAM), a real‐time semantic RGBD SLAM system with improved‐multiview geometry, built on the RGBD mode of ORB‐SLAM3. MVS‐SLAM tightly integrates semantic and geometric information to tackle the challenges posed by dynamic scenes. To meet the real‐time requirements, the semantic module leverages the latest and fastest object detection network, YOLOv7, to provide semantic prior knowledge for the geometric module. We introduce a novel geometric constraint method that capitalizes on depth images and semantic information to recover three‐dimensional (3D) feature points and initial camera pose. We use a 3D coordinate error threshold to identify dynamic points and remove them using the K‐means clustering algorithm. This approach effectively reduces the impact of dynamic points. We validate MVS‐SLAM using challenging dynamic sequences from the TUM data set, demonstrating that it significantly improves localization accuracy and system robustness in all types of dynamic environments.

Visual simultaneous localization and mapping (SLAM) in dynamic environments has received significant attention in recent years, and accurate segmentation of real dynamic objects is the key to enhancing the accuracy of pose estimation in such environments. In this study, we propose a visual SLAM approach based on ORB-SLAM3, namely dynamic multiple object tracking SLAM (DMOT-SLAM), which can accurately estimate the camera’s pose in dynamic environments while tracking the trajectories of moving objects. We introduce a spatial point correlation constraint and combine it with instance segmentation and epipolar constraint to identify dynamic objects. We integrate the proposed motion check method into DeepSort, an object tracking algorithm, to facilitate inter-frame tracking of dynamic objects. This integration not only enhances the stability of dynamic features detection but also enables the estimation of global motion trajectories for dynamic objects and the construction of object-level semi-dense semantic maps. We evaluate our approach on the public TUM, Bonn, and KITTI dataset, and the results show that our approach has a significant improvement over ORB-SLAM3 in dynamic scenes and performs better compared to other state-of-the-art SLAM approaches. Moreover, experiments in real-world scenarios further substantiate the effectiveness of our approach.

Developing an autonomous indoor mobile robot navigation system from scratch is very difficult and it takes a long time. It requires a series of complex tasks such as handling given sensor inputs, calculating all the information needed for autonomous navigation, and controlling actuators required for movement. In this paper, an autonomous navigation system for indoor mobile robots is introduced mainly based on open source provided by the robot operating system. The presented system is capable of autonomously navigating an unstructured indoor environment avoiding collision with static or dynamic objects. To this end, the system consists of three main modules: mapping, localization, and planning. The mapping module builds a global map for an unknown environment by means of a simultaneous localization and mapping algorithm based on laser scanner data. The localization module estimates the mobile robot’s pose within the prebuilt map by way of an adaptive Monte Carlo localization approach. The planning module builds a local cost map for collision avoidance, generates collision-free trajectories to reach a goal pose based on the cost map, and produces control commands to follow the trajectories. The presented system has been tested not only in simulation environments built in the Gazebo simulator but also in real environments utilizing the Jackal mobile robot, to validate its performance for autonomous navigation including collision avoidance.

Autonomous navigation in dynamic environments where people move unpredictably is an essential task for service robots in real-world populated scenarios. Recent works in reinforcement learning (RL) have been applied to autonomous vehicle driving and to navigation around pedestrians. In this paper, we present a novel planner (reinforcement learning dynamic object velocity space, RL-DOVS) based on an RL technique for dynamic environments. The method explicitly considers the robot kinodynamic constraints for selecting the actions in every control period. The main contribution of our work is to use an environment model where the dynamism is represented in the robocentric velocity space as input to the learning system. The use of this dynamic information speeds the training process with respect to other techniques that learn directly either from raw sensors (vision, lidar) or from basic information about obstacle location and kinematics. We propose two approaches using RL and dynamic obstacle velocity (DOVS), RL-DOVS-A, which automatically learns the actions having the maximum utility, and RL-DOVS-D, in which the actions are selected by a human driver. Simulation results and evaluation are presented using different numbers of active agents and static and moving passive agents with random motion directions and velocities in many different scenarios. The performance of the technique is compared with other state-of-the-art techniques for solving navigation problems in environments such as ours.

e19260 Background: A recent study (Foster, Tromanhauser et al. 2019) utilizing Electronic Health Record (EHR) data found that patients with MM rarely have both serum protein electrophoresis (SPEP) and 24-hour urine protein electrophoresis (UPEP) documented in routine clinical practice settings. These elements are necessary to classify response using the International Myeloma Working Group (IMWG) consensus criteria. It could be hypothesized that adding the additional requirements for bone marrow biopsy necessary to confirm a complete response would further reduce the rate of strict adherence to the IMWG criteria in the real world (RW). Clinicians are however assessing response and progression in RW settings and often report these assessments in published literature. In this study, we survey the literature to determine how RW response is being captured and reported outside of the clinical trial setting. Methods: A systematic search was performed using Medline 2010-2019. Using a standardized grading system, English language articles were evaluated that utilized EHR data from routine clinical practice to report RW response in patients with active MM. Registry based and/or pragmatic observational studies were excluded as many had mandatory reporting procedures in place. Studies were then categorized based on methods of assessing response, progression and use of IMWG criteria. Results: The majority of studies grade 1 studies identified (21/25) utilized best response achieved based on treating physician-documented assessment and either did not specify the use of IMWG(10/21) criteria or explicitly stated they did not use conventional criteria (11/21). Progression event capture and reporting varied greatly with many using physician-documented progression, change in regimen, addition of additional agents to existing regimen, stem cell transplant, or an individually created algorithm. Conclusions: Our findings suggest that there is a need for further research on objective techniques for the assessment of RW progression and response in patients with MM. Additionally, validation of an approach that utilizes partial adherence to IMWG criteria would help in the ability to compare findings across clinical trial and RW settings. This validation work is under way, and preliminary results will be reported at the meeting.