LiMoSeg: Real-time Bird’s Eye View based LiDAR Motion Segmentation

Voxel is one of the common structural representation of 3D point cloud. Due to the sparsity of point cloud generated by light detection and ranging (LiDAR), there is the extreme imbalance in the foreground and background voxels. It decreases the accuracy of 3D object detection, has the negative effect on intelligent driving safety. To overcome this problem, we present a saliency prediction based 3D object detector SP-Det in this article. Although foreground voxels have the sufficient feature of object, it is difficult to localize the foreground region from voxel space with the larger background region. We design an auxiliary learning task, saliency prediction (SP). It benefits 3D detector in identifying the foreground region. SP task uses label diffusion to alleviate the label imbalance. It reduces the learning difficulty of saliency in voxel and bird's eye view (BEV) spaces. After that, to strengthen feature interaction from the sparse foreground region, we design saliency fusion (SF) module to fuse the learning result in SP task. It utilizes voxel and BEV saliency maps as progressive attention to resist the redundant feature from background region. To aggregate more foreground feature inside 3D and BEV region of interest (RoI), we design hybrid grid maps based RoI pooling (Hybrid-RoI pooling). Experiments are conducted in STF dataset. The adverse weather enlarges the sparsity of LiDAR point cloud, increasing the difficulty of object detection. SP-Det identifies and leverages foreground region, and achieves the performance better than the current methods. Hence, we believe that SP-Det benefits to LiDAR based 3D scene understanding in the adverse weather.

This paper describes new vision-based registration methods utilizing not only cameras on a user's head-mounted display but also a bird's-eye view camera that observes the user from an objective viewpoint. Two new methods, the line constraint method (LCM) and global error minimization method (GEM), are proposed. The former method reduces the number of unknown parameters concerning the user's viewpoint by restricting it to be on the line of sight from the bird's-eye view. The other method minimizes the sum of errors, which is the sum of the distance between the fiducials on the view and the calculated positions of them based on the current viewing parameters, for both the user's view and the bird's-eye view. The methods proposed here reduce the number of points that should be observed from the user's viewpoint for registration, thus improving the stability. In addition to theoretical discussions, this paper demonstrates the effectiveness of our methods by experiments in comparison with methods that use only a user's view camera or a bird's-eye view camera.

Autonomous navigation in dynamic environments presents a significant challenge for mobile robotic systems. This paper proposes a novel approach utilizing Convolutional Neural Networks (CNNs) for multi-object detection in 3D space and 2D segmentation using bird’s eye view (BEV) maps derived from 3D Light Detection and Ranging (LiDAR) data. Our method aims to enable mobile robots to localize movable objects and their occupancy, which is crucial for safe and efficient navigation. To address the scarcity of labeled real-world datasets, a synthetic dataset based on a simulation environment is generated to train and evaluate our model. Additionally, we employ a subset of the NVIDIA r2b dataset for evaluation in the real world. Furthermore, we integrate our CNN-based detection and segmentation model into a Robot Operating System 2 (ROS2) framework, facilitating communication between mobile robots and a centralized node for data aggregation and map creation. Our experimental results demonstrate promising performance, showcasing the potential applicability of our approach in future assembly systems. While further validation with real-world data is warranted, our work contributes to advancing perception systems by proposing a solution for multi-source, multi-object tracking and mapping.

The Medieval Church of St. Andrews. Edited by David McRoberts. Pp. xii + 178 + 24 illustrations + plan and a facsimile of the 16th-century ‘bird's eye view’ of St. Andrews. Glasgow: Burns, 1976. £7. - Volume 29 Issue 3

Our research objective is construction of 3D map and development of its practical applications. As an application, the authors had proposed 3D control interface for an operator who controls remote control robot. It is well know that an operator can control a robot easily using bird's-eye view. Using 3D control interface, an operator can control the robot at bird's-eye view. The bird's-eye view is constructed from 3D map and robot model. The operator can recognize relationship between the robot and the environment from 3D map and robot model following real robot motion. In this paper, we will describe 3D control interface and its implementation for HELIOS carrier. We will show an experimental result of 3D control interface. From result, we confirmed the effectiveness of the proposed method.

Robotics combines a lot of different domains with sophisticated challenges such as computer vision, motion control and search algorithms. Search algorithms can be applied to calculate movements. The A∗ algorithm is a well-known and proved search algorithm to find a path within a graph. This paper presents an extended A∗ algorithm that is optimized for robot navigation using a bird's eye view as a map that is dynamically generated by image stitching. The scenario is a robot that moves to a target in an environment containing obstacles. The robot is controlled by a Xilinx Zynq platform that contains an ARM processor and an FPGA. In order to exploit the flexibility of such an architecture, the FPGA is used to execute the most compute-intensive task of the extended A∗ algorithm. This task is responsible for sorting the accessible nodes in the graph. Several environments with different complexity levels are used to evaluate the extended A∗ algorithm. The environment is captured by a Kinect sensor located directly on the robot. In order to dewarp the robot's view, the frames are transformed to a bird's eye view. In addition, a wider viewing range is achieved by image stitching. The evaluation of the extended A∗ algorithm shows a significant improvement in terms of memory utilization. Accordingly, this algorithm is especially practicable for embedded systems since they have often only limited memory resources. Moreover, the overall execution time for several use cases is reduced up to a speed-up of 2.88x.

Bird's Eye View (BEV) representation learning has greatly improved target detection and tracking in complex environments and autonomous systems, thanks to a multi-view aggregation strategy that enhances spatial awareness, reduces occlusions, and ensures continuous surveillance of targets. This paper presents an Enhanced Global Context Fusion Network for multi-target multi-camera tracking (MTMC), addressing the critical challenge of spatial association across views in large-scale surveillance systems, unlike conventional approaches that directly aggregate Bird's Eye View (BEV) features, our method hierarchically refines individual BEV embedding by integrating global context, generating bootstrap features that better preserve spatial relationships and reduce information loss. In order to improve the problem of poor robustness to real-world issues that may be caused by camera malfunctions, we introduce a data augmentation strategy that replaces the input of lost camera signals with black images during training. The results of extensive experiments on the Wildtrack dataset demonstrate the state-of-the-art performance of our approach, reaching 92.0% for MODA and 94.1% for IDF1, outperforming existing techniques with only a negligible increase in computational cost. These results validate the effectiveness of our approach in terms of improved BEV feature representation and system stability, providing a reliable solution for reliable MTMC detection and tracking in complex environments.

[This corrects the article DOI: 10.1098/rsos.160426.].

By matching roadway images in encountered scene with bird's eye view, the scope of humans' perception is expanded to satellite-roadway-vehicle network. Based on the geometric consistency of the satellite images with as-is local terrain, in this paper, a computa- tional mechanism is introduced for generation and adaptation of a roadway model transferable through the network. First, the chromatic complexity of the roadway area is represented as a palette of saliency colors via fractal sampling of the scene image. Next, the palette is adapted to the associated area in the bird's eye view. Finally, the palette is transferred to the bird's eye view for anticipatively following the roadway pattern. Experimental results demonstrate that the transferable model can be applied to the extension of the roadway pattern prior to physical access.

This work offers a method for the visual-based navigation of a mobile robot in indoor environments, using a single omni-directional camera. The geometry of the catadioptric sensor and the method used to obtain a bird's eye (orthographic) view of the ground plane are presented. This picture evocatively abridges the solution to steering problems, by eliminating any perspective effects. The nature of each navigation task is taken hooked on account when conniving the compulsory navigation skills and environmental representations. Topological Navigation is cast-off for itinerant long distances plus does not require knowledge of the exact location of the robot but rather, a qualitative position happening the topological map. For indoor robots in structured environments, we consume owed separately with the cases of geometrical and topological models of interplanetary. For shapeless environments, we consume discussed the cases of navigation through optical activities, using methods from the appearance-based paradigm, and by recognition of specific substances in the environment. The navigation process cartels appearance based approaches and visual serving upon some environmental features. Graphic Path Succeeding is obligatory for native, actual precise navigation, for e.g. door traversal, docking. The robot is controlled to follow a pre-specified trail accurately, by tracking visual milestones in bird's eye views of the ground plane. By clearly separating the nature of these triangulation tasks, a simple besides yet powerful navigation scheme is obtained.

Proposes a method for the visual-based navigation of a mobile robot in indoor environments, using a single omnidirectional (catadioptric) camera. The geometry of the catadioptric sensor and the method used to obtain a bird's eye (orthographic) view of the ground plane are presented. This representation significantly simplifies the solution to navigation problems, by eliminating any perspective effects. The nature of each navigation task is taken into account when designing the required navigation skills and environmental representations. We propose two main navigation modalities: topological navigation and visual path following. Topological navigation is used for traveling long distances and does not require knowledge of the exact position of the robot but rather, a qualitative position on the topological map. The navigation process combines appearance based methods and visual servoing upon some environmental features. Visual path following is required for local, very precise navigation, e.g., door traversal, docking. The robot is controlled to follow a prespecified path accurately, by tracking visual landmarks in bird's eye views of the ground plane. By clearly separating the nature of these navigation tasks, a simple and yet powerful navigation system is obtained.

This paper reports an experiment, conducted with 116 undergraduates at an institute of Higher Education in the United Kingdom, to determine if preferences exist in the way selected landscape features and the water (hydrological) cycle are portrayed. The results show that most students can draw simple diagrams of physical landforms from descriptions, but there is some confusion between bird's eye and side views. A significant number of students draw bird's eye views of drainage basins which flow downwards and coastlines which trend from east to west. In sketches of the side views of cliffs, waterfalls, river profiles and landslips, a significant number of students place the feature on the left, facing towards the right. No significant differences exist in the orientation of side views of the water cycle. It is proposed that the preferred orientations are explained by research findings into stroke patterns, although other factors are also identified to explain the results.

Driving requires interacting with road agents and predicting their future behaviour in order to navigate safely. We present FIERY: a probabilistic future prediction model in bird's-eye view from monocular cameras. Our model predicts future instance segmentation and motion of dynamic agents that can be transformed into non-parametric future trajectories. Our approach combines the perception, sensor fusion and prediction components of a traditional autonomous driving stack by estimating bird's-eye-view prediction directly from surround RGB monocular camera inputs. FIERY learns to model the inherent stochastic nature of the future solely from camera driving data in an end-to-end manner, without relying on HD maps, and predicts multimodal future trajectories. We show that our model outperforms previous prediction baselines on the NuScenes and Lyft datasets. The code and trained models are available at https://github.com/wayveai/fiery.

Accurate object detection is important for the safe navigation of autonomous vehicles (AVs) in dynamic environments. Conventional AVs rely on high-resolution but bulky, topmounted 3D Light Detection And Ranging (LiDAR) sensors, which are impractical for scalable production and computationally expensive to process. To address this, we utilize quasi-2D LiDAR sensors, which offer a compact and scalable solution for AVs. However, point clouds generated by these sensors have a narrow vertical field of view and contain a limited number of points compared to 3D LiDARs, which makes object detection significantly more difficult. To overcome this challenge, we transform the quasi-2D point clouds into Bird's-Eye-View (BEV) images and apply four different input encoding strategies. These encodings enhance the spatial and temporal context of the data, making them suitable for efficient deep-learning-based 2D object detection models to perform real-time detection with minimal computational overhead. In contrast to other approaches for 2D LiDARs, we are able to detect not only vehicles but also pedestrians and cyclists. We use a large real-world dataset to compare our proposed input encodings combined with three different state-of-the-art 2D object detection models. Our results demonstrate that these input encodings enhance object detection performance across all models, significantly reducing false positives, improving heading angle prediction, and boosting the detection accuracy of smaller traffic participants.

Sampling-based search algorithms such as Rapidly-Exploring Random Trees (RRT) have been utilized for mobile robot path planning and motion planning in high dimensional continuous spaces. This paper presents a path planning method for a planetary exploration rover in rough terrain. The proposed method exploits the framework of a sampling-based search, the optimal RRT (RRT*) algorithm. The terrain geometry used for planning is composed of point cloud data close to continuous space captured by a light detection and ranging (LIDAR) sensor. During the path planning phase, the proposed RRT*algorithm directly samples a point (node) from the LIDAR point cloud data. The path planner then considers the rough terrain traversability of the rover during the tree expansion process of RRT*. This process improves conventional RRT*in that the generated path is safe and feasible for the rover in rough terrain. In this paper, simulation study on the proposed path planning algorithm in various real terrain data confirms its usefulness.