Development of a real-time 3D camera based on micro-electromechanical systems mirrors

This paper presents an offline ground truth generation method using LIDAR(Light Detection and Ranging) scans and odometry. The generated ground truth allows quantitative evaluation of the performance of self-localization methods in urban areas where GNSS(Global Navigation Satellite System) cannot be trusted. The proposed method determines the vehicle pose (position and orientation) by aligning the LIDAR input with previously collected point cloud data. However, as alignment convergences are affected by the environment around the vehicle during each LIDAR scan, it can be erroneous. Incorrect estimates are removed and poses are interpolated by relying on odometry; which is locally accurate. A step by step optimization approach is adopted to yield the most accurate result. Experiments performed in a typical urban environment, with many buildings and surrounding obstacles, demonstrated the effectiveness of the proposed method.

3D LIDARs (Light Detection and Ranging) with 1.5μm nanosecond pulse lasers have been increasingly used in different applications. The main reason for their popularity is that these LIDARs have high performance while at the same time can be made eye-safe. Because the laser hazard effect on eyes or skin at this wavelength region (<1.4μm) is mainly from the thermal effect accumulated from many individual pulses over a period of seconds, scanning can effectively reduce the laser beam hazard effect from the LIDARs. Neptec LIDARs have been used in docking to the International Space Station, military helicopter landing and industrial mining applications. We have incorporated the laser safety requirements in the LIDAR design and conducted laser safety analysis for different operational scenarios. While 1.5μm is normally said to be the eye-safe wavelength, in reality a high performance 3D LIDAR needs high pulse energy, small beam size and high pulse repetition frequency (PRF) to achieve long range, high resolution and high density images. The resulting radiant exposure of its stationary beam could be many times higher than the limit for a Class 1 laser device. Without carefully choosing laser and scanning parameters, including field-of-view, scan speed and pattern, a scanning LIDAR can’t be eye- or skin-safe based only on its wavelength. This paper discusses the laser safety considerations in the design of eye-safe scanning LIDARs, including laser pulse energy, PRF, beam size and scanning parameters in two basic designs of scanning mechanisms, i.e. galvanometer based scanner and Risley prism based scanner. The laser safety is discussed in terms of device classification, nominal ocular hazard distance (NOHD) and safety glasses optical density (OD).

Routine bridge inspections require labor intensive and highly subjective visual interpretation to determine bridge deck surface condition. Light Detection and Ranging (LiDAR) a relatively new class of survey instrument has become a popular and increasingly used technology for providing as-built and inventory data in civil applications. While an increasing number of private and governmental agencies possess terrestrial and mobile LiDAR systems, an understanding of the technology’s capabilities and potential applications continues to evolve. LiDAR is a line-of-sight instrument and as such, care must be taken when establishing scan locations and resolution to allow the capture of data at an adequate resolution for defining features that contribute to the analysis of bridge deck surface condition. Information such as the location, area, and volume of spalling on deck surfaces, undersides, and support columns can be derived from properly collected LiDAR point clouds. The LiDAR point clouds contain information that can provide quantitative surface condition information, resulting in more accurate structural health monitoring. LiDAR scans were collected at three study bridges, each of which displayed a varying degree of degradation. A variety of commercially available analysis tools and an independently developed algorithm written in ArcGIS Python (ArcPy) were used to locate and quantify surface defects such as location, volume, and area of spalls. The results were visual and numerically displayed in a user-friendly web-based decision support tool integrating prior bridge condition metrics for comparison. LiDAR data processing procedures along with strengths and limitations of point clouds for defining features useful for assessing bridge deck condition are discussed. Point cloud density and incidence angle are two attributes that must be managed carefully to ensure data collected are of high quality and useful for bridge condition evaluation. When collected properly to ensure effective evaluation of bridge surface condition, LiDAR data can be analyzed to provide a useful data set from which to derive bridge deck condition information.

The emergence of smart headlights with reconfigurable light distributions that provide optimal illumination, highlight road objects, and project symbols to communicate with traffic participants further enhances road safety. Integrating all these functions in a single headlight usually suffers from issues of bulky multi-functional add-on modules with high cost or the use of conventional spatial light modulators with low optical efficiency and complex thermal design requirements. This paper presents a novel laser headlight prototype based on biaxially resonant microelectromechanical systems (MEMS) mirror light modulator for mapping blue laser patterns on phosphor plate to create structured white illumination and tunable road projection. The proposed headlight prototype system enables reconfigurable light distribution by leveraging laser beam scanning with fewer back-end lens and simple thermal design requirements. Built with thin-film lead zirconate titanate oxide (PbZrTiO3) actuators, the MEMS mirror achieved high-frequency biaxial resonance of 17.328 kHz, 4.81 kHz, and optical scan angle of 12.9°. The large mirror design of 2.0 mm facilitates more refined resolvable projection pixels, delivers more optical power, and provides moderate optical aperture to possibly serve as the common spatial light modulator of headlight and the light detection and ranging (LiDAR) towards all-in-one integration. The carefully designed bi-axial resonant frequency improves the device’s robustness by offsetting the lowest eigenmode away from the vehicle vibration. By establishing the laser headlight prototype systems of both 1D and 2D scanning modes, a mathematical model of laser modulation and MEMS electrical control principles of Lissajous scanning are proposed to tune the projection pattern density and shapes. It laid the foundation for developing a laser scanning control system with more complex project functions and prompting the application of MEMS for compact headlight system that addresses night driving visibility, eliminates glare effect, and renders interactive projection capabilities.

Light Detection and Ranging (LIDAR) scanning can be used to safely and remotely provide intelligence on the interior of dangerous structures for use by first responders that need to enter these structures. By scanning into structures through windows and other openings or moving the LIDAR scanning into the structure, in both cases carried by a remote controlled robotic crawler, the presence of dangerous items or personnel can be confirmed or denied. Entry and egress pathways can be determined in advance, and potential hiding/ambush locations identified. This paper describes an integrated system of a robotic crawler and LIDAR scanner. Both the scanner and the robot are wirelessly remote controlled from a single laptop computer. This includes navigation of the crawler with real-time video, self-leveling of the LIDAR platform, and the ability to raise the scanner up to heights of 2.5 m. Multiple scans can be taken from different angles to fill in detail and provide more complete coverage. These scans can quickly be registered to each other using user defi ned 'pick points', creating a single point cloud from multiple scans. Software has been developed to deconstruct the point clouds, and identify specific objects in the interior of the structure from the point cloud. Software has been developed to interactively visualize and walk through the modeled structures. Floor plans are automatically generated and a data export facility has been developed. Tests have been conducted on multiple structures, simulating many of the contingencies that a first responder would face. Keywords Hazards, laser scanning, LIDAR, model, segmentation Language: en

&lt;span lang="EN-US"&gt;In recent years, there has been a strong demand for self-driving cars. For safe navigation, self-driving cars need both precise localization and robust mapping. While global navigation satellite system (GNSS) can be used to locate vehicles, it has some limitations, such as satellite signal absence (tunnels and caves), which restrict its use in urban scenarios. Simultaneous localization and mapping (SLAM) are an excellent solution for identifying a vehicle’s position while at the same time constructing a representation of the environment. SLAM-based visual and light detection and ranging (LIDAR) refer to using cameras and LIDAR as source of external information. This paper presents an implementation of SLAM algorithm for building a map of environment and obtaining car’s trajectory using LIDAR scans. A detailed overview of current visual and LIDAR SLAM approaches has also been provided and discussed. Simulation results referred to LIDAR scans indicate that SLAM is convenient and helpful in localization and mapping.&lt;/span&gt;

Simultaneous localization and mapping (SLAM) has been investigated in the field of robotics for two decades, as it is considered to be an effective method for solving the positioning and mapping problem in a single framework. In the SLAM community, the Extended Kalman Filter (EKF) based SLAM and particle filter SLAM are the most mature technologies. After years of development, graph-based SLAM is becoming the most promising technology and a lot of progress has been made recently with respect to accuracy and efficiency. No matter which SLAM method is used, loop closure is a vital part for overcoming the accumulated errors. However, in 2D Light Detection and Ranging (LiDAR) SLAM, on one hand, it is relatively difficult to extract distinctive features in LiDAR scans for loop closure detection, as 2D LiDAR scans encode much less information than images; on the other hand, there is also some special mapping scenery, where no loop closure exists. Thereby, in this paper, instead of loop closure detection, we first propose the method to introduce extra control network constraint (CNC) to the back-end optimization of graph-based SLAM, by aligning the LiDAR scan center with the control vertex of the presurveyed control network to optimize all the poses of scans and submaps. Field tests were carried out in a typical urban Global Navigation Satellite System (GNSS) weak outdoor area. The results prove that the position Root Mean Square (RMS) error of the selected key points is 0.3614 m, evaluated with a reference map produced by Terrestrial Laser Scanner (TLS). Mapping accuracy is significantly improved, compared to the mapping RMS of 1.6462 m without control network constraint. Adding distance constraints of the control network to the back-end optimization is an effective and practical method to solve the drift accumulation of LiDAR front-end scan matching.

The work conducted in this thesis is a part of an EU Horizon 2020 research initiative project known as DigiArt. This part of the DigiArt project presents and explores the design, formulation and implementation of probabilistically orientated state estimation algorithms with focus towards unmanned system positioning and three-dimensional (3D) mapping. State estimation algorithms are considered an influential aspect of any dynamic system with autonomous capabilities. Possessing the ability to predictively estimate future conditions enables effective decision making and anticipating any possible changes in the environment. Initial experimental procedures utilised a wireless ultra-wide band (UWB) based communication network. This system functioned through statically situated beacon nodes used to localise a dynamically operating node. The simultaneous deployment of this UWB network, an unmanned system and a Robotic Total Station (RTS) with active and remote tracking features enabled the characterisation of the range measurement errors associated with the UWB network. These range error metrics were then integrated into an Range based Extended Kalman Filter (R-EKF) state estimation algorithm with active outlier identification to outperform the native approach used by the UWB system for two-dimensional (2D) pose estimation.The study was then expanded to focus on state estimation in 3D, where a Six Degreeof-Freedom EKF (6DOF-EKF) was designed using Light Detection and Ranging (LiDAR) as its primary observation source. A two step method was proposed which extracted information between consecutive LiDAR scans. Firstly, motion estimation concerning Cartesian states x, y and the unmanned system’s heading (ψ) was achieved through a 2D feature matching process. Secondly, the extraction and alignment of ground planes from the LiDAR scan enabled motion extraction for Cartesian position z and attitude angles roll (θ) and pitch (φ). Results showed that the ground plane alignment failed when two scans were at 10.5◦ offset. Therefore, to overcome this limitation an Error State Kalman Filter (ES-KF) was formulated and deployed as a sub-system within the 6DOF-EKF. This enabled the successful tracking of roll, pitch and the calculation of z. The 6DOF-EKF was seen to outperform the R-EKF and the native UWB approach, as it was much more stable, produced less noise in its position estimations and provided 3D pose estimation.

The demand for a reliable and accurate navigation system that can replace Global Positioning System (GPS) in GPS-denied environment has become increasingly imperative. For indoor environment where GPS is almost unavailable or unreliable, the utilization of other sensors such as inertial sensors becomes necessary. However, inertial sensors alone cannot sustain reliable long-term accuracy due to errors accumulation without external periodic corrections. Thus this paper proposes the utilization of Light Detection and Ranging (LiDAR) as an alternative system to provide periodic corrections. In this paper, a tightly-coupled integrated navigation system that integrates LiDAR, a single-axis gyroscope and wheel encoder is introduced. Straight lines detection and extraction algorithm is utilized to estimate the changes in orientation and range from LiDAR to the extracted line. LiDAR-estimated orientation change and range change to the extracted line feature between two consecutive LiDAR scans are first filtered out through a high rate extended Kalman Filter (EKF) to remove the effect of short-term noise associated with LiDAR scans. Then the smoothed orientation and range changes are fused by a low rate EKF with those predicted by gyroscope and wheel encoder. The proposed system is verified through real experiment on a wirelessly controlled Unmanned Ground Vehicle (UGV). Experimental results indicate that navigation accuracy has been improved to sub-meter and gyroscope bias is precisely estimated.

The two main categories of light detection and ranging (LIDAR) for mapping and monitoring landslides are static and mobile devices. Static devices are commonly named ground-based LIDAR or terrestrial laser scanners (TLSs). Regarding mobile devices, the different instruments are normally classified depending on the platform configuration: either boats, cars or flying devices. There are three main types of LIDAR, corresponding to three different ways of measuring the range, for instance, the distance along the line of sight (LOS) between the sensor and the terrain. The types include: pulse LIDAR measures the time of flight (TOF) of a laser pulse; phase LIDAR uses the phase shift between the emitted and received signals; and triangulation LIDAR uses a camera to locate the laser spot on the scanned surface. Georeferencing is the process of coordinate transformation from an internal referential system relative to the LIDAR device into a real-world system of reference.

The need for imaging and detecting in various instrument areas has brought light detection and ranging (LiDAR) to the forefront of consumer technology. Among different LiDAR, microelectromechanical system (MEMS) LiDAR has more appeal due to its small size and high integration. However, due to its low resolution and slight scanning angle, MEMS LiDAR does not apply to all scenes. Thus we presented a 360-deg scanning LiDAR emission optical system. Design formulas were deduced from theoretical formulas. In this system, the aspheric lens collimated the beam, MEMS micromirrors tracked the concentric circular beams, and the converging lens compensated for the divergence of the outgoing beam. By doing so, the target was scanned 360 deg at high resolution. With Zemax, the fast-axis divergence angle was 0.2484 mrad, the slow-axis divergence angle was 0.1546 mrad, and the system energy utilization rate was 84.16% with an angular resolution of 0.0142 deg at a distance of 10,000 mm. We have provided a potential solution for improving the scanning angle and resolution of MEMS LiDAR.

Water leakage mapping in concrete railway tunnels using LiDAR generated point clouds

Highly automated or autonomous vehicles will be dependent on systems that have to perceive the environment to make valid decisions during the driving cycle. One of the key enablers for autonomous and highly automated vehicles will be Light Detection And Ranging (LiDAR) technology. A Micro-Electro-Mechanical System (MEMS) based Micro-Scanning LiDAR is able to detect obstacles in a predefined Field-of-View (FoV). The point cloud stability of the scanned FoV is mandatory to be able to make a valid point where the obstacle is located in the scenery. Due to the fact that massive shocks can occur arbitrarily to the system, it is necessary to be able to detect and correct them as fast as possible that point cloud stability can be recovered as fast as possible. In this paper, we introduce a novel system architecture that enables a fast shock detection and correction of phase and frequency for MEMS-based Micro-Scanning LiDAR Systems. Our novel introduced fail-operational detection and correction system architecture was implemented in a 1D MEMS-based Micro-Scanning LiDAR FPGA platform to prove its feasibility and for performance evaluation.

Obtaining the three-dimensional structure of tree orchards from remote 2D terrestrial LIDAR scanning

Atmospheric aerosol is one of the major factors that cause environmental pollution. Light detection and ranging (LiDAR) is an effective remote sensing tool for aerosol observation. In order to provide a comprehensive understanding of the aerosol pollution from the physical perspective, this study investigated regional atmospheric aerosol pollution through the integration of measurements, including LiDAR, satellite, and ground station observations and combined the backward trajectory tracking model. First, the horizontal distribution of atmospheric aerosol wa obtained by a whole-day working scanning micro-pulse LiDAR placed on a residential building roof. Another micro-pulse LiDAR was arranged at a distance from the scanning LiDAR to provide the vertical distribution information of aerosol. A new method combining the slope and Fernald methods was then proposed for the retrieval of the horizontal aerosol extinction coefficient. Finally, whole-day data, including the LiDAR data, the satellite remote sensing data, meteorological data, and backward trajectory tracking model, were selected to reveal the vertical and horizontal distribution characteristics of aerosol pollution and to provide some evidence of the potential pollution sources in the regional area. Results showed that the aerosol pollutants in the district on this specific day were mainly produced locally and distributed below 2.0 km. Six areas with high aerosol concentration were detected in the scanning area, showing that the aerosol pollution was mainly obtained from local life, transportation, and industrial activities. Correlation analysis with the particulate matter data of the ground air quality national control station verified the accuracy of the LiDAR detection results and revealed the effectiveness of LiDAR detection of atmospheric aerosol pollution.