Efficient Camera–LiDAR Calibration Using Accumulated LiDAR Frames

Autonomous robotics emerged as a research and development field nearly forty years ago, but only fifteen years ago, after the DARPA (Defense Advanced Research Projects Agency of the United States Department of Defense) challenge, autonomous mobile systems started to be considered as a solution to the transportation and service problem. This chapter is focused on autonomous (i.e., robotic) vehicles used as human transportation service from two points of view: on one hand, the autonomous vehicle that leads to intelligent transportation systems; on the other hand, autonomous vehicles used for rehabilitation or for enhancing mobility capabilities of their users. Both perspectives of autonomous systems are linked by the use of rapid prototyping techniques, aimed at converting a previously commercial product into a robotic system with a specific transportation usage. This chapter shows, in particular, two cases: two electric commercial vehicles (one golf cart and one car) converted into an autonomous robot for transporting people in cities or for executing specific tasks in sites; and an assistive vehicle (an electric scooter) used by people with reduced mobility. The design of the different components needed to achieve such automation is shown in detail herein.

For applications in robotics and autonomous vehicles, fusing 3D LiDAR, RGB camera, and thermal camera can improve the perception in day and night environments. To fuse the sensors, accurate extrinsic calibration is important. However, most existing methods use offline calibration, which is tedious, as it requires the use of special targets and human intervention. Meanwhile, among the online calibration methods, little attention has been paid to LiDAR and thermal camera online calibration. Thus, in this paper, an online extrinsic calibration method is proposed to solve the problem, called RGBDTCalibNet. It can achieve the extrinsic calibration between a 3D LiDAR and an RGB camera, as well as a 3D LiDAR and a thermal camera. It leverages CNN-based deep learning methods for feature extraction and feature matching. The proposed network only takes a pair of images and point cloud as the input and outputs the extrinsic parameters in a single shot. It is trained and tested on two datasets: KITTI360 and our own dataset collected in the NTU campus, using two types of LiDAR: Velodyne HDL-64 and Livox Horizon. Both demonstrate the effectiveness, robustness and accuracy of RGBDTCalibNet. The code will be open-sourced at https://github.com/sanatmharolkar/RGBDTCalibNet.git

The integration of multiple cameras and 3D Li-DARs has become basic configuration of augmented reality devices, robotics, and autonomous vehicles. The calibration of multi-modal sensors is crucial for a system to properly function, but it remains tedious and impractical for mass production. Moreover, most devices require re-calibration after usage for certain period of time. In this paper, we propose a single-shot solution for calibrating extrinsic transformations among multiple cameras and 3D LiDARs. We establish a panoramic infrastructure, in which a camera or LiDAR can be robustly localized using data from single frame. Experiments are conducted on three devices with different camera-LiDAR configurations, showing that our approach achieved comparable calibration accuracy with the state-of-the-art approaches but with much greater efficiency.

Autonomous vehicle is the exponential growing topic in the research filed due its huge demand in the current industry. Where object detection plays the major role to take any major decision. Object detection in the normal vision yields the excellent performance due to deep learning models. But object detection in the night vision does not yield the good performance due to various challenges in it. In the proposed work created a night vision database called BDD-Darko where it contains night vision images with multi objects. Applied novel GHE-Ensemble model for image enhancement technique and trained using deep learning model Yolov5 for the multi-object detection in the night vision. Which resulted in 65.3 % accuracy. Proposed model is yielding better results for image enhancement technique for night vision than the previous existing model and detecting the multi object in the single frame with better performance. This paper proposes (1)A new database called as BDD-Darko which contains night vision image with multiple objects. (2) A Novel GHE-Ensemble model for image enhancement technique for night vision images

The performance of autonomous vehicle systems is currently limited by their inability to accurately analyze their surrounding environment. In order to function in a dynamic real world environment, an autonomous vehicle system must be capable of interpreting terrain based upon predetermined mission goals. This paper describes a concept of knowledge-based terrain analysis currently being developed to support the information needs of an autonomous helicopter system. The terrain analysis system consists of five integrated processing stages. Each process is discussed in detail and supported by a number of mission oriented examples. Int roduction One of the most unique system concepts being explored today is that of the development of totally autonomous land, sea, and air vehicles [1 - 3]. For military applications, autonomous vehicles provide a mechanism for removing humans from modern day battlefields while not impacting the tactical capabilities of the battle force. Civilian applications for such a device entail the investigation of hazardous areas (e.g. nuclear reactors and toxic waste dumps), search over large areas (e.g. downed aircraft or lost campers) and monitoring of large facilities (e.g. robot security police). As military funding for these systems is currently at an alltime high, a variety of autonomous system concepts are continually being presented in the technical literature. The primary limitation of an autonomous vehicle still lies in its inability to accurately analyze surrounding terrain information in order to generate tactical plans of action. This paper describes activities being performed in the Artificial Intelligence Branch of the Georgia Tech Research Institute on knowledge-based terrain analysis, particularly in support of the autonomous helicopter program.System OverviewThe knowledge-based tactical terrain analysis system consists of five separate processing phases: [1] image segmentation, [2] region classification, [3] visual model, [4] route planning, [5] threat location and coverage. Each processing phase is briefly discussed in the following subsections with actual image results given where appropriate.

This study presents the path-following control design for a group of autonomous underwater vehicles in a plane with obstacle and collision avoidance. A distributed backstepping controller is exploited for an autonomous vehicle to design a path-following controller. Then, the controller is applied for the path-following of a group of autonomous vehicle systems in the formation setting. The graph theory is used to model the interconnection between the vehicles. To avoid obstacles and collisions, a dynamic limit-cycle-based algorithm is proposed. It is assumed that each obstacle is involved an elliptical shape and if the collision or conflict of one vehicle to the surrounded environmental obstacles is detected, the obstacle avoidance algorithm will be active to prevent collisions. Finally, the introduced algorithm is implemented and simulated for a group of underwater autonomous vehicles in MATLAB software. The simulation results illustrate the effectiveness and performance of the proposed distributed controller apparently.

This special issue assembles five articles ensuing from a conference on “The Man and the Machine: When Systems Take Decisions Autonomously”, which took place on June 26 and 27, 2015, at the University of St. Gallen in Switzerland.The aim of the conference was to explore the broader implications of artificial intelligence, machine learning and autonomous robots and vehicles. Alphabet's Deep Mind is just one example about Whom we know, at least a little, and who, we are told, will be good. Autonomous vehicles are also about to enter the market and our phones have begun to verbalize at us. Private drones are being regulated by the US Federal Aviation Administration. The five papers in this special issue address some of the legal issues the broader development raises.The first article is on “The Implications of Modern Business-Entity Law for the Regulation of Autonomous Systems” and is written by Shawn Bayern.

Autonomous vehicles are one of the most attractive applications for light detection and ranging sensors, where they help with scene understanding. For this understanding, object detection is crucial, and it must be done in a frame by frame basis. This detection on a single frame is a challenging task due to the sparse and disordered nature of the information. This paper presents an alternative spherical representation for this data aiming to improve object detection. This proposal registers the light detection and ranging data in a 2-dimensions angle map using most of the 3-dimensions points in three layers, adding reflectivity information, and a logarithmic representation of distance. For evaluating this representation, we employed an object detector based on the algorithm: you only look once version 3, and we used a public reference dataset of 3-dimensional objects for training. This framework yielded a classification accuracy of 85.9% and 74.5% of intersection over union factor when estimating seven classes simultaneously. This approach presents an alternative for processing this data that helps to benefit the most from the light detection and ranging information with high accuracy, helping in the reduction of the associated risks of autonomous vehicles.

When underwater vehicles are performing operations in the water, an autonomous positioning system can provide them with their location information in a convert and flexible way. In Wang et al. Design of autonomous underwater vehicle positioning system, 2016 [1], an autonomous positioning system for underwater vehicles is designed. The buoys on the water surface obtain their location with the help of GPS or Beidou Positioning system, and transmit spread spectrum acoustic signal to the underwater vehicle. In this system, the buoy array configuration is a key factor regarding to positioning precision. In this paper, the influence exerted by buoy array configuration on positioning precision is analyzed; to design a better configuration, two principles of configuration design are proposed, which are nonsingularity of matrix G and minimal position dilution of precision (PDOP); and with the limit of maximum working distance of acoustic transducers, a central radiation configuration with minimum elevation angle is proposed. Through simulation, the PDOP values of three different configurations are compared upon the same region. It is concluded that the central radiation configuration has relatively smaller average PDOP value and no singular matrix G in the locating region, which is, therefore, a better configuration for practice.

Introduction:Corridor selection plays a critical role in determining the optimal locations for deploying autonomous vehicles. With the rapid advancement of self-driving technology, identifying suitable corridors becomes paramount to maximize the efficiency, safety, and overall performance of autonomous vehicle systems. The process involves analyzing various factors such as traffic patterns, infrastructure conditions, regulatory frameworks, and user preferences. This paper explores the significance of corridor selection and highlights the key considerations involved, shedding light on the crucial decision-making process for locating autonomous vehicles in the most favorable corridors. Research significance: : The research on corridor selection for locating autonomous vehicles holds significant importance in the field of transportation and autonomous systems. It addresses the need to identify the most suitable corridors for deploying self-driving vehicles, taking into account factors such as traffic flow, infrastructure conditions, and regulatory aspects. The outcomes of this research can inform policymakers, urban planners, and transportation companies in making informed decisions regarding the placement of autonomous vehicles, ultimately leading to optimized traffic management, enhanced mobility, and improved safety on roadways. Methodology: The methodology for studying the supplier selection process in the dairy industry will involve a combination of qualitative and quantitative approaches. Firstly, a comprehensive literature review will be conducted to gather insights into the existing supplier selection criteria and methodologies used in the industry. This will be followed by qualitative interviews and surveys with key stakeholders such as dairy company managers, procurement professionals, and suppliers to gather their perspectives on the supplier selection process. Additionally, quantitative analysis will be conducted to measure the importance and performance of various supplier selection criteria through statistical tools and models such as Analytic Hierarchy Process (AHP) and TOPSIS. Alternative parameters: Availability of road space, Segregation from vehicle traffic, Segregation from pedestrians, Number of passengers, Revenue potential. Evaluation parameters:Availability of road space, Segregation from vehicle traffic, Segregation from pedestrians, Number of passengers, Revenue potential Result:from the result the availability of road has secured the first place and segregation from pedestrians stays at the least. Conclusion:: availability of road plays an important role in building corridor selection for locating autonomous vehicles. Keywords: Automated driving, policy and societal implications, ripple effect, first, second, and third order impacts.

The fundamental task in an autonomous vehicle navigation system is localization from the available sensor measurements. GPS in the vehicles locates it with error of 1 to 10 meters so localization process should be performed to avoid fatal accidents. The realization of algorithms to estimate our vehicle’s position precisely is Localization. Odometry, Kalman Filter, Particle Filter and SLAM(Simultaneous Localization And Mapping) are the techniques used in an autonomous vehicle to localize itself in the map. Among these the particle filter is widely employed in the localization of autonomous vehicles as it provides accurate position of the vehicle in the environment. This paper aims at a localization technique for autonomous vehicles or robots using Particle Filter algorithm. The position estimator is implemented using the GPS and IMU sensor measurements. The map contains specific landmarks identified such as buildings and poles which assist the vehicle to know its position accurately by matching the distance between them in the particle filtering process. The results show that this algorithm can deliver accurate vehicle positioning even in erroneous GPS data.

Targetless LiDAR-camera extrinsic calibration remains challenging due to unreliable cross-modal correspondences and sensitivity to initialization. We present a targetless extrinsic calibration framework based on class-agnostic boundary mask alignment in a shared image-plane representation. This scheme first constructs consistent LiDAR-camera mask pairs from image-plane depth and intensity projections of LiDAR data and camera images. It then obtains robust initial pose candidates through bounded rotation-only global initialization and refines them using a computationally efficient stochastic gradient approximation to estimate the optimal extrinsic parameters. Experiments on the KITTI benchmark demonstrate a superior accuracy-runtime trade-off compared with a segmentation-based global optimization baseline, while real-world driving tests confirm stable cross-modal alignment under vibration and inter-modal timing jitter.

The integration of autonomous vehicles (AVs) into public transportation systems has the potential to transform urban mobility by enhancing efficiency, accessibility, and sustainability. As cities continue to expand, traditional transit systems face challenges such as traffic congestion, high operational costs, and limited coverage. AVs, with their ability to optimize routes, reduce human error, and operate continuously, offer a promising solution to improve public transit networks. The adoption of AV technology in public transportation can redefine mobility patterns, particularly by bridging first- and last-mile connectivity gaps, enhancing commuter convenience, and improving mobility for elderly and disabled passengers. Unlike conventional transit modes, autonomous buses, shuttles, and taxis can dynamically adjust routes based on real-time data, reducing travel time and increasing overall system efficiency. Additionally, AVs can function seamlessly with multi-modal transport networks, integrating with metro, rail, and bus services to create a more cohesive and accessible urban transit system. Beyond improving passenger experience, AVs offer significant economic and environmental benefits. Their reduced dependence on human drivers leads to lower labor costs, while advanced energy-efficient systems contribute to lower fuel consumption and carbon emissions. Autonomous public transit solutions also help alleviate traffic congestion by minimizing unnecessary stops and optimizing roadway usage through vehicle-to-infrastructure (V2I) and vehicle-to vehicle (V2V) communication. However, the transition to a fully autonomous public transport system is not without challenges. Cities and governments must address issues such as infrastructure readiness, regulatory frameworks, liability concerns, cyber security risks, and public acceptance to ensure a smooth and secure transition. By analyzing real-world case studies and pilot projects, this chapter provides insights into how different cities worldwide are integrating AVs into their transit systems. Several pilot programs have demonstrated the feasibility of self driving shuttle services, particularly in urban areas with high commuter demand and limited transit accessibility. Furthermore, technological advancements in artificial intelligence (AI), machine learning, and sensor-based navigation continue to refine AV capabilities, making them increasingly viable for large-scale deployment in public transportation. This chapter also explores future trends, policy recommendations, and best practices that can accelerate AV adoption in public transport. Governments, urban planners, and transportation authorities must collaborate to design regulatory policies that encourage innovation while ensuring passenger safety and data security. Investment in smart infrastructure, including intelligent traffic management systems and dedicated AV lanes, will be crucial for optimizing AV operations in public transit. Ultimately, the successful integration of AVs into public transportation can lead to a more efficient, inclusive, and sustainable urban mobility ecosystem. With the right combination of technological innovation, policy support, and public engagement, AVs have the potential to redefine how people commute in the cities of the future, offering safe, affordable, and environmentally friendly solutions.

Today's autonomous vehicles are equipped with a variety of sensors, control techniques and actuators depending on the objectives and requirements of the addressing issue. In research, some even employ commercial cars to house their devices. The high cost of all these components represents a limitation in the development of an autonomous vehicle system. Therefore, researchers have no option but to rely on experimentation under simulated environments or to use small vehicles that in many cases do not accurately represent a real parametric dynamic system. This article describes the design of an autonomous electric vehicle prototype, which includes control systems, microcontrollers, encoders, motor controllers, LIDAR, and cameras, with the aim of having a reference in the literature that is low-cost and whose program code is employable in a real-life context. The present vehicle, whose structure is based on a conventional tubular go-kart, has been provided with the capability of moving autonomously by using computer vision and artificial intelligence algorithms. This document records the first steps of the design taking into account parameters of a real-life environment and the flexibility of A.I. algorithms.

LiDAR-based simultaneous localization and mapping (SLAM) approaches have obtained considerable success in autonomous robotic systems. This is in part owing to the high accuracy of robust SLAM algorithms and the emergence of new and lower-cost LiDAR products. This study benchmarks the current state-of-the-art LiDAR SLAM algorithms with a multi-modal LiDAR sensor setup, showcasing diverse scanning modalities (spinning and solid state) and sensing technologies, and LiDAR cameras, mounted on a mobile sensing and computing platform. We extend our previous multi-modal multi-LiDAR dataset with additional sequences and new sources of ground truth data. Specifically, we propose a new multi-modal multi-LiDAR SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. With these maps, we then match real-time point cloud data using a normal distributions transform (NDT) method to obtain the ground truth with a full six-degrees-of-freedom (DOF) pose estimation. These novel ground truth data leverage high-resolution spinning and solid-state LiDARs. We also include new open road sequences with GNSS-RTK data and additional indoor sequences with motion capture (MOCAP) ground truth, complementing the previous forest sequences with MOCAP data. We perform an analysis of the positioning accuracy achieved, comprising ten unique configurations generated by pairing five distinct LiDAR sensors with five SLAM algorithms, to critically compare and assess their respective performance characteristics. We also report the resource utilization in four different computational platforms and a total of five settings (Intel and Jetson ARM CPUs). Our experimental results show that the current state-of-the-art LiDAR SLAM algorithms perform very differently for different types of sensors. More results, code, and the dataset can be found at GitHub.