A Reinforcement Learning based Decision-making System with Aggressive Driving Behavior Consideration for Autonomous Vehicles

Along with the rapid development of autonomous driving technology, autonomous vehicles are showing a trend of practicality and popularity. Autonomous vehicles perceive environmental information through sensors to provide a basis for the decision making of vehicles. Based on this, this paper investigates the lane-changing decision-making behavior of autonomous vehicles. First, the similarity between autonomous vehicles and moving molecules is sought based on a system-similarity analysis. The microscopic lane-changing behavior of vehicles is analyzed by the molecular-dynamics theory. Based on the objective quantification of the lane-changing intention, the interaction potential is further introduced to establish the molecular-dynamics lane-changing model. Second, the relationship between the lane-changing initial time and lane-changing completed time, and the dynamic influencing factors of the lane changing, were systematically analyzed to explore the influence of the microscopic lane-changing behavior on the macroscopic traffic flow. Finally, the SL2015 lane-changing model was compared with the molecular-dynamics lane-changing model using the SUMO platform. SUMO is an open-source and multimodal traffic experimental platform that can realize and evaluate traffic research. The results show that the speed fluctuation of autonomous vehicles under the molecular-dynamics lane-changing model was reduced by 15.45%, and the number of passed vehicles was increased by 5.93%, on average, which means that it has better safety, stability, and efficiency. The molecular-dynamics lane-changing model of autonomous vehicles takes into account the dynamic factors in the traffic scene, and it reasonably shows the characteristics of the lane-changing behavior for autonomous vehicles.

Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL) has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision-making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic (MA2C) method is proposed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is designed to incorporate fuel efficiency, driving comfort, and the safety of autonomous driving. A comprehensive experimental study is made that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety, and driver comfort.

<div class="section abstract"><div class="htmlview paragraph">Segment with lane drops are very important in freeway systems since they are major constrains to traffic flow and safety. The frequency of capacity reductions and higher safety risks is proportional to an increase in lane-changing actions, which worsen traffic congestion, decrease road capacity, and increase the risk of an accident. Traditional traffic management strategies that rely on physical structures and driver’s decision making often fail under such conditions. This paper provides a detailed lane change control strategy specific to freeway segments with lane reduction in the connected and autonomous vehicle (CAV) environment. The strategy combines both centralized and decentralized techniques to improve the vehicle’s lane-changing behavior and density. A cellular transmission model of lane-level is proposed for the centralized control of the linked vehicles based on the ratio of the driver compliance. The model derives the density equation and transforms the lane-changing problem before the work zone into merging traffic flow problem. The optimization model is developed based on the total trip time, density deviation, and total lane changes, with constraints on the cell reception capacity and lane changing ratios. Control parameters for lane change distribution are identified using genetic algorithms to solve the problem. For the decentralized control, a reinforcement learning solution is introduced which uses deep Q-networks (DQN) to improve lane-changing actions. The reward function takes into account the traffic efficiency and the impact of lane changing, and the continuous action space is discretized for application. The control mechanism is evaluated by the simulation of a work zone scenario that includes two restricted lanes on the Shanghai-Nanjing Expressway. It also shows that there is an improvement of 3% to 6% in traffic flow and velocity as compared to single-strategy approaches. The collaborative control strategy significantly enhances traffic flow and reduces congestion at bottlenecks and offers valuable information for future traffic control in CAV environments.</div></div>

This article describes the design, implementation, and evaluation of an active lane change control algorithm for autonomous vehicles with human factor considerations. Lane changes need to be performed considering both driver acceptance and safety with surrounding vehicles. Therefore, autonomous driving systems need to be designed based on an analysis of human driving behavior. In this article, manual driving characteristics are investigated using real-world driving test data. In lane change situations, interactions with surrounding vehicles were mainly investigated. And safety indices were developed with kinematic analysis. A safety indices–based lane change decision and control algorithm has been developed. In order to improve safety, stochastic predictions of both the ego vehicle and surrounding vehicles have been conducted with consideration of sensor noise and model uncertainties. The desired driving mode is decided to cope with all lane changes on highway. To obtain desired reference and constraints, motion planning for lane changes has been designed taking stochastic prediction-based safety indices into account. A stochastic model predictive control with constraints has been adopted to determine vehicle control inputs: the steering angle and the longitudinal acceleration. The proposed active lane change algorithm has been successfully implemented on an autonomous vehicle and evaluated via real-world driving tests. Safe and comfortable lane changes in high-speed driving on highways have been demonstrated using our autonomous test vehicle.

Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach

Nowadays, scientific developments in the vehicle and infrastructure automation sectors are developing steadily to benefit end users in ways like reducing traffic congestion, enhancing safety, facilitating stress-free travel, reducing fuel costs, and enabling smart parking. Autonomous vehicles sometimes known as self-driving cars, can entirely replace conventional automobiles to accomplish safe and reliable intelligent transportation systems (ITS). Autonomous vehicles have improved sensing capabilities and can drive on roads that would be difficult for humans. Software, technology, and people, together with their interactions, are the building blocks of Autonomous vehicles. Even though Autonomous vehicles have been studied extensively, many issues have yet to be fixed. One of the biggest obstacles to developing Autonomous vehicles is improving their ability to communicate with other vehicles and pedestrians. Second, it's unclear whether society will benefit or suffer from people's tendency to place an excessive amount of faith in Autonomous vehicles and assume that they will never malfunction. Capabilities analysis is essential to making sure autonomous vehicles are safe and widely accepted. The proposed review outlines the key benefits, drawbacks, and difficulties of autonomous vehicle technology and provides a detailed analysis of the different autonomous vehicle mechanisms. Further, the developed review intends to formulate future prospects by examining the advantages and disadvantages of autonomous driving technology to encourage the development of innovative autonomous vehicle technologies to prevent human error and increase road safety and security.

The rapid growth in Autonomous Vehicle (AV) technology endeavors increased attention towards road safety in recent days. Particularly, a higher number of road accidents occurs when the AV tries to overtake or change the lane. To cut down the number of accidents and improve traffic reliability, the AV should be capable of making intelligent decisions and communicating those to other AVs. Therefore, in this paper, a Cooperative Collision avoidance scheme for AVs at Overtaking and Lane Changing maneuver (CCAV-OLC) is proposed. The Inverse Reinforcement Learning (IRL) in the CCAV-OLC scheme, processes on the given number of expert demonstrations for automatically acquiring the reward function, and thereby imitating actual human driving strategy and decisions. However, the adaptability of IRL to a high-dimensional AV environment restricts the performance of the CCAV-OLC scheme. To overcome this, the IRL in CCAV-OLC leverages the Gaussian Process (GP) regression model (IRL-GP), which enables data-efficient Bayesian prediction even when the number of demonstrations is very low. After taking intelligent decisions in overtaking and lane changing maneuvers, the AVs cooperatively communicate and exchange the decisions with each other by 6th Generation Vehicle-to-Everything (6G-V2X) communications, which further improves the accuracy and lessens the time taken for making optimal decisions. The experimental results show that the AVs clone the expert's optimal driving strategy and avoid the collisions to a greater extent.

This paper presents the implementation of an autonomous electric vehicle (EV) project in the National Taiwan University of Science and Technology (NTUST) campus in Taiwan. The aim of this work was to integrate two important practices of realizing an autonomous vehicle in a campus environment, including vehicle positioning and path tracking. Such a project is helpful to the students to learn and practice key technologies of autonomous vehicles conveniently. Therefore, a laboratory-made EV was equipped with real-time kinematic GPS (RTK-GPS) to provide centimeter position accuracy. Furthermore, the model predictive control (MPC) was proposed to perform the path tracking capability. Nevertheless, the RTK-GPS exhibited some robust positioning concerns in practical application, such as a low update rate, signal obstruction, signal drift, and network instability. To solve this problem, a multisensory fusion approach using an unscented Kalman filter (UKF) was utilized to improve the vehicle positioning performance by further considering an inertial measurement unit (IMU) and wheel odometry. On the other hand, the model predictive control (MPC) is usually used to control autonomous EVs. However, the determination of MPC parameters is a challenging task. Hence, reinforcement learning (RL) was utilized to generalize the pre-trained datum value for the determination of MPC parameters in practice. To evaluate the performance of the RL-based MPC, software simulations using MATLAB and a laboratory-made, full-scale electric vehicle were arranged for experiments and validation. In a 199.27 m campus loop path, the estimated travel distance error was 0.82% in terms of UKF. The MPC parameters generated by RL also achieved a better tracking performance with 0.227 m RMSE in path tracking experiments, and they also achieved a better tracking performance when compared to that of human-tuned MPC parameters.

Connected vehicle (CV) and autonomous vehicle (AV) technologies are rapidly advancing. Traffic control for future hurricane evacuation can be radically different compared to the current approaches in use. Extensive modeling research and limited field deployments have demonstrated the benefits of CV, AV technologies in achieving higher mobility, safety, and environmentally friendly solutions. One area that has not been explored in prior research is the benefits these technologies may offer to the emergency evacuation performance. Specifically, evacuation traffic control is one area where CV and AV technologies can have a transformative effect. At-grade intersections and grade-separated interchanges on evacuation routes are susceptible to congestion due to the increased demand competing for road usage. Traffic control methods that efficiently route traffic at intersections and interchanges also enhance the overall evacuation performance. This paper proposes an autonomous reservation-based intersection control (AReBIC) system for evacuation traffic control. The AReBIC system receives and processes reservation requests from approaching vehicles in real-time and routes them free of conflict. The results of a case study of Hampton Roads, Virginia, showed that the AReBIC system outperformed the existing traffic control practice in terms of mobility and safety. The average speeds increased by 29% and total delays reduced by 88% when intersection control was changed to the AReBIC system. The number of crossing conflicts decreased by 88% and lane change conflicts decreased by 47%.

Preparing drivers for the future: Evaluating the effects of training on drivers’ performance in an autonomous vehicle landscape

In contemporary development, autonomous vehicles (AVs) have emerged as a potential solution for sustainable and smart transportation to fulfill the increasing mobility demands whilst alleviating the negative impacts on society, the economy, and the environment. AVs completely depend on a machine to perform driving tasks. Therefore, their quality and safety are critical concerns for driving users. AVs use advanced driver assistance systems (ADASs) that heavily rely on sensors and camera data. These data are processed to execute vehicle control functions for autonomous driving. Furthermore, AVs have a voice communication system (VCS) to interact with driving users to accomplish different hand-free functions. Some functions such as navigation, climate control, media and entertainment, communication, vehicle settings, vehicle status, and emergency assistance have been successfully incorporated into AVs using VCSs. Several researchers have also implemented vehicle control functions using voice commands through VCSs. If a situation occurs when AV has lost control due to malfunctioning or fault in the installed computer, sensors and other associated modules, driving users can control the AV using voice notes to perform some driving tasks such as changing speeds, lanes, breaking, and directing the car to reach a safe condition. Furthermore, driving users need manual control over AV to perform these tasks in some situations, like lane changing or taking an exit due to divergence. These tasks can also be performed with the help of voice commands using VCSs. Therefore, finding the exact voice note used to instruct different actuators in risk situations is crucial. As a result, VCSs can greatly improve safety in critical situations where manual intervention is necessary. AVs’ functions and quality can be significantly increased by integrating a VCS with an ADAS and developing an interactive ADAS. Now, the driver functions are controlled by voice features. Therefore, natural language processing is utilized to extract the features to determine the user’s requirements. The extracted features control the vehicle functions and support driving activities. The existing techniques consume high computation while predicting the user command and causing a reduction in the AVs’ functions. This research issue is overcome by applying the variation continuous input recognition model. The proposed approach utilizes the linear training process that resolves the listening and time-constrained problems and uncertain response issues. The proposed model categorizes the inputs into non-trainable and trainable data, according to the data readiness and listening span. Then, the non-distinguishable data were validated by dividing it into the linear inputs used to improve the response in the AVs. Thus, effectively utilizing training parameters and the data decomposition process minimizes the uncertainty and increases the response rate. The proposed model has significantly improved the exact prediction of users’ voice notes and computation efficiency. This improvement enhances the VCS quality and reliability used to perform hand-free and vehicle control functions. The reliability of these functions ultimately improves the safety of AVs’ driving users and other road users.

Advanced Driver Assistance Systems (ADAS) have emerged as a critical technology in enhancing vehicle safety, providing convenience to drivers, and facilitating the transition toward fully autonomous vehicles. The core of these systems relies heavily on computer vision (CV) technologies, which enable vehicles to interpret and understand their surrounding environment in real time. The integration of computer vision in ADAS allows vehicles to recognize and process various objects such as pedestrians, vehicles, traffic signs, and lane markings, significantly improving both safety and the driving experience. In this paper, we explore the pivotal role of computer vision in the development and implementation of ADAS in autonomous vehicles. We begin by reviewing the evolution of ADAS, detailing the different sub-systems such as lane departure warning, automatic emergency braking, adaptive cruise control, and parking assistance, all of which depend on sophisticated image processing techniques. Furthermore, the paper examines the intersection of computer vision with other emerging technologies such as machine learning, deep learning, and sensor fusion. We discuss how these innovations enable ADAS to function more effectively in complex, dynamic environments. The methodology section focuses on cutting-edge algorithms used for object detection, semantic segmentation, and real-time image processing, which are integral to the functionality of modern ADAS. Additionally, we evaluate the challenges that computer vision systems face in real-world applications, including issues with sensor calibration, environmental conditions, computational limitations, and the integration of data from multiple sensors (e.g., cameras, LiDAR, radar). The paper also presents an analysis of the results obtained from various real-world implementations of ADAS technologies, highlighting improvements in driver safety and the reduction of traffic-related accidents. We explore the potential of computer vision in enabling fully autonomous driving by addressing the current gaps in technology and the advancements needed to close these gaps. Finally, the paper concludes with a discussion on the future of ADAS and the role of computer vision in realizing fully autonomous, reliable, and efficient transportation systems. We argue that continued research and development in computer vision, combined with innovations in deep learning and sensor technologies, are critical to advancing the capabilities and reliability of autonomous vehicles in the near future.

Different perspectives discuss autonomous vehicle (AV) as predicted future transportation that includes design, engineering, economics, computer science, social, economic, legislation, and ethical aspects. On the other hand, AV technology offers new problems such as new engineering aspects, social and ethical problems, and presents in the form of an idealized decision-making problem. There are many benefits to AV vehicles, which are at the forefront of technology development. However, the level of cybersecurity provided by this technology is poorly understood. Cyber-attacks against AV can have severe implications for safety, privacy, manufacturers' reputation, and overall perception. In this research, we examine public opinion regarding the implications and challenges that might face AV technology in the United Arab Emirates (UAE). We claim that the successful adoption of AV in future transport systems will be achieved by reasonable risk management approaches that deal with different potential threats of cybersecurity. This survey produced usable responses from 204 participants. The significant result was that the respondents had not previously heard of vehicle-related technology; however, most had a positive first opinion on technology. Around 85% of participants felt that the expected benefits were likely to occur. They expressed a high level of concern regarding security and performance issues. Respondents stated that safety is the most crucial aspect of driverless vehicles. Mostly, it stated that personal communication devices should integrate with the AV's internet connection. The primary outcomes are that the population feels positive about AV and have optimistic expectations regarding its benefits. While maintaining some concerns, most of the people are willing to drive the AV.

Liability Design for Autonomous Vehicles and Human-Driven Vehicles: A Hierarchical Game-Theoretic Approach

Problem definition: Autonomous vehicles (AVs) are predicted to enter the consumer market in less than a decade. There is currently no consensus on whether their presence will have a positive impact on users and society. The skeptics of automation foresee increased congestion, whereas the advocates envision smoother traffic with shorter travel times. We study the automation controversy and advise policymakers on how and when to promote AVs. Academic/practical relevance: The AV technology is advancing rapidly and there is a need to study its impact on social welfare and the likelihood of its adoption by the public. Methodology: We use supply-demand theory to find the equilibrium number of trips for autonomous and regular households. We develop a simulation model of peer-to-peer AV sharing. We compare the socially optimal level of automation with the selfish adoption patterns where households independently choose their vehicle type. Results: We establish that the optimal social welfare is influenced by: (i) the network connectivity, that is, the ability of the infrastructure to serve AVs, (ii) the additional comfort provided by AVs that allows passengers to engage in other productive activities instead of driving, and (iii) the AV sharing patterns that reduce ownership costs, but create empty vehicle trips that increase congestion. Managerial implications: We investigate the impact of AVs in a case study of Toronto and show that partial automation maximizes social welfare. We show that the comfort of AVs may add traffic that compromises social welfare. Moreover, although traffic increases with automation, travel times may decrease because of significant improvements in traffic flow caused by AV connectivity in the network.