Merging planning in dense traffic scenarios using interactive safe reinforcement learning

Autonomous navigation in dense traffic scenarios remains challenging for autonomous vehicles (AVs) because the intentions of other drivers are not directly observable and AVs have to deal with a wide range of driving behaviors. To maneuver through dense traffic, AVs must be able to reason how their actions affect others (interaction model) and exploit this reasoning to navigate through dense traffic safely. This paper presents a novel framework for interaction-aware motion planning in dense traffic scenarios. We explore the connection between human driving behavior and their velocity changes when interacting. Hence, we propose to learn, via deep Reinforcement Learning (RL), an interaction-aware policy providing global guidance about the cooperativeness of other vehicles to an optimization-based planner ensuring safety and kinematic feasibility through constraint satisfaction. The learned policy can reason and guide the local optimization-based planner with interactive behavior to pro-actively merge in dense traffic while remaining safe in case other vehicles do not yield. We present qualitative and quantitative results in highly interactive simulation environments (highway merging and unprotected left turns) against two baseline approaches, a learning-based and an optimization-based method. The presented results show that our method significantly reduces the number of collisions and increases the success rate with respect to both learning-based and optimization-based baselines.

In this contribution, we propose a novel nonlinear Model Predictive Control (NMPC) formulation based on the optimal control method Discrete Mechanics and Optimal Control (DMOC). We discuss the properties of the novel approach and compare it with the standard NMPC formulation based on the Multiple-Shooting method using numerical simulations. We also consider the real-time application of the NMPC algorithm by extending the DMOC based NMPC using the Real-Time Iteration scheme. We apply the extended NMPC controller to the swing-up and stabilization task of a double pendulum on a cart with limited rail length to show its capability for controlling nonlinear dynamical systems with input and state constraints.

We propose a framework for safe reinforcement learning that can handle stochastic nonlinear dynamical systems. We focus on the setting where the nominal dynamics are known, and are subject to additive stochastic disturbances with known distribution. Our goal is to ensure the safety of a control policy trained using reinforcement learning, e.g., in a simulated environment. We build on the idea of model predictive shielding (MPS), where a backup controller is used to override the learned policy as needed to ensure safety. The key challenge is how to compute a backup policy in the context of stochastic dynamics. We propose to use a tube-based robust nonlinear model predictive controller (NMPC) as the backup controller. We estimate the tubes using sampled trajectories, leveraging ideas from statistical learning theory to obtain high-probability guarantees. We empirically demonstrate that our approach can ensure safety in stochastic systems, including cart-pole and a non-holonomic particle with random obstacles.

This letter proposes a novel Nonlinear Model Predictive Control (NMPC) framework for Micro Aerial Vehicle (MAV) autonomous navigation in indoor enclosed environments. The introduced framework allows us to consider the nonlinear dynamics of MAVs, nonlinear geometric constraints, while guarantees real-time performance. Our first contribution is to reveal underlying planes within a 3D point cloud, obtained from a 3D lidar scanner, by designing an efficient subspace clustering method. The second contribution is to incorporate the extracted information into the nonlinear constraints of NMPC for avoiding collisions. Our third contribution focuses on making the controller robust by considering the uncertainty of localization in NMPC using Shannon's entropy to define the weights involved in the optimization process. This strategy enables us to track position or velocity references or none in the event of losing track of position or velocity estimations. As a result, the collision avoidance constraints are defined in the local coordinates of the MAV and it remains active and guarantees collision avoidance, despite localization uncertainties, e.g., position estimation drifts. The efficacy of the suggested framework has been evaluated using various simulations in the Gazebo environment.

An autonomous vessel can improve the intelligence, efficiency and economy of shipping logistics. To realize autonomous navigation control of underactuated vessels (USVs), this paper presents a controller that can make USV track a reference trajectory with only 2 inputs, i.e., surge and yaw. A nonlinear state-space model with 2 inputs considering environmental disturbances induced by wind, current and waves for a 3 degree of freedom (DOF) surface vessel is considered. Based on this model, a trajectory tracking controller using Nonlinear Model Predictive Control (NMPC) is designed. System constraints on inputs, input increment and output are incorporated in the NMPC framework. Simulation results show that the controller can track an ellipse trajectory well and that the tracking errors are within acceptable ranges, while system constraints are satisfied.

This paper presents a local path planning approach that avoids both static and dynamic multi-obstacles for autonomous car. The proposed algorithm is able to alter the predefined waypoints from global path planning when obstacles are detected. A set of paths are generated based on the sigmoid function for overtaking obstacles. Every single path is assessed according to a cost function which includes path safety, comfortability, and length. Genetic algorithm is used to optimize parameters of the selected sigmoid function for smoothness and minimum length. Non-linear Model Predictive Controller is designed for trajectory tracking considering vehicle's dynamics. Simulation results reveal that the algorithm carries out obstacle avoidance and can handle multi-obstacles furthermore, the lateral acceleration is reduced by 44.27% and 50% in both static and dynamic scenarios effectively.

Global path planning and waypoint following for heterogeneous unmanned surface vehicles assisting inland water monitoring

Optimal energy allocation strategy for electric vehicles based on the real-time model predictive control technology

In this paper, we propose a real-time motion planning system and a precise trajectory tracking controller for unmanned aerial vehicles autonomous navigation in complex environments. Based on the simple path planned by the path searching method, we generate the convex flight corridors for quadrotor safe flying by cube inflation. We generate the trajectories composed with piecewise Bezier curves in these convex flight corridors by formulating the´ problem as a quadratic program. The time of the trajectories is re-allocated to satisfy the dynamic feasibility. We apply a nonlinear model predictive controller (NMPC) considering the full system model to track the generates trajectories. The performance of our proposed method is tested in a complex simulation system.

Obstacle avoidance problem for the autonomous vehicle has received more and more attention these years, especially for the autonomous heavy vehicle, which has the high gravity center, large carrying capacity and promising commercial prospect. However, the vehicle dynamic properties have been rarely considered for path planning, which is a crucial factor for driving safety. An integrated collision avoidance framework that consists of a new path planner and a tracking controller considering multiple dynamic constraints is proposed for an autonomous multi-axis distributed vehicle. In the upper level, a new optimal path generator based on the B-splines is designed for safety and stability purposes, and the collision avoidance path is optimized considering multiple dynamic constraints, including the tire nonlinear dynamic, lateral stability limits and the anti-slip constraints. Meanwhile, a safety assessment evaluator according to the time to collision calculation is designed for risk level recognition and path planning weight determination. In the lower level, a path tracking controller based on the nonlinear model predictive control (NMPC) is designed to improve the control accuracy, and a high-fidelity nonlinear dynamic predictive vehicle model with 16 degrees of freedom is established to depict the vehicle dynamic properties comprehensively. To solve the nonlinear optimization problem and get the optimal control steering angle output, an adaptive weight adjustment strategy and a varying predictive duration method are formulated. The optimization constraints related to steering actuator ability, plane stability limits, rollover prevention thresholds and tire adhesion requirements are constructed and converted as the bounds of the vehicle dynamic states. To verify the proposed framework, the simulation and HIL test platform are built and different driving scenarios are validated. The test results show the satisfactory performance of path planning and path tracking considering the dynamic constraints for safety and stability

In order to solve the problem of autonomous localization and navigation caused by the loss of the position information of the mobile robot in a complex environment, a method for autonomous navigation of a mobile robot based on ROS is proposed in this paper. The ROS platform is used to avoid the problems of code redundancy, poor portability and so on. In this method, firstly, monte carlo algorithm is used to realize the autonomous localization of the robot, raster map is constructed with lidar data, and the concept of raster occupancy is introduced to reduce the influence of sensor noise. Then an improved ant colony algorithm is proposed for global path planning to realize autonomous navigation of the robot. At the end of this paper, the robot control system is combined with ROS navigation system for map construction and autonomous navigation experiments. The experimental results show that the improved ant colony algorithm can effectively shorten the number of iterations, reduce the number of iterations by 38%, improve the adaptability to the environment, facilitate the fast finding of the shortest path, and accurately realize the autonomous navigation task of the robot.

Flexure hinge mechanism plays a key part in realization of terminal nano-positioning. The performance of flexure hinge mechanism is determined by its positioning design. Based on the actual working conditions, its finite element model is built and calculated in ANSYS. Moreover, change trends of deformation and natural frequency with positioning design parameters are revealed. And sensitivity analysis is performed for exploration response to these parameters. These parameters are used to build four objective functions. To solve it conveniently, the multi-objective optimization problem is transferred to the form of single-objective function with constraints. An optimal mechanism is obtained by an optimization method combining ANSYS with MATLAB. Finite element numerical simulation has been carried out to demonstrate the superiority of the optimal flexure hinge mechanism, and the superiority can be further verified by experiment. Measurements and tests have been conducted at varying accelerations, velocities, and displacements, to quantify and characterize the amount of acceleration responses obtained from flexure hinge mechanism before and after optimization. Both time- and frequency-domain analyses of experimental data show that the optimal flexure hinge mechanism has superior effectiveness. It will provide a basic for realizing high acceleration and high precision positioning of macro–micro motion platform.

In this article, a safe deep reinforcement learning (DRL) control method based on a safe reward shaping method is proposed and applied to the constrained control for an electro‐hydraulic servo system (EHSS). The proposed control method improves the safety of the constrained control for a nonlinear system with the minimal intervention to the optimization of the performance objective, while the convergence speed of the DRL process has accelerated. By introducing control barrier functions (CBFs) to the reward shaping, a CBF‐based potential difference term is designed to shape the safe reward, which not only provides the safe guidance for the DRL process by encoding the safety constraints of the nonlinear system, but also considers effects of the complex safety transformation on the convergence process in the DRL. Then the safe reward‐based DRL control method is presented to learn the optimal safety policy of position tracking for the EHSS with position error constraints by planning and optimizing the safety together with the performance objective. Theoretical analysis is given to demonstrate that the proposed control method with the safe reward can achieve the optimal safety performance for the constrained control system. Experimental results of the constrained control for the EHSS with system uncertainties and perturbations are also exhibited, to show that the proposed control method converges fast and performs safer and better than the conventional control methods.

With Multi-Objective Optimization (MOO) mechanisms, many practical scenarios are imitated in Wireless Sensor Networks (WSNs). In MOO numerous desirable conflicting or non-conflicting objectives contend with one another and the decision has to be done among multiple available solutions. Based on the type of situation, Programme, and issue to be solved, the MOO problem has varied solutions. The solution chosen is a tradeoff solution on several occasions. In WSN, it is possible to identify MOO issues and associated solutions based on network architecture, node deployment, MAC strategies, routing, data aggregation, node mobility, etc. In this context, the paper proposes mobility aware, competent; delay tolerant Energy Efficient Hybrid Routing Protocol (EEHRP). Optimizing several metrics to pick the best route from the source to the target node is the cornerstone of the EEHRP. Multi-Objective optimization from optimization theory is a NP-hard problem. EEHRP seeks to obtain a Pareto optimal solution for the section of best MOO-based route under sensor node. The simulation results demonstrate that, relative to state-of-the-art solutions, EEHRP is efficient in terms of energy, throughput, delay, control- and routing-overheads. Furthermore, the paper investigates statistical significance of the findings obtained across confidence intervals. To prove EEHRP’s competence, a confidential interval of 95% is inserted into the simulation results obtained to represent margin of error around the estimated points. The on-hand state-of-art solutions and the propensity of the research fraternity in relation to MOO are also analyzed in this paper.

As human–robot shared service environments become increasingly common, autonomous navigation in narrow space environments (NSEs), such as indoor corridors and crosswalks, becomes challenging. Mobile robots must go beyond reactive collision avoidance and interpret surrounding risks to proactively select safer routes in dynamic and spatially constrained environments. This study proposes a deep reinforcement learning (DRL)-based navigation framework that enables mobile robots to interact with pedestrians while identifying and traversing open and safe spaces. The framework fuses 3D LiDAR and RGB camera data to recognize individual pedestrians and estimate their position and velocity in real time. Based on this, a human-aware occupancy map (HAOM) is constructed, combining both static obstacles and dynamic risk zones, and used as the input state for DRL. To promote proactive and safe navigation behaviors, we design a state representation and reward structure that guide the robot toward less risky areas, overcoming the limitations of traditional approaches. The proposed method is validated through a series of simulation experiments, including straight, L-shaped, and cross-shaped layouts, designed to reflect typical narrow space environments. Various dynamic obstacle scenarios were incorporated during both training and evaluation. The results demonstrate that the proposed approach significantly improves navigation success rates and reduces collision incidents compared to conventional navigation planners across diverse NSE conditions.