Autonomous Agents for Traffic Simulation and Control

The traffic congestion problem in urban areas is worsening since traditional traffic signal control systems cannot provide efficient traffic control. Therefore, dynamic traffic signal control in intelligent transportation system (ITS) recently has received increasing attention. This study devised an adaptive and cooperative multi-agent fuzzy system for a decentralized traffic signal control. To achieve this goal we have worked on a model, which has three levels of control. Every intersection is controlled by its own traffic situation, its neighboring intersections recommendations and a knowledge base, which provides the traffic pattern of each intersection in any particular day of the week and hour of the day. The proposed architecture comprises a knowledge base, prediction module and a traffic observer that provide data to real traffic data preparation module, then a decision-making layer takes decision to how long should the intersection green light be extended. The proposed architecture can achieve dynamic traffic signal control. We have also developed a NetLogo-based traffic simulator to serve as the agents' world. Our approach is tested with traffic control of a large connected junction and the result obtained is promising; The average delay time can be reduced by 21.76% compared to the conventional fixed sequence traffic signal and 14.77% compared to the vehicle actuated traffic signal control strategy.

The acceleration of urbanization has left cities around the world facing significant challenges, including traffic congestion and inefficient management. Intelligent transportation systems provide new technological means to address these issues, especially in traffic signal control, where advanced communication technologies play an important role in facilitating the process. The potential of long range radio (LoRa) technology in intelligent traffic signal control systems is attracting growing interest, largely due to its capacity for long-range transmission and low power consumption. This paper explores the potential benefits of LoRa technology in traffic flow monitoring and signal control by reviewing existing studies, especially in the field of long-distance transmission and low-power communication. The study indicates that LoRa technology can effectively boost the efficacy of traffic management, reduce traffic congestion, and provide technical support for the development of future intelligent transportation systems. Nevertheless, the stability of LoRa technology in high-density urban environments and its integration with other technologies still present challenges. Further research can investigate the anti-jamming ability, communication rate optimization, and its potential for wider application of LoRa technology in Intelligent Transport Systems (ITS) to promote more efficient urban traffic management

Adaptive traffic signal control (ATSC) based on deep reinforcement learning (DRL) has shown promising prospects to reduce traffic congestion. Most existing methods keeping traffic signal phases fixed adopt two agent actions to match a four-phase suffering unstable performance and undesirable operation in a four-phase signalized intersection. In this paper, a Double Deep Q-Network (DDQN) with a dual-agent algorithm is proposed to obtain a stable traffic signal control policy. Specifically, two agents are denoted by two different states and shift the control of green lights to make the phase sequence fixed and control process stable. State representations and reward functions are presented by improving the observability and reducing the leaning difficulty of two agents. To enhance the feasibility and reliability of two agents in the traffic control of the four-phase signalized intersection, a network structure incorporating DDQN is proposed to map states to rewards. Experiments under Simulation of Urban Mobility (SUMO) are carried out, and results show that the proposed traffic signal control algorithm is effective in improving traffic capacity.

The integration of urban traffic signal control (TSC) system and dynamic route guidance (DRG) system became a development trend of intelligent transportation system (ITS). This paper firstly describes the characters of integration of the two systems and the theory of multi-agent system. Based on the multi-agent system theory, we present a system architecture of the integration system of TSC and DRG with two kinds interactive agent model architecture: traffic signal control agent model and dynamic route guidance agent model. We then use the TSC agent model as an example to discuss the architecture and interactive mechanism of the agent models. The multi-agent system provides a possible means for the integration of TSC and DRG. Finally this paper studies the coordination model, indicates how these agent models coordinate.

Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework

The efficiency of traffic flows in urban areas largely depends on signal operation. The state-of-the-art traffic signal control strategies are not able to efficiently deal with varying or over-saturated conditions. To optimize the performance of existing traffic signal infrastructure, we present an end-to-end autonomous intersection control agent, based on Deep Reinforcement Learning (DRL). In the recent years, DRL has emerged as a powerful tool, solving control problems involving sequential decision making and demonstrating unprecedented success in complex settings. Our DRL traffic intersection control agent configures the traffic signal regimes based solely on live photo-realistic camera footage. We demonstrate that our agent consistently, significantly outperforms state-of-the-art fixed (pre-defined) and adaptive (induction loop-based) signal control methods under a wide range of ambient conditions, by increasing the traffic throughput and decreasing the intersection traversal time for individual vehicles.

Cooperative traffic signal control using Multi-step return and Off-policy Asynchronous Advantage Actor-Critic Graph algorithm

Introduction Drag on city traffic has escalated to be a pivotal problem in almost all cities in the world, and commuter travel time, fuel consumption and emission have also soared. Fixed-time signal control systems have been traditionally used in traffic signal control, but their optimality highly depends on the time of day they are applied and they are not capable of real-time traffic control. In order to overcome these drawbacks, intelligent and adaptive traffic control approaches are being explored by both scientists and traffic developers. Objective The purpose of this research is to create and assess a dynamic traffic light control system by utilizing Snake Optimization (SO), a novel meta-heuristic algorithm. In order to minimize vehicle queues, cut down on delays, and enhance overall traffic flow, the main goal is to examine how well SO optimizes traffic signal timings based on real-time traffic data. by contrasting conventional static control techniques and meta-heuristics with the SO-based dynamic approach. Proposed Method To find the best traffic light phase durations at intersections, the suggested approach uses the Snake Optimization (SO) algorithm, a bio-inspired meta-heuristic technique. Real traffic data from the Algerian city of Algiers, which depicts realistic traffic volumes and patterns, is used to test the method. The SUMO (Simulation of Urban Mobility) simulator, an open-source program popular for simulating microscopic traffic, is used to construct the experimental framework. Traffic scenario modeling, control algorithm implementation, and performance evaluation in a simulated urban setting are all made possible by SUMO. The following steps are part of the optimization process: gathering of data, using the SUMO simulator to model traffic flow and intersections, incorporating the SO algorithm to modify traffic light timings in real time in response to inputs, evaluation of performance using metrics like vehicle throughput, average waiting time, reduction of CO2 emission and total travel time, and evaluation of the benefits obtained by the SO-based approach through comparison with a baseline static control system and ACO-DSOS meta-heuristic. Conclusion According to the simulation results, the suggested approach, which is based on the Snake Optimization algorithm, performs better than other meta-heuristics and conventional static control systems. The algorithm's effectiveness in controlling traffic flows is validated by quantitative comparisons. According to the study's findings, SO is a practical and successful strategy for maximizing traffic light control in practical settings.

Intelligent Traffic Lights: Green Time Period Negotiaton

This paper deals with modeling method and application of Fuzzy Discrete Event System(FDES). FDES have characteristics which Crisp Discrete Event System(CDES) can't deals with and is constituted with the events that is determined by vague and uncertain judgement like biomedical or traffic control. We proposed Real-time Fuzzy Temporal Logic Framework(RFTLF) to model Fuzzy Discrete Event System. It combines Temporal Logic Framework with Fuzzy Theory. We represented the model of traffic signal systems for intersection to have the property of Fuzzy Discrete Event System with Real-time Fuzzy Temporal Logic Framework and designed a traffic signal controller for smooth traffic flow. Moreover, we proposed the method to find the minimum-time route to reach the desired destination with information obtained in each intersection. In order to evaluate the performance of Real-time Fuzzy Temporal Logic Framework model proposed in this paper, we simulated unit-time extension traffic signal controller model of the latest signal control method on the same condition.

Dynamic traffic assignment (DTA) applications require traffic signal control data that are typically difficult to obtain and cumbersome to code in the required format. In addition, the evaluation of future scenarios requires future traffic signal settings consistent with the forecast demand. These signal settings are not known a priori and are costly to estimate. Intuitively, future signal timings need to be reasonably optimized so as to represent what the traffic management agency will do. In the literature, integration between traffic control and DTA models has been formulated as a bi-level or single-level optimization problem with system or user optimal constraints. Most existing solution procedures require certain nested structure with an inner-loop algorithm solving the problem of user-equilibrium or system-optimal assignment and the outer-loop algorithm searching for the optimal signal-timing settings. Most of these solution approaches remain only research tools without practical use because of computational intractability. This research proposes an efficient solution algorithm to the problem. An adaptive traffic signal control model is embedded in a simulation-based DTA model. For each inbound approach at an intersection of interest, the adaptive model uses upstream information and a dynamic rolling-horizon approach to project traffic flow conditions for a dynamic but short (projection) period. The adaptive model provides the signal settings during the entire process of traffic flow simulation and for every iteration of the solution algorithm. Thus, during the entire solution process, the experienced travel times and resulting traffic assignment flows are based on the adaptive (demand-responsive) signal settings, allowing the DTA flows and the adaptive signal settings to be generated simultaneously in a single-loop algorithmic structure. Simulation experiments illustrate the capabilities of the proposed approach.

Rapid urbanization and increasing ownership of private vehicles have led to severe traffic congestion, which has become a critical challenge in modern urban areas operated with signalized intersections. This study proposes and evaluates a novel AI-based composite framework for traffic signal control optimization, which integrates a Long Short-Term Memory (LSTM) model for traffic flow prediction, a Deep Neural Network (DNN) for vehicle speed estimation, and a Genetic Algorithm (GA) for signal phase optimization under open-loop and closed-loop objectives. It has been validated through high-fidelity microscopic simulations (i.e. VISSIM), calibrated with field data at the atypical six-leg intersection in Busan, Korea. This study found that while the overall performance of the framework for 24 hours exhibits clear trade-offs (i.e. slight total traffic throughput decrease of 2.2%), its primary value lies in its demonstrated ability to effectively mitigate severe congestion during peak hours (i.e. speed increase of up to 38.5% during the evening peak), delivering significant gains in traffic quality in need. This study also revealed that the system actively performs traffic flow redistribution as an intelligent traffic control strategy. The contributions of this study are threefold. First, it provides a methodological blueprint for an interpretable, modular control framework that decouples prediction processes from optimization, offering a transparent alternative to end-to-end Reinforcement Learning (RL) models. Second, this study offers crucial empirical insight for the evaluation of adaptive traffic control systems. Their true effectiveness may be best measured by peak hour performance rather than 24-hour averages. Finally, this study makes a conceptual contribution by identifying and quantifying the ‘reality gap’ between the predictions of an offline-trained optimizer and the outcomes of a high-fidelity simulation, highlighting a fundamental challenge for the field. HIGHLIGHTS A hierarchical AI framework is an alternative to reinforcement learning model Closed-loop outperforms open-loop in optimizing real time traffic signal control A speed stabilization objective is adopted over a speed maximization approach Leading to tangible gains in traffic quality during peak hour over 24-hour average

Having recognized the mutual interaction that exists between traffic signal control and route choice behavior, this paper considers the traffic assignment problem for a traffic signal controlled network. The interaction is characterized by (1) the effects of delay, due to traffic signals, on route choice, and (2) the influence of the resulting traffic assignment, due to the changes in route selection, on the timing of traffic signals. In this paper Webster’s method for calculating green splits and the HCM delay model are used to develop a link performance function that implicitly produces optimal signal settings for the arriving traffic volumes. This link-performance function is then used in an assignment procedure to develop a combined traffic signal control/traffic assignment model. The model provides simultaneous solution of the traffic signal control problem and the traffic assignment problem. The result is a set of signal settings that optimally matches the assigned traffic flows. The combined model results in a non-convex problem which produces multiple equilibria. A modification of Wardrop’s two principles is proposed to produce an assignment which converges to a single solution.KeywordsTraffic SignalRoute ChoiceDelay FunctionTraffic AssignmentArrival Flow RateThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Traffic signal control plays a pivotal role in the management of urban traffic flow. With the rapid advancement of reinforcement learning, the development of signal control methods has seen a significant boost. However, a major challenge in implementing these methods is ensuring that signal lights do not change abruptly, as this can lead to traffic accidents. To mitigate this risk, a time-delay is introduced in the implementation of control actions, but usually has a negative impact on the overall efficacy of the control policy. To address this challenge, this paper presents a novel Traffic Signal Control Framework (PRLight), which leverages an On-policy Traffic Control Model (OTCM) and an Online Traffic Prediction Model (OTPM) to achieve efficient and real-time control of traffic signals. The framework collects multi-source traffic information from a local-view graph in real-time and employs a novel fast attention mechanism to extract relevant traffic features. To be specific, OTCM utilizes the predicted traffic state as input, eliminating the need for communication with other agents and maximizing computational efficiency while ensuring that the most relevant information is used for signal control. The proposed framework was evaluated on both simulated and real-world road networks and compared to various state-of-the-art methods, demonstrating its effectiveness in preventing traffic congestion and accidents.

The control of traffic flows on urban roads intersections through the traffic control signals is very important, not only does the jammed traffic statement valuably relieve and the ratio of the traffic availability increase, but also the traffic accidents evidently decrease. In this paper, an adaptive algorithm of traffic control signals on the urban roads intersections is designed in , this new algorithm can actively adjust the concrete control times of the traffic control signals based on the perceiving information of the waiting vehicles in real time, then the dynamic balance between the traffic control signals and the traffic flows can be realized. Furthermore, through experiment testing and demonstrating, this adaptive algorithm expresses some fine performances, it also shows good application prospect in the field of smart city.