A Robust Traffic-Aware City-Scale Multi-Camera Vehicle Tracking Of Vehicles

Multi-Camera Multi-Target tracking (MCMT) is an essential task in intelligent transportation systems. It is highly challenging due to several problems such as heavy occlusion and appearance variance caused by various camera perspectives and congested vehicles. In this paper, we propose a practical framework for dealing with the city-scale MCMT task, consisting of four modules. The vehicles detection and ReID feature extraction are the first two modules, which locate all vehicles and extract the appearance features for all cameras. The third module is Single-Camera Multi-Target tracking (SCMT), which tracks multiple vehicles to generate candidate trajectories within each camera on the basis of the detected boxes and appearance features. The last module is Inter-Camera Association (ICA), which associates all candidate trajectories between two successive cameras using the K-reciprocal nearest neighbors algorithm, and combines all successively matched trajectories for final results. The ICA module takes the constraints of traveling time, road topology structures, and traffic rules into consideration to reduce the searching space and accelerate the matching speed. Experiments results on the public test set of 2022 AI CITY CHALLENGE Track1 demonstrate the effectiveness of our method, which achieves IDF1 of 84.86%, ranking 1st on the leaderboard.

In this paper we propose a novel set of techniques for real-time Multi-Target Multi-Camera (MTMC) tracking of vehicles in congested, low speed environments, such as those of drive-thru scenarios, where metrics such as the number of vehicles, time of stay, and interactions between vehicles and staff are needed and must be highly accurate. Traditional methods of tracking based on Intersection over Union (IoU) and basic appearance features produce fragmented trajectories of misidentifications under these conditions. Furthermore, detectors, such as YOLO (You Only Look Once) architectures, exhibit different types of errors due to vehicle proximity, lane changes, and occlusions. Our methodology introduces a new tracker algorithm, Multi-Object Tracker based on Corner Displacement (MTCD), that improves the robustness against bounding box deformations by analysing corner displacement patterns and several other factors involved. The proposed solution was validated on real-world drive-thru footage, outperforming standard IoU-based trackers like Nvidia Discriminative Correlation Filter (NvDCF) tracker. By maintaining accurate cross-camera trajectories, our framework enables the extraction of critical operational metrics, including vehicle dwell times and person–vehicle interaction patterns, which are essential for optimizing service efficiency. This study tackles persistent tracking challenges in constrained environments, showcasing practical applications for real-world surveillance and logistics systems where precision is critical. The findings underscore the benefits of incorporating geometric resilience and delayed decision-making into MTMC architectures. Furthermore, our approach offers the advantage of seamless integration with existing camera infrastructure, eliminating the need for new deployments.

Multi-Target Multi-Camera Tracking has a wide range of applications and is the basis for many advanced inferences and predictions. This paper describes our solution to the Track 3 multi-camera vehicle tracking task in 2021 AI City Challenge (AICITY21). This paper proposes a multi-target multi-camera vehicle tracking framework guided by the crossroad zones. The framework includes: (1) Use mature detection and vehicle re-identification models to extract targets and appearance features. (2) Use modified JDE-Tracker (without detection module) to track single-camera vehicles and generate single-camera tracklets. (3) According to the characteristics of the crossroad, the Tracklet Filter Strategy and the Direction Based Temporal Mask are proposed. (4) Propose Sub-clustering in Adjacent Cameras for multi-camera tracklets matching. Through the above techniques, our method obtained an IDF1 score of 0.8095, ranking first on the leaderboard <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> . The code will be released later.

Multi-target multi-camera tracking (MTMCT) is an important application in intelligent transportation systems (ITS). The conventional works follow the tracking-by-detection scheme and use the information of the object image separately while matching the object from different cameras. As a result, the association information from the object image is lost. To utilize this information, we propose an efficient MTMCT application that builds features in the form of a graph and customizes graph similarity to match the vehicle objects from different cameras. We present algorithms for both the online scenario, where only the past images are used to match a vehicle object, and the offline scenario, where a given vehicle object is tracked with past and future images. For offline scenarios, our method achieves an IDF1-score of 0.8166 on the Cityflow dataset, which contains the actual scenes of the city from multiple street cameras. For online scenarios, our method achieves an IDF1-score of 0.75 with an FPS of 14. Our codes and datasets are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/elituan/GraphBasedTracklet_MTMCT</uri> .

Vehicle reidentification is the process of reidentifying or tracking vehicles from one point on the roadway to the next. By performing vehicle reidentification, important traffic parameters including travel time, section density and partial dynamic origin/destination demands can be obtained. This provides for anonymous tracking of vehicles from site-to-site and has the potential for improving intelligent transportation systems (ITS) by providing more accurate data. The paper presents a fusion based vehicle reidentification algorithm that uses four different features to achieve high accuracy: (1) the wavelet transform of the inductive signature vector acquired from loop detectors; (2) vehicle velocity; (3) traversal time; (4) color information (based on images acquired from video cameras). A nearest neighbor approach classifies the features and linear feature fusion is shown to improve performance. With the fusion of four features, more than a 92% accuracy is obtained on real data collected from a parkway in California. Also, it is found that the wavelet transform improves performance and reduces the dimension of the feature vector when compared to the raw vehicle signatures.

Enormous advance has proven throughout the years in the area of traffic surveillance by the growth of intelligent traffic video surveillance system. In the current work, through the traffic videos, the traffic video surveillance automatically keyed out the vehicles like ambulance and trucks, which in turn assisted us in directing the vehicles at the time of emergency. Nevertheless, it doesn’t provide us a vital solution for the regulating the traffic. Moreover, this idea just identifies the vehicles, but it couldn’t notice the accidents expeditiously. Therefore in the proposed work, expeditious traffic video surveillance and monitoring system are presented along with dynamic traffic signal control and accident detection mechanism. Hybrid median filter has been utilized at the beginning for pre-processing of traffic videos, and to remove the noise. Hybrid support vector machine (SVM with extended Kalman filter) has been utilized to chase the vehicles. Next, the histogram of flow gradient features are drew-out to categories the vehicles. According to the traffic density and through video files, vehicles are computed, and then for emergency vehicles, the traffic signal gets switched dynamically. To realize the arrival of ambulances, the cameras have been set to catch traffic videos minimum at 500 m of the signal and deep learning neural networks has been employed. Hence dynamic signal control has been incorporated expeditiously. Likewise, multinomial logistic regression has been utilized in real-time live streaming videos, to identify the accidents correctly. The observational solution shows that the proposed intelligent traffic video surveillance system render expeditious dynamic control of traffic signals and it raises the identification of accidents correctly.

In order to ensure recognition accuracy, intelligent traffic video tracking system usually requires various types of information. Therefore, multi-features fusion becomes a good choice. In this paper, a new recognition approach for vehicle types based multi-feature fusion is proposed, which is used for vehicle tracking in a multi-camera traffic system. An improved Canny operator is presented for edge detection. SURF (Speeded Up Robust Features) is used for local feature extraction. To improve the performance of distance calculation between features, a refined method based on Hellinger kernel is put forward. A position constraint rule is applied to reduce unnecessary fake matchings. Finally, the information of vehicle types combined with LBP (Local Binary Pattern), HOG (Histogram of Oriented Gradients) is used for a multi-camera vehicle tracking platform, which adopts Hadoop to realize the parallel computing of the system. Experimental results show that the proposed approach has good performance for the platform.

When overpopulated cities face frequent crowded events like strikes, demonstrations, parades or other sorts of people gatherings, they are confronted to multiple security issues. To mitigate these issues, security forces are often involved to monitor the gatherings and to ensure the security of their participants. However, when access to technology is limited, the security forces can quickly become overwhelmed. Fortunately, more and more important smart cities are adopting the concept of intelligent surveillance systems. In these situations, intelligent surveillance systems require the most advanced techniques of crowd analysis to monitor crowd events properly. In this review, we explore various studies related to crowd analysis. Crowd analysis is commonly broken down into two major branches: crowd statistics and crowd behavior analysis. When crowd statistics determines the Level Of Service (LoS) of a crowded scene, crowd behavior analysis describes the motion patterns and the activities that are observed in a scene. One of the hottest topics of crowd analysis is anomaly detection. Although a unanimous definition of anomaly has not yet been met, each of crowd analysis subtopics can be subjected to abnormality. The purpose of our review is to find subareas, in crowd analysis, that are still unexplored or that seem to be rarely addressed through the prism of Deep Learning.

Video security monitoring has always been an important mission for safety reason. In an entire surveillance system, there are usually several cameras distributed sparsely to cover a wide range of public areas (e.g., school, shopping mall or infrastructure). Tracking person through this cameras network is challenging due to different camera perspectives, illumination changes and pose variations. Several algorithms for Multi-Target Multi-Camera tracking (MTMCT) have been proposed in offline method which has delay in getting result. Addressing the need for real-time computation of people tracks through multi camera, the paper proposes an online tracking algorithm. The contributions include (1) online real-time framework which can be used in practical application, (2) extend a single camera multi object tracking (MOT) algorithm to be suitable for multi-camera tracking and (3) use spatial-temporal information to strengthen cross camera person recall performance. The proposed algorithm has been benchmarked against the literature review of MTMC algorithms.

Real-time video image processing requires video compression techniques. Efficient Motion Estimation (ME) and Motion Compensation (MC) algorithms and their successful hardware implementation are the key to video compression. Work on developing and implementing efficient ME and MC algorithms for multi-camera systems is ongoing. The low power consumption yet high speed of Heterogeneous Reconfigurable Devices such as Field Programmable Gate Arrays (FPGA) can be suitable for implementation of real-time optic flow computation using multi-camera systems. This paper focuses on a search into the current state-of-the-art for multi-camera motion tracking. It has been found the tracking systems mainly focused on people and vehicle tracking in both indoor and outdoor conditions, tracking under occlusion, tracking for surveillance, single and multi-view tracking, etc. The review search also found use of multi-camera tracking in the medical sector, such as multi-camera tracking systems for respiratory motion tracking. Research on this field has not been explored much and this holds possibilities for further work.

Multi-target multi-camera tracking (MTMCT) plays an important role in intelligent video analysis, surveillance video retrieval, and other application scenarios. Nowadays, the deep-learning-based MTMCT has been the mainstream and has achieved fascinating improvements regarding tracking accuracy and efficiency. However, according to our investigation, the lacking of datasets focusing on real-world application scenarios limits the further improvements for current learning-based MTMCT models. Specifically, the learning-based MTMCT models training by common datasets usually cannot achieve satisfactory results in real-world application scenarios. Motivated by this, this paper presents a semi-automatic data annotation system to facilitate the real-world MTMCT dataset establishment. The proposed system first employs a deep-learning-based single-camera trajectory generation method to automatically extract trajectories from surveillance videos. Subsequently, the system provides a recommendation list in the following manual cross-camera trajectory matching process. The recommendation list is generated based on side information, including camera location, timestamp relation, and background scene. In the experimental stage, extensive results further demonstrate the efficiency of the proposed system.

Smart transportation based on big data traffic analysis is an important component of smart city. With millions of ubiquitous street cameras and intelligent analytic algorithms, public transit systems of the next generation can be safer and smarter. We participated the IEEE Smart World 2017 NVIDIA AI City Challenge which consists of two tracks of contests that serve this spirit. In the AI City Track 1 contest on visual detection, we built a competitive street object detector for vehicle and person localization and classification. In the AI City Track 2 contest on transportation applications, we developed a traffic analysis framework based on vehicle tracking that can assist the surveillance and visualization of the traffic flow. Both developed methods demonstrated practical, and competitive performance when evaluated with state-of-art methods on real-world traffic videos provided in the challenge contest.

Vehicle speed detection and violation detection is in automated way is very useful task for intelligent traffic surveillance system. Vehicle tracking in a precise way is critical task for the process of estimating the speed using only video data. This paper proposes an algorithm to estimate the speed and detecting the violation in precise way. The proposed approach gives the method which contains the optical flow algorithm with corner detection for tracking the vehicle. Proposed algorithm results the better tracking accuracy and better accuracy for speed detection. Also less computation complexity as optical flow is calculated for only corner points.

Vehicle counting is a process to estimate traffic density on roads to assess the traffic conditions for intelligent transportation systems (ITS). Real-time traffic management systems have become popular recently due to the availability of high end cameras and technology. The present traffic management systems focus on speed detection, signal jumping, zebra crossing but not on traffic density estimation. Proposed video-based vehicle counting and tracking method using a video captured on CCTV and handheld mobile cameras. The system can be used in smart cities to create smart traffic light signals, in which duration of each signal depends on real time vehicle density in a particular lane of road. Vehicle counting is performed in two steps: the captured video is sent to You Only Look Once (YOLO) based deep learning framework to detect, count and classify the vehicles. Multi vehicular tracking is adopted using Deep SORTalgorithm to track the vehicles in video frames. Model was trained for six different classes, using Google Colaboratory. Datasets of vehicles specifically pertaining to Indian roads environment is considered for implementation. The performance of the model was analyzed, proposed model has tested and obtained an average counting accuracy of 86.56% while the average precision is 93.85%. The model can be implemented for ascertaining the traffic density on roads and this provides knowledge for infrastructural development to authorities. It can also be an integral part of smart city projects to develop intelligent and smart traffic surveillance system.

Intelligent Transportation Systems (ITS) are increasingly shifting toward data-driven and artificial intelligence-based frameworks that integrate sensing, prediction, and control to enhance mobility efficiency, safety, and adaptability. This dissertation contributes to the development of next-generation ITS through three complementary studies focusing on flexible traffic monitoring, real-time traffic forecasting, and stability-aware vehicle control. The first study develops a multi-drone framework for wide-view corridor traffic analysis and origin-destination (OD) estimation. By coordinating multiple unmanned aerial vehicles, as demonstrated with two drones in this study, the framework enables continuous vehicle tracking across extended corridors. A deep learning-based pipeline combining vehicle detection and tracking automatically reconstructs vehicle trajectories and matches them across drone views. Field experiments conducted at a freeway weaving section achieved over 91% accuracy in OD flow estimation compared with ground-truth data, demonstrating the framework’s effectiveness for scalable and high-resolution traffic monitoring. The second study proposes a day-specific spatial-temporal graph convolutional network (Day-STGCN) for real-time traffic forecasting. Building upon the classical STGCN architecture, the model integrates rolling historical time-dependent features with day-type categorization to capture both short-term variations and long-term recurring patterns. Using real-world sensor data, the model significantly improves prediction accuracy over baseline graph neural networks, providing robust day-aware forecasting that reflects realistic daily traffic dynamics. The third study investigates cooperative adaptive cruise control (CACC) and its effects on traffic string stability under varying market penetration rates of connected and automated vehicles (CAVs). Analytical derivations establish explicit stability conditions linking controller parameters, time headway, and control intervals in both homogeneous and heterogeneous traffic flows. The results reveal that appropriately designed CACC controllers enhance string stability even in mixed environments with different types of vehicles. Collectively, these studies present an integrated framework that advances ITS through data-driven sensing, learning, and control, offering theoretical and practical foundations for more intelligent, responsive, and stable transportation systems.