Real-Time Passenger Detection and Counting Using Edge AI Camera

Due to limited energy resources in wireless multimedia sensor networks WMSNs, there is a need to perform data reduction and elimination over raw video data at the camera sensors before transmission. Nonetheless, this data reduction and elimination may create imprecision and uncertainty in the data, reducing the quality of decision making. In this paper, we propose a reactive mechanism for not only fusing uncertain data at the sink but also for automated processing of data using active rules, extending the classical event-condition-action structure. In this mechanism, data fusion is performed using fuzzy logic to handle uncertainty in the received data. The fused data is then processed to infer certain predefined actions. These actions are triggered based on both the predefined event definitions and temporal and spatial data about the detected objects. Through experimentation, the proposed mechanism is shown to provide good accuracies while making significant energy savings at the cameras.

Public transportations such as trains, commuter, buses, and taxis have become main transportation for major citizen in Malaysia. In recent years, Malaysia’s government has improving a modern public transportation with a big cost such as MRT project and ECRL. In order to improve the public transportation system, the management must propose proper schedule as well as updating the number of passenger counts. In general, the counting process is conducted by manual traditional headcount system which the driver or bus conductor will manually count their passenger. Hence, this process is time consuming. Therefore, this research proposes an automatic passengers counting system using images processing based on YCbCr and HSV colour model analysis. To obtain the passengers counting result, there are several image processing steps to be implemented; (1) data collection by capturing 50 passenger images, (2) colour conversion from RGB to YCbCr and HSV colour spaces, (3) image segmentation using Otsu thresholding method, (4) removing of unwanted object and, (5) passenger counting process. This study will analyze two different colour conversions which are from RGB to HSV, as well as from RGB to YCbCr colour conversion. Overall, segmentation using HSV colour model has proven to be the best in segmenting the passenger’s image with segmentation accuracy of 85.63%. Overall, this project is expected to help buses administration to improve their services by updating their passengers counting automatically.

Target detection and tracking is a key technology in intelligent monitoring system. However, it is still difficult to design a robust, accurate and real-time target detection and tracking algorithm. Taking target Tracking as the research focus, this paper focuses on one of the current mainstream algorithms, namely, TLD (tracking-learning-detection) target Tracking algorithm. Based on the theory of TLD target tracking and corner detection, a new target tracking method was proposed to solve the existing problems. A classifier is built online to track and learn the target in real time, and a target contour detector is added in the system to complement the random forest detector to solve the problem of tracking failure. Experimental results show that the proposed algorithm can effectively overcome the influence of fast change or depth occlusion, and improve the tracking efficiency of moving targets.

In visual tracking, developing an efficient appearance model is a challenging task due to the influence of various factors, such as illumination variation, occlusion, background clutter, and so on. Existing tracking algorithms use appearance samples from previous frames to form a template set upon which target appearance models are built. However, these appearance models are data-dependent, so they may be corrupted by significant appearance variation. It is difficult to update the templates in challenging environments. In this paper, we propose a robust visual tracking algorithm with an adaptive appearance model using a point-to-set metric learning technique. To do this, we first model a target representation using a set of target templates and a regularized affine hull (RAH) spanned by the target templates. Then, we learn a point-to-set distance metric, which is incorporated into the optimization process to obtain an adaptive target representation. The RAH model covers unseen target appearances by affine combinations of the target templates. Based on the proposed target appearance model, we design an effective template update scheme by adjusting the weights of the target templates. Experimental results on challenging video sequences with comparisons to several state-of-the-art tracking algorithms demonstrate the effectiveness and robustness of the proposed tracking algorithm.

There is tremendous growth in the amount of digital video creation and communication with the proliferation of mobile consumer electronic devices such as cellular phones with video capture and display capabilities. This trend will continue with the demand for higher spatial and temporal video resolutions accompanied by 3D and multiview video. Since it is not practical to store or transmit the raw digital video data, it is essential to reduce it with digital video compression. To cope with the vast amount of raw digital video data, digital video compression standards use increasingly more complex algorithms. These algorithms achieve higher compression at the same quality at the expense of significant computational complexity. Therefore, it is essential to design low power digital video compression hardware especially for mobile devices. In this talk, I will present two sets of power reduction techniques for digital video compression hardware. The ones in the first set reduce the power consumption with no impact on the video quality. The ones in the second set trade off video quality for power consumption which also enables adjusting video quality based on available energy in the battery of a mobile device for graceful degradation. I will show the impact of these techniques on the power consumption of digital video compression hardware implementing H.264/AVC and H.265/HEVC standards.

Urban transport planning and the integration of various mobility options have become increasingly complex, necessitating a thorough understanding of user mobility patterns and their diverse needs. This paper focuses on benchmarking different Automatic Passenger Counting (APC) technologies, which play a key role in Mobility as a Service (MaaS) systems. APC systems provide valuable data for analysing mobility patterns and informing decisions about resource allocation. Our study presents a comprehensive data collection and benchmark analysis of APC solutions. The literature review emphasises the significance of passenger counting for transport companies and discusses various existing APC technologies, such as pressure sensors, wireless sensors, optical infrared sensors (IR), and video image technology. Real-world applications of APC systems are examined, highlighting experimental results and their potential for improving accuracy. The methodology outlines the data collection process, which involved identifying APC companies, conducting interviews with companies and customers, and administering an ad hoc survey to gather specific information about APC systems. The collected data were used to establish criteria and key performance indicators (KPIs) for the benchmarking analysis. The benchmarking analysis compares APC devices and companies based on ten criteria: technology, accuracy, environment, coverage, interface, interference, robustness (for devices), price, pricing model, and system integration (for companies). KPIs were developed to measure performance and make comparison easier. The results of the benchmarking analysis offer insights into the costs and accuracy of different APC systems, enabling informed decision making regarding system selection and implementation. The findings fill a research gap and provide valuable information for transport companies and policy makers, and we offer a comprehensive analysis of APC systems, highlighting their strengths, weaknesses, and business strategies. The paper concludes by discussing limitations and suggesting future research directions for APC technologies.

Visual tracking has been widely used in robot systems, and numerous approaches for visual tracking have been proposed. However, developing a robust and real-time visual tracking algorithm which can adaptively track the varying appearance of target under challenging conditions for mobile robot is still an open problem. This paper presents an adaptive probabilistic tracking algorithm with multiple cues integration. An effective evaluation function is proposed to evaluate each cue used for tracking based on their discriminating abilities between foreground and background. Then the likelihood functions of the cues are integrated in particle filter framework with different weights determined based on the evaluation scores. A novel target model updating strategy is proposed to adapt to the varying appearance of target resisting gradual drift which is still an unsolved problem in many adaptive tracking algorithms. Experimental results on a mobile robot demonstrate the robust performance of the proposed algorithm under challenging conditions.

Object tracking in real-world scenarios is often hampered by the simultaneous challenges of low light and partial object occlusion. While existing evaluation datasets have tackled these scenarios separately-focusing either on low-light settings or amodal perception-their co-occurrence has rarely been studied. To bridge this gap, we introduce LAOT (Low-Light Amodal Object Tracking), a benchmark tailored to assess how state-of-the-art tracking algorithms perform under the combined challenges of dim lighting and partial object occlusion. The dataset consists of 201 diverse video sequences with over 16K frames, each meticulously annotated with both modal bounding boxes (representing the visible part of the object) and amodal bounding boxes (estimating the complete object extent, including occluded regions). To characterize occlusion severity, object instances are categorized into three levels based on visible proportion: 0-20% (no occlusion), 20-80% (partial occlusion), and 80-100% (heavy occlusion). We benchmark 21 state-of-the-art tracking algorithms, including ARTrack [1], AVTrack [2], DropTrack [3], SeqTrack [4], ProContEXT [5], and HIPTrack [6], using Average Precision (AP) as the evaluation metric across occlusion levels. The results reveal a consistent decline in tracking accuracy as occlusion increases. For example, DropTrack achieves 0.8382 AP without occlusion but drops to 0.1730 AP with heavy occlusion; similarly, AVTrack falls from 0.7375 to 0.1276. These findings expose the limitations of current methods in coping with simultaneous challenges in low light and amodal perception. LAOT serves as a comprehensive and diagnostic benchmark to guide the development of robust perception-aware tracking algorithms suited for visually degraded environments.The LAOT is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/LSW-CVLab/LAOT</uri>

A robust model-based lane detection and tracking algorithm has been studied. Initially, Dynamic programming (DP) and Hough transform are employed to detect lane boundaries. The Dynamic programming is known to be a powerful algorithm for optimal path finding on a cost field. Utilizing the intensities of edge image as costs of the DP, most likely road boundaries are obtained in this paper. An improved Hough Transform is applied to detect practically useful road boundaries with straight line segments. In the following stage, based on the prior information of lane boundaries in the detection procedure, the road boundary region has been obtained. A linear lane model has been employed with an energy function in tracking algorithm. Experiment results in inter-frame processing describe the proposed method can fit for real condition on expressway.

The robust marker tracking and relative navigation algorithms are presented for precise UAV vision-based autonomous landing. To recognize the marker in close-range, the concentric circles are adopted as the marker with ellipse fitting algorithm based on Direct Least Square. We assume that IMU provides vehicle's attitude and altitude so that we consider GPS-denied situation. Also multiple ellipses are used to estimate its center pixel coordinate makes UAV land more accurately. To verify the vision-based relative navigation algorithm we suggest, numerical simulations are obtained by using virtual reality toolbox in MATLAB™. We predetermine the true position and attitude of UAV, and the result of the relative position calculated from vision software including the filter is compared. The simulation results show that the algorithm is robust and very accurate.

This paper proposes a robust face tracking algorithm based on the CONDENSATION algorithm that uses skin color and facial shape as observation measures. Two independent trackers are used for robust tracking: one is tracking for the skin colored region and another is tracking for the facial shape region. The two trackers are coupled by using an importance sampling technique, where the skin color density obtained from the skin color tracker is used as the importance function to generate samples for the shape tracker. The samples of the skin color tracker within the chosen shape region are updated with higher weights. The proposed face tracker shows a robust tracking performance over the skin color based tracker or the facial shape based tracker given the presence of clutter background and/or illumination changes.KeywordsSkin ColorColor HistogramFacial ShapeClutter BackgroundFace TrackingThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Inspired by the recent advances in sparse theory, we propose a new sparse based object tracking framework in this paper. In order to effectively detect the target object to be tracked, we formulate and present a new sparse based representative dictionary learning algorithm and candidate selection scheme for robust object tracking. Specifically, we have used Pearson's correlation based dictionary learning algorithm to engender a highly representative dictionary. This representative dictionary is then used for modeling an adaptive appearance model for the target, which is robust to challenges like partial occlusions, illumination, clutter scenes and pose changes. We also present a new Appearance Model Structural Similarity test algorithm for meticulously identifying the target from various candidates in the consecutive frames of video sequences. Results of extensive experimentations carried out using various publicly available datasets shows that our framework produces an exorbitant performance compared to many state-of-the-art works.

The Constrained Admissible Region-Multiple Hypothesis Filter (CAR-MHF) has previously been successfully applied to the problem of determining initial state and parameter estimates for non-operational space objects in the near-geosynchronous Earth orbit (GEO) regime. The application of CAR-MHF to a dense population of synthetically-created uncorrelated tracks (UCT) has been primarily challenged by two characteristics of the current implementation. First, the rapid uncertainty growth due to the noise characteristics of processing short-arc optical tracklets leads to ambiguous uncertainty overlap and the prevalence of mis-associated observations, primarily with a large statistical association gate. Second, the reliance on measurement update decisions made on a frame-by-frame basis without a process that evaluates the track estimate based upon all available associated data has prevented additional data from resolving these ambiguous conditions (i.e. if data from another object is statistically valid at a given instant of large ambiguity, an update will be performed and usually drives the filter to diverge). The work addressed in this paper addresses these challenges by harmoniously applying joint probabilistic data association in concert with backward smoothing. This is coupled with the McReynolds Filter/Smoother consistency checks to further minimize mis-associations. These methodologies are implemented to reduce the impacts of early ambiguity by sharing information across track updates and delaying hard decisions on track formation until sufficient data have been processed. Through filter/smoother assessment against both sparse and dense synthetically-created GEO optical data, the utility of mutually applying these two distinct techniques within the construct of the CAR-MHF algorithm for robust multiple space object tracking is be demonstrated.

This paper presents a robust algorithm for automatic tracking of feature points on the human heart. The emphases and key contributions of the proposed algorithm are uniform distribution of the feature points and sustained tolerable tracking error. While in many methods in the literature, detection takes place independently from the tracking procedure, adopting a different approach, we selected a data-driven detection stage, which works based on the feedback from tracking results from the Lucas–Kanade (LK) tracking algorithm to avoid unacceptable tracking errors. To ensure a uniform spatial distribution of the total detected feature points for tracking, a cost function is employed using the simulated annealing optimizer, which prevents the newly detected points from accumulating near the previously located points or stagnant regions. Implementing the proposed algorithm on a real human heart dataset showed that the presented algorithm yields more robust tracking and improved motion reconstruction, compared with the other available methods. Furthermore, to predict the motion of feature points for handling short-term occlusions, a state space model is utilized, and thin-plate spline (TPS) interpolation was also employed to estimate motion of any arbitrary point on the heart surface.

Many organisations face challenges in accurately and efficiently identifying human activities using manually operated CCTV cameras and small datasets, particularly in densely populated and dynamic environments. This paper presents a real‐time human activity detection and recognition system that leverages You Only Look Once version 8 (YOLOv8) for object detection and integrates the deep simple online real‐time tracking algorithm for associate frames, robust tracking and individual identity preservation. Evaluated on a combination of public and experimental datasets containing over 10,000 frames, the system achieves notable enhancements in efficiency along with accuracy. Through data preprocessing and algorithm optimisation, the Adam optimiser outperformed stochastic gradient descent (SGD) by more than 5% in accuracy. K ‐fold cross‐validation was applied to reduce overfitting, while anchor boxes of varying scales and aspect ratios improved the detection of objects of different sizes. The aspect ratio, which is used to mitigate the effect of object size and control edge‐orientation issues by computing the ratio of width to height of the bounding boxes in YOLOv8, plays a crucial role in object detection.YOLOv8l, combined with DeepSORT, delivers state‐of‐the‐art results, achieving a mean average precision of 96.1% in mixed datasets. The system assigns unique IDs to track individuals, enabling precise frame association during motion in crowded areas, which is crucial for maintaining accuracy in complex environments. This study presents a robust and scalable model, offering a significant improvement for public surveillance technology.