AI-based Surveillance for Exam Integrity: Real-Time Detection of Abnormal Student Behavior

As society develops, the demand for security is rapidly increasing. Accordingly, there is growing interest in research on methods to detect and prevent abnormal behavior using surveillance cameras in public places and private spaces such as shopping malls and airports for human safety. Many detection techniques based on deep learning models have been researched in the field of abnormal behavior detection. However, due to the lack of labeled abnormal behavior data, there are significant difficulties in developing an effective detection system. This paper surveys methods for deep learning methods to detect abnormal human behavior in surveillance video and presents recent techniques. First, I will introduce popular datasets that have often been used in previous research. After that, we categorized the existing methods for detecting abnormal behavior using deep learning into three types: supervised learning, unsupervised learning, and partially-supervised learning. We then explained the basic concepts and advantages of each method and summarized their shortcomings. We also briefly describe future research directions based on the advantages and disadvantages of each method. Based on this, it is expected that the technology of video surveillance systems that apply abnormal behavior detection will further develop.

Abnormal crowd behavior detection has recently attracted increasing attention due to its wide applications in computer vision research areas. However, it is still an extremely challenging task due to the great variability of abnormal behavior coupled with huge ambiguity and uncertainty of video contents. To tackle these challenges, we propose a new probabilistic framework named variational abnormal behavior detection (VABD), which can detect abnormal crowd behavior in video sequences. We make three major contributions: (1) We develop a new probabilistic latent variable model that combines the strengths of the U-Net and conditional variational auto-encoder, which also are the backbone of our model; (2) We propose a motion loss based on an optical flow network to impose the motion consistency of generated video frames and input video frames; (3) We embed a Wasserstein generative adversarial network at the end of the backbone network to enhance the framework performance. VABD can accurately discriminate abnormal video frames from video sequences. Experimental results on UCSD, CUHK Avenue, IITB-Corridor, and ShanghaiTech datasets show that VABD outperforms the state-of-the-art algorithms on abnormal crowd behavior detection. Without data augmentation, our VABD achieves 72.24% in terms of AUC on IITB-Corridor, which surpasses the state-of-the-art methods by nearly 5%.

Abnormal behaviour detection is crucial given the reliance on video surveillance for public safety. However, most vision-based methods fail to adapt to the variability and complexity of the crowd scenes inherent in real-world scenarios. This study introduced an abnormal crowd behaviour detection algorithm based on lightweight Graphic Neural Networks (GNNs) using trajectory similarities and spatial-temporal attributes. Integrating Graph Attention Networks (GATs) to encode the graph structure, creating latent embeddings that capture both local and global relationships within crowds. Also introducing an autoencoder (AE) structure to identify anomalies by comparing true movement with the decoded output. Anomaly is detected when the reconstruction error is high. Experiments on UCSD Pedestrians and ShanghaiTech campus datasets show the successful use of trajectory similarities as graph features for GNNs to generate logical embeddings of crowd movement and highlight the impact of time windows on anomaly detection precision as reflected in F1-scores. The Area Under the Curve (AUC) metric shows stability, highlighting the model’s balanced performance. Qualitative analysis of the ShanghaiTech campus dataset illustrates the model’s ability to align detection with visual anomalies. Overall, this study demonstrates the effectiveness of the proposed graph-based anomaly detection framework for crowd monitoring through a lightweight model suitable for CCTV.

A novel automatic detection method for abnormal behavior of single fish using image fusion

We propose a method to detect abnormal crowd behavior using integrated multiple behavior models. Traditional abnormal detection methods are based only on personal behavior models. However, a single behavior model cannot accurately reflect complex crowd behavior. To solve this problem, we introduce an integrated multiple behavior model for accurate abnormal behavior detection in a complex crowd scene. We use not only the personal behavior model, but also multiple social behavior models. The experimental results show that our proposed method efficiently detects the abnormal behavior in a crowded scene.

Amid rapid advancements in artificial intelligence, the detection of abnormal human behaviors in complex traffic environments has garnered significant attention. However, detection errors frequently occur due to interference from complex backgrounds, small targets, and other factors. Therefore, this paper proposes a research methodology that integrates the anomaly detection YOLO-SGCF algorithm with the tracking BoT-SORT-ReID algorithm. The detection module uses YOLOv8 as the baseline model, incorporating Swin Transformer to enhance global feature modeling capabilities in complex scenes. CBAM and CA attention are embedded into the Neck and backbone, respectively: CBAM enables dual-dimensional channel-spatial weighting, while CA precisely captures object location features by encoding coordinate information. The Neck layer incorporates GSConv convolutional modules to reduce computational load while expanding feature receptive fields. The loss function is replaced with Focal-EIoU to address sample imbalance issues and precisely optimize bounding box regression. For tracking, to enhance long-term tracking stability, ReID feature distances are incorporated during the BoT-SORT data association phase. This integrates behavioral category information from YOLO-SGCF, enabling the identification and tracking of abnormal pedestrian behaviors in complex environments. Evaluations on our self-built dataset (covering four abnormal behaviors: Climb, Fall, Fight, Phone) show mAP@50%, precision, and recall reaching 92.2%, 90.75%, and 86.57% respectively-improvements of 3.4%, 4.4%, and 6% over the original model-while maintaining an inference speed of 328.49 FPS. Additionally, generalization testing on the UCSD Ped1 dataset (covering six abnormal behaviors: Biker, Skater, Car, Wheelchair, Lawn, Runner) yielded an mAP score of 92.7%, representing a 1.5% improvement over the original model and outperforming existing mainstream models. Furthermore, the tracking algorithm achieved an MOTA of 90.8% and an MOTP of 92.6%, with a 47.6% reduction in IDS, demonstrating superior tracking performance compared to existing mainstream algorithms.

Artificial features are adopted in most of the existing abnormal behavior detection. However, it is difficult to choose and design an effective behavior feature for the reason of highly computational complexity and complex scenarios. To solve this problem, temporal consistency based weak supervised abnormal behavior detection method is proposed in this paper. First, temporal gram matrices are constructed for a given video sequence. Then, a pair of behavior units (Candidate action fragment) are formed by exploiting the temporal consistency and the smoothness of human behavior, which helps to locate the start frame and end frame of the related class of abnormal behavior in the video sequence to train the corresponding classifier. Finally, sparse reconstruction is here utilized to detect abnormal behavior. Experiments conducted on common database including CAVIAR and Crossing demonstrate the effectiveness of the proposed method.

The dangerous and harmful factors of machinery manufacturing mainly include human unsafe behavior and consciousness factors, unsafe state factors of equipment and adverse environmental factors. The characteristics and classification of abnormal behavior are analyzed, the detection methods of operators' abnormal running behavior are compared, and the discrimination basis of operators' abnormal running behavior is analyzed. The wearable inertial sensor is studied. The acceleration sensor is used to measure the acceleration of the measured object, and the gyroscope sensor can measure the angle of the human body relative to the ground. MPU9250 is selected as the measurement tool of acceleration and angle to identify the abnormal behavior of the human body, and cooperate with the single chip microcomputer, It is conducive to the detection and early warning of abnormal behavior of mechanical operators.

In this paper; we propose a novel crowd behavior representation method to detect abnormal behaviors in videos. An adaptive optical flow filtering method is proposed to utilize low-level optical flow informations. Furthermore, a simple framework is developed to detect and to localize abnormal crowd behavior using adaptive optical flow filtering result. The proposed method is more robust than other modeling methods in representing different behaviors. In this model, a normal behavior is presented by the general value. Some outliers in the temporal domain or spatial domain are presented by a higher value. Spatio-temporal cuboids are extracted from the filtering result to present the likelihood of anomaly in the frame. Experimental evaluations are performed on two public datasets with comparison to the provisos abnormal behavior detection methods in the literature. Experimental results show that the proposed methods outperform previous abnormal behavior detection techniques in the literature.

ABSTRACTThe detection of abnormal behaviours with fast and automatic recognising is significant in crowded areas to provide higher security to the public. The adoption of deep learning and machine learning‐based abnormal behaviour detection models enhances the influential detection and real‐time security monitoring in crowds. The researchers have remotely evaluated the heart rate based on physiological information to detect abnormal activities in various years. Over the past few years, several progress have been made, and there are still some issues concerning processing time, accuracy, and computational complexity. The developed approaches detects the activities of anomalies like traffic rule violations, riots, fighting, and stampede, in addition, several anomalous entities such as abandoned luggage and weapons at the sensitive place automatically in time. However, the identification of video anomalies methods poses several challenges because of various environmental conditions, the ambiguous nature of the anomaly, lack of proper datasets, and the complex nature of human characteristics. In recent days, there have been only a few devoted surveys associated with deep learning related video anomaly identification as the research domain is in its initial stages. In this review work, the abnormal behaviour analysis models using deep learning are reviewed in depth in for security applications. Based on the traditionally used abnormal behaviour analysis models in crowded scenes, we widely categorised the methods into classification using object tracking, classification using handcrafted extracted features, classification using non‐contact heart rate variability and blood pressure, analysing motion patterns from the visual frames, and classification using face images. We also discuss the comparative analysis of the previous methods with respect to datasets, computational infrastructure, and performance measures for both qualitative and quantitative analysis. In addition, the open and trending research challenges are analysed for future research.

Video abnormal behavior detection is an important research direction in the field of computer vision and has been widely used in surveillance video. This paper proposes an abnormal behavior detection model based on human skeletal structure and recurrent neural network, which uses dynamic skeletal features for abnormal behavior detection. The model in this paper extracts key points from multiple frames through the human pose estimation algorithm, obtains human skeleton information, and inputs it into the established autoencoder recurrent neural network to detect abnormal human behavior from video sequences. Compared with traditional appearance-based models, our method has better anomaly detection performance under multi-scene and multi-view.KeywordsAbnormal behavior detectionGRUAutoencoderSkeletal information

This paper proposes a new algorithm for abnormal behavior detection in surveillance video. Firstly, the motion information image of each frame is constructed by calculating the optical flow size and the angle difference between the optical flow vectors between consecutive frames, and then the obtained motion image information is input into the convolutional neural network (CNN) for training, and used for video abnormal behavior detection. In the algorithm, the motion information image generated based on optical flow information can provide the motion information features in the video image more accurately, which makes it easier to distinguish the normal behavior and abnormal behavior of the video. The experiment of this algorithm is carried out on the commonly used data set PETS 2009. Experimental results show that the proposed method and other algorithms have a significant improvement in the accuracy of abnormal behavior detection.

Abnormal behavior detection in surveillance videos based on deep learning has becomea hot topic in current research. However, due to the complexity and variability of crowd movement and external environment, detecting abnormal behavior faces great challenges. Current abnormal behavior recognition models have limitations in feature extraction and pay insufficient attention to dynamic temporal features. To address these problems, a spatiotemporal enhancement anomaly detection method based on multi-channel coupling is proposed. Based on the SlowFast network, a multi-channel coupled temporal enhancement module and a spatial enhancement module are introduced respectively to extract more discriminative static and dynamic feature information. A large number of experiments are carried out on three benchmark datasets (Violent Flow, Hockey Fight, and Real-life Violence Situations). The results show that the prediction accuracy of the proposed abnormal behavior recognition method reaches 95.3%、97.3%and respectively 94%, thus verifying the effectiveness of the method.

The increase of emergency situations that cause mass panic in mass gatherings, such as terrorist attacks, random shooting, stampede, and fires, sheds light on the fact that advancements in technology should contribute in timely detecting and reporting serious crowd abnormal behaviour. The new paradigm of the ‘Internet of Things’ (IoT) can contribute to that. In this study, a method for real-time detection of abnormal crowd behaviour in mass gatherings is proposed. This system is based on advanced wireless connections, wearable sensors and machine learning technologies. It is a new crowdsourcing approach that considers humans themselves as the surveillance devices that exist everywhere. A sufficient number of the event’s attendees are supposed to wear an electronic wristband which contains a heart rate sensor, motion sensors and an assisted-GPS, and has a wireless connection. It detects the abnormal behaviour by detecting heart rate increase and abnormal motion. Due to the unavailability of public bio-dataset on mass panic, dataset of this study was collected from 89 subjects wearing the above-mentioned wristband and generating 1054 data samples. Two types of data collected were: firstly, the data of normal daily activities and secondly, the data of abnormal activities resembling the behaviour of escape panic. Moreover, another abnormal dataset was synthetically generated to simulate panic with limited motion. In our proposed approach, two-phases of data analysis are done. Phase-I is a deep machine learning model that was used to analyze the sensors’ collected readings of the wristband and detect if the person has indeed panicked in order to send alerting signals. While phase-II data analysis takes place in the monitoring server that receives the alerting signals to conclude if it is a mass panic incident or a false positive case. Our experiments demonstrate that the proposed system can offer a reliable, accurate, and fast solution for panic detection. This experiment uses the Hajj pilgrimage as a case study.

The installation of surveillance systems in public or private settings is increasingly integrated with intelligent and real-time video anomaly detection to proactive threat consideration. A framework is proposed for real-time abnormal behavior detection for streaming video data based on Dynamic Graph Neural Networks (DGNNs). Unlike traditional frame-based models, the DGNN is not restricted to a static set of nodes; instead, it treats each frame as a graph, where nodes denote detected physical objects such as people and vehicles. DGNN introduces real-time frame-wise graph updates unlike prior methods with static or slow graph refresh using motion, pose, and interaction cues, supported by a lightweight spatio-temporal GCN optimized for live anomaly detection. The results convey high performance for the model, which recorded an accuracy of 99.65%, along with the corresponding precision, recall, and F1-score. An FNR and FPR of only 0.35% and 0.03%, respectively, highlighted its strong performance in discriminating normal from anomalous behaviors. The ROC and precision-recall AUC scores zoomed close to 1.0000 in almost all categories, thereby attesting to the system’s robustness where even slight stakes are involved. Following this was the fusion of TensorRT optimization in ensuring the real-time processing capability of the model, with an inference latency of fewer than 200 milliseconds per segment. The DGNN system, therefore, represents a scalable, low-latency option that is capable of real-time detection and classification of diverse and complex real-world abnormal behaviors, making it a novel approach toward smart public safety systems with very little dependency on huge annotated datasets.