Pedestrian detection in video surveillance using fully convolutional YOLO neural network

Event detection in crowded surveillance videos is a challenging yet important problem. In this paper, we present our eSur (Event detection system on SURveillance video) system, which is derived from TRECVid'12 surveillance tasks. Currently, eSur attempts to detect two categories of events: 1) pair-wise events (e.g., PeopleMeet, PeopleSplitUp and Embrace); 2) action-like events (e.g., ObjectPut, CellToEar, PersonRuns and Pointing). In eSur system, we first employ people detection and tracking algorithms to locate target persons in 3D space-time domain. Then the video sequences in which target persons occur are partitioned into several spatio-temporal cubes. Visual features (i.e. cubic feature and MoSIFT) are computed over these cubes. After that, a sequence learning method, (namely SVM with dynamic time alignment kernel), is employed to infer the existence of an event for the video sequence. According to the TRECVid SED formal evaluation, eSur has yielded fairly encouraging results on TRECVid'12 dataset.

This paper proposes a new algorithm for abnormal behavior detection in surveillance video. Firstly, the motion information image of each frame is constructed by calculating the optical flow size and the angle difference between the optical flow vectors between consecutive frames, and then the obtained motion image information is input into the convolutional neural network (CNN) for training, and used for video abnormal behavior detection. In the algorithm, the motion information image generated based on optical flow information can provide the motion information features in the video image more accurately, which makes it easier to distinguish the normal behavior and abnormal behavior of the video. The experiment of this algorithm is carried out on the commonly used data set PETS 2009. Experimental results show that the proposed method and other algorithms have a significant improvement in the accuracy of abnormal behavior detection.

The combination of traditional methods (e.g., ACF) and Convolutional Neural Networks (CNNs) has achieved great success in pedestrian detection. Despite effectiveness, design of this method is intricate. In this paper, we present an end-to-end network based on Faster R-CNN and neural cascade classifier for pedestrian detection. Different from Faster R-CNN that only makes use of the last convolutional layer, we utilize features from multiple layers and feed them to a neural cascade classifier. Such an architecture favors more low-level features and implements a hard negative mining process in the network. Both of these two factors are important in pedestrian detection. The neural cascade classifier is jointly trained with the Faster R-CNN in our unifying network. The proposed network achieves comparable performance to the state-of-the-art on Caltech pedestrian dataset with a more concise framework and faster processing speed. Meanwhile, the detection result obtained by our method is tighter and more accurate.

Brightness guided preprocessing for automatic cold steel weapon detection in surveillance videos with deep learning

The point source target (PST) can provide high object and image positioning accuracy and is expected to play an important role in the precise geometric processing of optical remote sensing sensors in the future. This paper proposes a method for intelligently recognizing PST image control points (ICPs) from satellite imagery, which can improve the intelligent level of geometric processing of optical remote sensing sensors. Two deep convolutional neural networks (DCNNs), Faster R-CNN and CenterNet are selected to complete the recognition task. Due to the lack of training data, a large number of simulated samples are generated considering the PST image characteristics. The simulated and real PST ICPs are then used to test the trained DCNNs. The two DCNNs complete the recognition task on the simulated dataset successfully. The Recall and Precision values of the two DCNNs are close to 100%. The performance of the two DCNNs on real PST ICPs is worse than that on the simulated data, but the recognition task is also well completed when the quality of PST ICPs is good. The Recall values of both models are above 95%, and the Precision values are close to 100%. Experiment results also show that the performance of CenterNet is better than Faster R-CNN and the image quality has a great impact on the recognition performance.KeywordsPoint source target (PST)Image control point (ICP)Deep convolutional neural networks (DCNNs)Faster R-CNNCenterNet

This paper presents a novel hybrid approach for real-time anomaly detection in video surveillance systems by integrating YOLOv8 object detection with advanced motion-based analysis techniques. The proposed system addresses critical limitations of existing single-modality detection methods through innovative fusion of deep learning and temporal analysis. The architecture incorporates parallel processing pipelines for YOLOv8 detection and optical flow computation, combined with an isolation forest-based anomaly decision framework that leverages historical detection patterns. Experimental evalution on a custom dataset of 5,000 surveillance video clips demonstrates superior performance with 92.3% accuracy, 89.7% precision, 94.1% recall, and 91.8% F1-score, while maintaining real-time processing at 30 FPS. The system significantly outperforms traditional approaches with 15.2% accuracy improvement over YOLO-only methods and 18.7% improvement over motion-only techniques. The proposed hybrid framework provides robust anomaly detection capabilities suitable for practical deployment in security-critical surveillance applications with reduced false positive rates and enhanced temporat consistence.

Posture detection targeted towards providing assessments for the monitoring of health and welfare of pigs has been of great interest to researchers from different disciplines. Existing studies applying machine vision techniques are mostly based on methods using three-dimensional imaging systems, or two-dimensional systems with the limitation of monitoring under controlled conditions. Thus, the main goal of this study was to determine whether a two-dimensional imaging system, along with deep learning approaches, could be utilized to detect the standing and lying (belly and side) postures of pigs under commercial farm conditions. Three deep learning-based detector methods, including faster regions with convolutional neural network features (Faster R-CNN), single shot multibox detector (SSD) and region-based fully convolutional network (R-FCN), combined with Inception V2, Residual Network (ResNet) and Inception ResNet V2 feature extractions of RGB images were proposed. Data from different commercial farms were used for training and validation of the proposed models. The experimental results demonstrated that the R-FCN ResNet101 method was able to detect lying and standing postures with higher average precision (AP) of 0.93, 0.95 and 0.92 for standing, lying on side and lying on belly postures, respectively and mean average precision (mAP) of more than 0.93.

This composition bargains with the Tomato leaf infection discovery and classification utilizing different strategies like Convolutional Neural Network (CNN), Regions with CNN (R-CNN), Fast R-CNN and Faster R-CNN. The main issue in the agricultural sector is leaf diseases, which have an impact on crop yield and financial gain. Early detection of leaf diseases in plants is essential to prevent losses to the agricultural sector. There are several tomato leaf diseases that affect the crop's leaves, including the Mosaic virus, Early Blight, Healthy, Septoria leaf, and Bacterial spot. Using deep learning algorithms and image processing methods, we can identify the diseases in tomato leaves using developing deep learning approaches. The implementation procedures in our proposed work involves data collection, pre-processing, training, feature extraction, testing, and classification utilising the Visual Geometry Group (VGG 16) to identify damaged or healthy leaves. VGG 16 is incorporated to categorise the leaves as healthy or diseased based on the data and Regression's boundary box method is adopted. Therefore, using Faster RCNN, a model is created to identify and categorise diseases from every image of a tomato leaf that is used as an input, providing a forecast with a considerably greater degree of accuracy. We obtain an accuracy of approximately 98% after fitting the collected features into the neural network over 20 iterations.

With the development of artificial intelligence, pedestrian detection has become an important research topic in the field of intelligent video surveillance. Fish-eye camera is a useful tool for video monitoring. However, due to the edge distortion of the fish-eye image, which puts higher requirements and challenges on the pedestrian detection technology of fish-eye images. In this paper, an effective method is proposed by rotating cutting to address the problem, a fish-eye image is divided into an edge portion and a center portion. The effectiveness and performance of our method is verified by the traditional pedestrian detection method HOG+SVM and the Faster R-CNN based on convolutional neural network. The experimental results demonstrate the efficacy of the proposed approach, and Faster R-CNN achieves better performance than traditional method.

A number of studies have been conducted to enhance the pedestrian detection accuracy of intelligent surveillance systems. However, detecting pedestrians under outdoor conditions is a challenging problem due to the varying lighting, shadows, and occlusions. In recent times, a growing number of studies have been performed on visible light camera-based pedestrian detection systems using a convolutional neural network (CNN) in order to make the pedestrian detection process more resilient to such conditions. However, visible light cameras still cannot detect pedestrians during nighttime, and are easily affected by shadows and lighting. There are many studies on CNN-based pedestrian detection through the use of far-infrared (FIR) light cameras (i.e., thermal cameras) to address such difficulties. However, when the solar radiation increases and the background temperature reaches the same level as the body temperature, it remains difficult for the FIR light camera to detect pedestrians due to the insignificant difference between the pedestrian and non-pedestrian features within the images. Researchers have been trying to solve this issue by inputting both the visible light and the FIR camera images into the CNN as the input. This, however, takes a longer time to process, and makes the system structure more complex as the CNN needs to process both camera images. This research adaptively selects a more appropriate candidate between two pedestrian images from visible light and FIR cameras based on a fuzzy inference system (FIS), and the selected candidate is verified with a CNN. Three types of databases were tested, taking into account various environmental factors using visible light and FIR cameras. The results showed that the proposed method performs better than the previously reported methods.

Recent advances in the field of object detection and face recognition have made it possible to develop practical video surveillance systems with embedded object detection and face recognition functionalities that are accurate and fast enough for commercial uses. In this paper, we compare some of the latest approaches to object detection and face recognition and provide reasons why they may or may not be amongst the best to be used in video surveillance applications in terms of both accuracy and speed. It is discovered that Faster R-CNN with Inception ResNet V2 is able to achieve some of the best accuracies while maintaining real-time rates. Single Shot Detector (SSD) with MobileNet, on the other hand, is incredibly fast and still accurate enough for most applications. As for face recognition, FaceNet with Multi-task Cascaded Convolutional Networks (MTCNN) achieves higher accuracy than advances such as DeepFace and DeepID2+ while being faster. An end-to-end video surveillance system is also proposed which could be used as a starting point for more complex systems. Various experiments have also been attempted on trained models with observations explained in detail. We finish by discussing video object detection and video salient object detection approaches which could potentially be used as future improvements to the proposed system.

Convolutional neural networks (CNN) applications range from recognition and reasoning (such as handwriting recognition, facial expression recognition and video surveillance) to intelligent text applications such as semantic text analysis and natural language processing applications. Two key observations drive the design of a new architecture for CNN. First, CNN workloads exhibit a widely varying mix of three types of parallelism: parallelism within a convolution operation, intra-output parallelism where multiple input sources (features) are combined to create a single output, and inter-output parallelism where multiple, independent outputs (features) are computed simultaneously. Workloads differ significantly across different CNN applications, and across different layers of a CNN. Second, the number of processing elements in an architecture continues to scale (as per Moore's law) much faster than off-chip memory bandwidth (or pin-count) of chips. Based on these two observations, we show that for a given number of processing elements and off-chip memory bandwidth, a new CNN hardware architecture that dynamically configures the hardware on-the-fly to match the specific mix of parallelism in a given workload gives the best throughput performance. Our CNN compiler automatically translates high abstraction network specification into a parallel microprogram (a sequence of low-level VLIW instructions) that is mapped, scheduled and executed by the coprocessor. Compared to a 2.3 GHz quad-core, dual socket Intel Xeon, 1.35 GHz C870 GPU, and a 200 MHz FPGA implementation, our 120 MHz dynamically configurable architecture is 4x to 8x faster. This is the first CNN architecture to achieve real-time video stream processing (25 to 30 frames per second) on a wide range of object detection and recognition tasks.

本文在传统的依靠RGB通道模板和卷积神经网络的行人检测框架下，提出了结合多通道模板和卷积神经网络的行人检测方法。主要工作是：一是采用运动侦测方法初步检测感兴趣区域，提高行人检测效率和降低虚警。二是采用多通道模板和最大值融合策略，降低单纯依靠RGB通道模板可能存在的行人漏检现象。 In this paper, based on the traditional pedestrian detection framework of RGB channel template and convolution neural network, a pedestrian detection method combining multi-channel tem-plate and convolution neural network is proposed. The main contribution is: firstly, the motion detection method is used to detect the interest area, improve pedestrian detection efficiency and reduce the false alarm. Secondly, multi-channel template and maximum fusion strategy are adopted to reduce the possible pedestrian leakage detection based on RGB channel template.

Sun orientation estimation from a single image using short-cuts in DCNN

The main work of human motion gesture recognition is to recognize and analyze the behavior of human objects in the video. Although the current research in the field of human motion gesture recognition has achieved certain results, the human motion gesture recognition in real life scenes has great effects due to factors such as camera movement, target scale transformation, dynamic background, viewing angle, and illumination. This article first proposes a new method of constructing human motion posture features to describe human behavior. This method is based on deep convolutional neural network features and topic models. Experiments have verified that compared with the traditional feature map extracted from the convolutional neural network fully connected layer, the feature map extracted from the convolutional neural network convolutional layer is not only lower in dimension but also has higher discrimination. Secondly, based on the feature map of the convolutional neural network, the training map downsampling strategy is used to overcome the interference caused by the object's scale change and shape change. Finally, based on the basketball gesture recognition method, the behavior performance of the legs and arms in 9 basketball actions of walking, running, jumping, standing dribbling, walking dribbling, running dribbling, shooting, passing and receiving is analyzed. As well as the corresponding signal waveform characteristics, a two-stage data division method for basketball is proposed. The unit action data is extracted for analysis to realize feature extraction. In order to select the most suitable classifier for basketball gesture recognition, the constructed feature vector uses four Different classifiers are trained to construct different classifiers to realize the division of actions.