A Real-Time Pedestrian Situation Detection Method Using CNN and DeepSORT with Rule-Based Analysis for Autonomous Mobility

Recently, pedestrian detection has become an important problem of interest. Our work primarily depends on robust and fast deep neural network architectures. This paper used very efficient and recent methods for pedestrian detection. Recently, pedestrian detection has become an important problem of interest. This paper suggests robust convolutional neural network models to solve this problem. We primarily evaluate accuracy and speed. Our work primarily depends on robust and fast deep neural network architectures; substantial changes to those models achieve results that are competitive with prior state-of-the-art methods. As a result, we outperformed all the prior state-of-the-art pedestrian detection methods. We also overtook other models that use extra information during testing and training. All experiments used three pedestrian detection challenge benchmarks: Caltech-USA, INRIA, and ETH.

本文在传统的依靠RGB通道模板和卷积神经网络的行人检测框架下，提出了结合多通道模板和卷积神经网络的行人检测方法。主要工作是：一是采用运动侦测方法初步检测感兴趣区域，提高行人检测效率和降低虚警。二是采用多通道模板和最大值融合策略，降低单纯依靠RGB通道模板可能存在的行人漏检现象。 In this paper, based on the traditional pedestrian detection framework of RGB channel template and convolution neural network, a pedestrian detection method combining multi-channel tem-plate and convolution neural network is proposed. The main contribution is: firstly, the motion detection method is used to detect the interest area, improve pedestrian detection efficiency and reduce the false alarm. Secondly, multi-channel template and maximum fusion strategy are adopted to reduce the possible pedestrian leakage detection based on RGB channel template.

Pedestrian detection is considered one of the most challenging problems in computer vision, as it involves the combination of classification and localization within a scene. Recently, convolutional neural networks (CNNs) have been demonstrated to achieve superior detection results compared to traditional approaches. Although YOLOv3 (an improved You Only Look Once model) is proposed as one of state-of-the-art methods in CNN-based object detection, it remains very challenging to leverage this method for real-time pedestrian detection. In this paper, we propose a new framework called SA YOLOv3, a scale-aware You Only Look Once framework which improves YOLOv3 in improving pedestrian detection of small scale pedestrian instances in a real-time manner. Our network introduces two sub-networks which detect pedestrians of different scales. Outputs from the sub-networks are then combined to generate robust detection results. Experimental results show that the proposed SA YOLOv3 framework outperforms the results of YOLOv3 on public datasets and run at an average of 11 fps on a GPU.

An improved tiny-yolov3 pedestrian detection algorithm

A number of studies have been conducted to enhance the pedestrian detection accuracy of intelligent surveillance systems. However, detecting pedestrians under outdoor conditions is a challenging problem due to the varying lighting, shadows, and occlusions. In recent times, a growing number of studies have been performed on visible light camera-based pedestrian detection systems using a convolutional neural network (CNN) in order to make the pedestrian detection process more resilient to such conditions. However, visible light cameras still cannot detect pedestrians during nighttime, and are easily affected by shadows and lighting. There are many studies on CNN-based pedestrian detection through the use of far-infrared (FIR) light cameras (i.e., thermal cameras) to address such difficulties. However, when the solar radiation increases and the background temperature reaches the same level as the body temperature, it remains difficult for the FIR light camera to detect pedestrians due to the insignificant difference between the pedestrian and non-pedestrian features within the images. Researchers have been trying to solve this issue by inputting both the visible light and the FIR camera images into the CNN as the input. This, however, takes a longer time to process, and makes the system structure more complex as the CNN needs to process both camera images. This research adaptively selects a more appropriate candidate between two pedestrian images from visible light and FIR cameras based on a fuzzy inference system (FIS), and the selected candidate is verified with a CNN. Three types of databases were tested, taking into account various environmental factors using visible light and FIR cameras. The results showed that the proposed method performs better than the previously reported methods.

In this paper, we propose the dense disparity map-based pedestrian detection method for intelligent vehicle. The dense disparity map is utilized to improve the pedestrian detection performance. Our method consists of several steps namely, obstacle area detection using road feature information and column detection, pedestrian area detection using dense disparity map-based segmentation, and pedestrian detection using optimal feature. The first step is to detect all obstacle areas using column detection and pedestrian height information. However, there are many objects in single obstacle area. Thus each obstacle area needs to be separated into single object for improving pedestrian detection performance. Thus, the second step is performed to segment the detected obstacle area. The last step is to detect only pedestrian using classifier trained by optimal feature. The optimal feature is extracted by positive and negative training images. ETH database is utilized to evaluate our proposed pedestrian detection method.

Gait recognition is a noncontact biometric procedure that determines the identity or health status of a person by analyzing his or her walking posture and habits, including skeletal and joint movements. The most remarkable feature of this method is the possibility of conducting recognition without demanding much cooperation from participants. Therefore, this recognition technique has attracted much attention from scholars. Additionally, because of the rapid development of graphics processing unit technology, related hardware and computation performance, the applications of deep-learning technology are considerably enhanced. The objective of this study was to apply a deep neural network (DNN), which employs deep-learning technology, to achieve gait-based automatic pedestrian detection and recognition. In contrast to using wearable devices to precisely capture skeletal and joint movements, pedestrian color-image sequences were used as input in this study. Subsequently, a pretraining convolutional neural network (CNN) was employed to capture pedestrian location and extract pedestrian dense optical flow to serve as concrete low-level feature inputs. Then, a finely-tuned DNN based on the wide residual network was employed to extract high-level abstract features. In addition, to overcome the difficulty of obtaining local temporal features by using a 2D CNN, part of the 3D convolutional structure was introduced into the CNN. This design enabled use of limited memory to acquire more effective features and enhance the DNN performance. The experimental results show that the proposed method has exceptional performance for pedestrian detection and recognition.

Pedestrian detection is an essential technology in robotics, intelligent transportation system, and intelligent video surveillance. Pedestrians in the surveillance scene have the characteristics of dense crowds and high degree of occlusion, meanwhile, it needs to meet the requirements of real-time detection. To solve this problem, the method based on head model with YOLOv3 algorithm was proposed. This work re-selects the number and dimensions of anchor boxes by k-means method on the training set, fine-tuning the network and training it to get the optimal model. The experiment results show that this work effectively improves the detection performance and real-time of pedestrian detection in the surveillance scene.

In order to solve the problem that the traditional pedestrian recognition and ranging are not satisfactory, a monocular visual nighttime pedestrian recognition and fast ranging method suitable for vehicle infrared night vision images is proposed. Based on the non-contact thermal imaging principle of far-infrared night vision goggles and mathematical morphology method, an improved pedestrian detection and recognition method for three-frame differential monocular vision and geometric motion rate is established. On the basis of pedestrian recognition, monocular camera fast ranging algorithm is used for fast ranging pedestrians. The experimental results show that this method has the characteristics of high pedestrian recognition accuracy and high real-time ranging accuracy.

MAPD: An improved multi-attribute pedestrian detection in a crowd

Vision-based pedestrian detection technology is widely used in security surveillance, such as transportation, prisons, and important places. Security monitoring requires pedestrian detection not only to be accurate and reliable, but also to meet the needs of real-time monitoring, and the false alarm rate is low. The existing pedestrian learning method based on deep learning has high reliability, but the timeliness cannot meet the monitoring requirements. Traditional pedestrian detection methods are time-sensitive, but false alarm rates tend to be higher. This paper proposes a self-learning pedestrian detection method, combined with motion detection, HOG feature extraction, and SVM classification for preliminary pedestrian detection. The false alarms caused during the pedestrian detection process are manually confirmed, and then the system automatically detects the false alarm image at a specific time. Self-learning, retraining the classifier, so as to continuously reduce the pedestrian detection false alarm rate and improve the pedestrian detection performance index through self-learning during the system operation, and the timeliness can meet the needs of video surveillance. Experiments show that the false alarm rate of this method is low, timeliness is high, and the recognition rate is also high.

While remarkable progress has been made to pedestrian detection in recent years, robust pedestrian detection in the wild e.g., under surveillance scenarios with occlusions, remains a challenging problem. In this paper, we present a novel approach for joint pedestrian and body part detection via semantic relationship learning under unconstrained scenarios. Specifically, we propose a Body Part Indexed Feature (BPIF) representation to encode the semantic relationship between individual body parts (i.e., head, head-shoulder, upper body, and whole body) and highlight per body part features, providing robustness against partial occlusions to the whole body. We also propose an Adaptive Joint Non-Maximum Suppression (AJ-NMS) to replace the original NMS algorithm widely used in object detection, leading to higher precision and recall for detecting overlapped pedestrians. Experimental results on the public-domain CUHK-SYSU Person Search Dataset show that the proposed approach outperforms the state-of-the-art methods for joint pedestrian and body part detection in the wild.

Pyramid binary pattern features for real-time pedestrian detection from infrared videos

In recent years, security monitoring of public places and critical infrastructure has heavily relied on the widespread use of cameras, raising concerns about personal privacy violations. To balance the need for effective security monitoring with the protection of personal privacy, we explore the potential of optical fiber sensors for this application. This article proposes FiberFlex, an intelligent and distributed fiber sensor system. Ultizing Field Programmable Gate Arrays (FPGA) high-level synthesis (HLS) acceleration, FiberFlex offers real-time pedestrian detection by co-designing the entire pipeline of optical signal acquisition, processing, and recognition networks based on the principles of optical fiber sensing. As a promising alternative to traditional camera-based monitoring systems, FiberFlex achieves pedestrian detection by analyzing the vibration patterns caused by pedestrian footsteps, enabling security monitoring while preserving individual privacy. FiberFlex comprises three modules: First , fiber-optic sensing system: A fiber-optic distributed acoustic sensing (DAS) system is built and used to measure the ground vibration waves generated by people walking. Second , algorithms: We first collect the training data by measuring the ground vibration waves, label the data, and use the data to train the neural network models to perform pedestrian recognition. Third , hardware accelerators: We use HLS tools to design hardware modules on FPGA for data collection and pre-processing and integrate them with the downstream neural network accelerators to perform in-line real-time pedestrian detection. The final detection results are sent back from FPGA to the host CPU. We implement our system FiberFlex with the in-house built DAS system and AMD/Xilinx Kintex7 FPGA KC705 board and verify the whole system using the real-world collected data. We conduct recognition tests on five test subjects of varying ages, heights, and weights in a fixed sensing area. Each subject experienced 20 real-time recognition tests using their daily walking habits, and the subjects were given adequate rest between tests. After 100 tests on five test subjects, the overall real-time recognition accuracy exceeded \(88.0\%\) . The whole system uses 55 W of power, 33 W in the optical DAS system and 22 W in the FPGA. Relying on its end-to-end interdisciplinary design, FiberFlex seamlessly combines fiber-optic sensors with FPGA accelerators to enable low-power real-time security monitoring without compromising privacy, making it a valuable addition to the existing security monitoring network. According to FiberFlex, more valuable research can be conducted in the future, such as fall monitoring for the elderly, migration of identification networks between different application scenarios, and improvement of anti-interference performance in more complex environments. In future perception networks, where the “eyes” are not feasible, let’s use fiber optic touch instead.

More than 80% of video surveillance systems are used for monitoring people. Old human detection algorithms, based on background and foreground modelling, could not even deal with a group of people, to say nothing of a crowd. Recent robust and highly effective pedestrian detection algorithms are a new milestone of video surveillance systems. Based on modern approaches in deep learning, these algorithms produce very discriminative features that can be used for getting robust inference in real visual scenes. They deal with such tasks as distinguishing different persons in a group, overcome problem with sufficient enclosures of human bodies by the foreground, detect various poses of people. In our work we use a new approach which enables to combine detection and classification tasks into one challenge using convolution neural networks. As a start point we choose YOLO CNN, whose authors propose a very efficient way of combining mentioned above tasks by learning a single neural network. This approach showed competitive results with state-of-the-art models such as FAST R-CNN, significantly overcoming them in speed, which allows us to apply it in real time video surveillance and other video monitoring systems. Despite all advantages it suffers from some known drawbacks, related to the fully-connected layers that obstruct applying the CNN to images with different resolution. Also it limits the ability to distinguish small close human figures in groups which is crucial for our tasks since we work with rather low quality images which often include dense small groups of people. In this work we gradually change network architecture to overcome mentioned above problems, train it on a complex pedestrian dataset and finally get the CNN detecting small pedestrians in real scenes.