Research on pedestrian targe detection based on deep learning

Gait recognition is a noncontact biometric procedure that determines the identity or health status of a person by analyzing his or her walking posture and habits, including skeletal and joint movements. The most remarkable feature of this method is the possibility of conducting recognition without demanding much cooperation from participants. Therefore, this recognition technique has attracted much attention from scholars. Additionally, because of the rapid development of graphics processing unit technology, related hardware and computation performance, the applications of deep-learning technology are considerably enhanced. The objective of this study was to apply a deep neural network (DNN), which employs deep-learning technology, to achieve gait-based automatic pedestrian detection and recognition. In contrast to using wearable devices to precisely capture skeletal and joint movements, pedestrian color-image sequences were used as input in this study. Subsequently, a pretraining convolutional neural network (CNN) was employed to capture pedestrian location and extract pedestrian dense optical flow to serve as concrete low-level feature inputs. Then, a finely-tuned DNN based on the wide residual network was employed to extract high-level abstract features. In addition, to overcome the difficulty of obtaining local temporal features by using a 2D CNN, part of the 3D convolutional structure was introduced into the CNN. This design enabled use of limited memory to acquire more effective features and enhance the DNN performance. The experimental results show that the proposed method has exceptional performance for pedestrian detection and recognition.

Pedestrian detection is a challenging and hot research topic in the field of computer vision, especially for the crowded scenes where occlusion happens frequently. In this paper, we propose a novel AutoPedestrian scheme that automatically augments the pedestrian data and searches for suitable loss functions, aiming for better performance of pedestrian detection especially in crowded scenes. To our best knowledge, it is the first work to automatically search the optimal policy of data augmentation and loss function jointly for the pedestrian detection. To achieve the goal of searching the optimal augmentation scheme and loss function jointly, we first formulate the data augmentation policy and loss function as probability distributions based on different hyper-parameters. Then, we apply a double-loop scheme with importance-sampling to solve the optimization problem of data augmentation and loss function types efficiently. Comprehensive experiments on two popular benchmarks of CrowdHuman and CityPersons show the effectiveness of our proposed method. In particular, we achieve 40.58% in MR on CrowdHuman datasets and 11.3% in MR on CityPersons reasonable subset, yielding new state-of-the-art results on these two datasets.

The goal of this study is to clarify the usefulness of deep learning methods for pedestrian collision detection using dashcam videos for advanced automatic collision notification, focusing on pedestrians, as they make up the highest number of traffic fatalities in Japan. First, we created a dataset for deep learning from dashcam videos. A total of 78 dashcam videos of pedestrian-to-automobile accidents were collected from a video hosting website and from the Japan Automobile Research Institute (JARI). Individual frames were selected from the video data amounting to a total of 1,212 still images, which were added to our dataset with class and location information. This dataset was then divided to create training, validation, and test datasets. Next, deep learning was performed based on the training dataset to learn the features of pedestrian collision images, which are images that capture pedestrian behavior at the time of the collision. Pedestrian collision detection performance of the trained model was evaluated as the percentage of correct predictions of pedestrian collisions in image data according to varied test sets with different combinations of characteristics. Our results for the proposed method show high-precision collision detection for daytime, clear pedestrian wrap trajectory accident data, including accurate detection of pedestrian collision location information. However, nighttime, unclear accident data resulted in false detection or no detection. Reduction of exposure value and resolution was confirmed to reduce detection rate. The results of the present study suggest the possibility of pedestrian collision detection by deep learning using dashcam videos.

The advanced intelligent driving assistance system has improved the current traffic congestion to a great extent and effectively reduced frequent traffic safety accidents. Pedestrian detection technology is the core of autonomous driving technology, and its accuracy, real-time and complexity will directly determine the safe operation of autonomous driving. In the case of heavy traffic, detecting a single pedestrian in a crowd is still a challenging problem. Considering the problem of mutual occlusion between pedestrians in dense crowds, an improved function algorithm based on YOLOv3 is proposed to optimize the loss function and increase the accuracy of detection by replacing the anchor frame. Experimental results show that this method can effectively reduce the missed detection rate, increase the average accuracy, and help improve the effectiveness of pedestrian occlusion detection, ensure accurate pedestrian detection in traffic congestion scenarios, and ensure driving safety.KeywordsComputer visionDeep learningPedestrian detectionLoss function

Pedestrian detection is another significant special application of object detection in autonomous vehicles. In contrast to universal object detection, it has similarities and special traits. Nevertheless, there are some difficulties that influence pedestrian detection performance, namely (i) occlusion and deformation (ii) low-quality and multispectral images consisting primarily of lighting conditions, small-scale detection, and target detection extensively, and (iii) true-false pedestrians. Deep learning (DL) methods are a class of artificial intelligence method that can solve the issues mentioned above of pedestrian detection. This paper initially gives an elaborate description of pedestrian detection, difficulties in pedestrian detection, and latest advancements in solving them using the assistance of DL methods with informative discussions, aiming to provide insights to the readers. (2) A new pedestrian detection algorithm (PDA) of true/false pedestrian is suggested here, in which a new YOLO-3D CNN model is applied to reject true/false pedestrian. The primary purpose is to evaluate the performance of the existing 3D CNN taking into consideration the problem of rejecting true false pedestrians based on images captured using the car's onboard cameras and light detection and ranging (LiDAR) sensors. PDA initially utilizes YOLOv3 to capture the entire image for training detector model capable of real-time forecasting. Next, as a feature extractor, it utilizes the MobileNet-SSD that provides great accuracy as well as good trading uptime. PDA then implements the Faster R-CNN method to detect different parts of the object, over the convolutional layer. Lastly, data augmentation techniques are applied in PDA to augment the data coverage by fully exploiting available training data. Simulation results indicate that the proposed pedestrian detection model and PDA improve the accuracy of real and false pedestrians while maintaining real-time requirements.

With the rapid development of driverless cars, pedestrian detection has been a canonical instance of object detection. Although recent deep learning detectors such as RPN+BF and MS-CNN have shown excellent performance for pedestrian detection, they have limited success for detecting pedestrian, and the importance of final feature receptive field has been awared by previous leading deep learning pedestrian detectors. Applying the dilated convolution to the feature learning of pedestrian detection, we constructed a pedestrian detection framework along with the region proposal network and boosted decision trees. Pipeline of our proposed framework can be briefly generalized as follows: firstly, the fine-tuned RPN with specified aspect ratio is used to get boxes and scores. Secondly, the designed dilated convolution feature extraction model is used to get features. As different dilation factors provide different receptive field scales, we concat the features of different layers with the dilated convolutional features to get the final features. Finally, the candidate boxes are sent to the boosted decision trees to be classified using the scores and features. We evaluated our method on the Caltech Pedestrian Detection Benchmark. Comparing with other state-of-the-art detection methods, the proposed framework with dilated convolution has better performance.

Despite significant research efforts in pedestrian detection over the past decade, there is still a ten-fold performance gap between the state-of-the-art methods and human perception. Deep learning methods can provide good performance but suffers from high computational complexity which prohibits their deployment on affordable systems with limited computational resources. In this paper, we propose a pedestrian detection framework that provides a major fillip to the robustness and run-time efficiency of the recent top performing non-deep learning Filtered Channel Feature (FCF) approach. The proposed framework overcomes the computational bottleneck of existing FCF methods by exploiting vector form filters to efficiently extract more discriminative channel features for pedestrian detection. A novel dual-stage group cost-sensitive RealBoost algorithm is used to explore different costs among different types of misclassification in the boosting process in order to improve detection performance. In addition, we propose two strategies, selective classification and selective scale processing, to further accelerate the detection process at the channel feature level and image pyramid level respectively. Experiments on the Caltech and INRIA datasets show that the proposed method achieves the highest detection performance among all the state-of-the-art non-CNN methods and is about 148X faster than the existing best performing FCF method on the Caltech dataset.

In this paper, we propose the dense disparity map-based pedestrian detection method for intelligent vehicle. The dense disparity map is utilized to improve the pedestrian detection performance. Our method consists of several steps namely, obstacle area detection using road feature information and column detection, pedestrian area detection using dense disparity map-based segmentation, and pedestrian detection using optimal feature. The first step is to detect all obstacle areas using column detection and pedestrian height information. However, there are many objects in single obstacle area. Thus each obstacle area needs to be separated into single object for improving pedestrian detection performance. Thus, the second step is performed to segment the detected obstacle area. The last step is to detect only pedestrian using classifier trained by optimal feature. The optimal feature is extracted by positive and negative training images. ETH database is utilized to evaluate our proposed pedestrian detection method.

Pedestrian detection is an essential technology in robotics, intelligent transportation system, and intelligent video surveillance. Pedestrians in the surveillance scene have the characteristics of dense crowds and high degree of occlusion, meanwhile, it needs to meet the requirements of real-time detection. To solve this problem, the method based on head model with YOLOv3 algorithm was proposed. This work re-selects the number and dimensions of anchor boxes by k-means method on the training set, fine-tuning the network and training it to get the optimal model. The experiment results show that this work effectively improves the detection performance and real-time of pedestrian detection in the surveillance scene.

Coronavirus Disease 2019 (COVID-19) is a highly infectious virus that has created a health crisis for people all over the world. Social distancing has proved to be an effective non-pharmaceutical measure to slow down the spread of COVID-19. As unmanned aerial vehicle (UAV) is a flexible mobile platform, it is a promising option to use UAV for social distance monitoring. Therefore, we propose a lightweight pedestrian detection network to accurately detect pedestrians by human head detection in real-time and then calculate the social distancing between pedestrians on UAV images. In particular, our network follows the PeleeNet as backbone and further incorporates the multi-scale features and spatial attention to enhance the features of small objects, like human heads. The experimental results on Merge-Head dataset show that our method achieves 92.22% AP (average precision) and 76 FPS (frames per second), outperforming YOLOv3 models and SSD models and enabling real-time detection in actual applications. The ablation experiments also indicate that multi-scale feature and spatial attention significantly contribute the performance of pedestrian detection. The test results on UAV-Head dataset show that our method can also achieve high precision pedestrian detection on UAV images with 88.5% AP and 75 FPS. In addition, we have conducted a precision calibration test to obtain the transformation matrix from images (vertical images and tilted images) to real-world coordinate. Based on the accurate pedestrian detection and the transformation matrix, the social distancing monitoring between individuals is reliably achieved.

Corrosion is a natural phenomenon that causes significant material and economic losses. Timely inspection of corroded assets is vital for preventing such losses. Visual inspection is a non-destructive method that is commonly used for corrosion detection. The ability of deep learning methods to capture corrosion visual information allows for the automation of the manual inspection process. The literature includes several examples of deep learning models to detect metal corrosion. In developing such models, loss functions are crucial and directly influence the models’ performance. Although cross-entropy is a commonly used objective function for corrosion detection, systematic evaluation of alternative loss functions—and the performance gains they may unlock—has been largely neglected. This study uses eight loss functions, namely, cross-entropy loss, Focal loss, Unified Focal loss, Dice loss, Tversky loss, Focal Tversky loss, Combo loss, and Hybrid Focal loss, to train UNet-based models for corrosion segmentation and compares their performance. The results demonstrated that the model trained with the Dice loss achieved higher performance, with a mean Jaccard index value of 64.28%, while the Focal loss exhibited the lowest performance, with a mean Jaccard value of 57.20%. This study contributes to academia and practice by demonstrating the performance gains that can be achieved by tweaking the deep learning models using optimal loss functions. • Corrosion is a phenomenon that considerably impacts the environment and economy. • Deep learning models can be trained for corrosion detection, and loss functions are essential to developing accurate models. • This study evaluates the use of different loss functions, including the Focal loss, Unified Focal loss, Dice loss, Tversky loss, Focal Tversky loss, Combo loss, and Hybrid Focal loss, to train UNet-based models for corrosion segmentation. • On a standard dataset, the models developed with Combo loss, Tversky loss, Unified Focal loss, and Dice loss had higher performance than those developed with cross-entropy loss. • With the hyperparameter tuning, higher performance gains can be achieved.

We present a unified multi-task learning architecture for fast and accurate pedestrian detection. Different from existing methods which often focus on either a new loss function or architecture, we propose an improved multi-task convolutional neural network learning architecture to effectively and efficiently interfuse the task of pedestrian detection and semantic segmentation. To achieve this, we integrate a lightweight semantic segmentation branch to Faster R-CNN detection framework that enables end-to-end hard parameter sharing in order to boost the detection performance and maintain computational efficiency as follows. Firstly, a Semantic Segmentation to Feature Module (SS2FM) refines the convolutional features in RPN stage by integrating the features generated from the semantic segmentation branch. Secondly, a Semantic Segmentation to Confidence Module (SS2CM) refines the classification confidence in RPN stage by fusing it with the semantic segmentation confidence. We also introduce an effective anchor matching point transform to alleviate the problem of feature misalignment for heavily occluded pedestrians. The proposed unified multi-task learning architecture lends itself well to more robust pedestrian detection in diverse scenarios with negligible computation overhead. In addition, the proposed architecture can achieve high detection performance with low resolution input images, which significantly reduces the computational complexity. Experiment results on CityPersons and Caltech datasets show that our method is the fastest among all state-of-the-art pedestrian detection methods while exhibiting competitive detection performance.

Vision-based pedestrian detection technology is widely used in security surveillance, such as transportation, prisons, and important places. Security monitoring requires pedestrian detection not only to be accurate and reliable, but also to meet the needs of real-time monitoring, and the false alarm rate is low. The existing pedestrian learning method based on deep learning has high reliability, but the timeliness cannot meet the monitoring requirements. Traditional pedestrian detection methods are time-sensitive, but false alarm rates tend to be higher. This paper proposes a self-learning pedestrian detection method, combined with motion detection, HOG feature extraction, and SVM classification for preliminary pedestrian detection. The false alarms caused during the pedestrian detection process are manually confirmed, and then the system automatically detects the false alarm image at a specific time. Self-learning, retraining the classifier, so as to continuously reduce the pedestrian detection false alarm rate and improve the pedestrian detection performance index through self-learning during the system operation, and the timeliness can meet the needs of video surveillance. Experiments show that the false alarm rate of this method is low, timeliness is high, and the recognition rate is also high.

Detecting pedestrians accurately is the first fundamental step for many computer vision applications such as video surveillance, smart vehicles, intersection traffic analysis and so on. The authors present an experimental study on pedestrian detection using state-of-the-art local feature extraction and support vector machine (SVM) classifiers. The performance of pedestrian detection using region covariance, histogram of oriented gradients (HOG) and local receptive fields (LRF) feature descriptors is experimentally evaluated. The experiments are performed on the DaimlerChrysler benchmarking data set, the MIT CBCL data set and ’Intitut National de Recherche en Informatique et Automatique (INRIA) data set. All can be publicly accessed. The experimental results show that region covariance features with radial basis function kernel SVM and HOG features with quadratic kernel SVM outperform the combination of LRF features with quadratic kernel SVM. Furthermore, the results reveal that both covariance and HOG features perform very well in the context of pedestrian detection.

Convolutional neural network (CNN) has developed such a large network size in last few years, so reducing the storage requirement without hurting its accuracy becomes necessary. In this paper, in order to reduce the number of high dimensional feature maps in shallow layers, we propose a feature map selection method, which cuts the feature map number by correlation coefficient between kernels and finishes detection by HOG+SVM method. Firstly, we extract feature maps of shallow layers from trained CNN. Then, we merge strongly relevant feature maps and choose all maps among weakly relevant feature maps by analyzing correlation coefficient of kernels. Finally, we extract HOG features of the chosen feature maps and use SVM to complete the training and classification. The experimental results show that the proposed method can effectively prune high dimensional feature maps and stabilize or even advance the performance in pedestrian detection.