- Research Article
68
- 10.1016/j.iatssr.2016.11.002
Human-like motion planning model for driving in signalized intersections
- Nov 15, 2016
- IATSS Research
- Yanlei Gu + 4 more +4
Human-like motion planning model for driving in signalized intersections
Early detection of vulnerable road users is a crucial requirement for autonomous vehicles to meet and exceed the object detection capabilities of human drivers. One of the most complex outstanding challenges is that of partial occlusion where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of leading pedestrian detection benchmarks provide annotation for partial occlusion, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. Research demonstrates that a high degree of subjectivity is used to classify occlusion level in these cases and occlusion is typically categorized into 2–3 broad categories such as “partially” and “heavily” occluded. In addition, many pedestrian instances are impacted by multiple inhibiting factors which contribute to non-detection such as object scale, distance from camera, lighting variations and adverse weather. This can lead to inaccurate or inconsistent reporting of detection performance for partially occluded pedestrians depending on which benchmark is used. This research introduces a novel, objective benchmark for partially occluded pedestrian detection to facilitate the objective characterization of pedestrian detection models. Characterization is carried out on seven popular pedestrian detection models for a range of occlusion levels from 0%–99% to demonstrate the impact of progressive levels of partial occlusion on pedestrian detectability. Results show that the proposed benchmark provides more objective, fine grained analysis of pedestrian detection algorithms than the current state of the art.
Human-like motion planning model for driving in signalized intersections
Human-like motion planning model for driving in signalized intersections
Feature optimization for pedestrian detection based on faster R-CNN
Due to the rise of convolutional neural network, pedestrian detection has achieved great success. However, the features of many existing methods are not fully utilized, which results in unsatisfactory detection results. A new pedestrian detection model which is named of SE-Faster R-CNN is proposed in this paper. By adding SENet block to Faster RCNN, it can strengthen the expressive force of feature. Then, the GN-Faster R-CNN, which is generated by adding the normalization layer -- Group Normalization layer to Faster R-CNN, is proposed. The proposed architecture is trained and tested on Caltech dataset. In addition, VGG16 model and ZF model are used as the backbone structure of detection network. A comparative experiment is implemented to compare the effectiveness of the two optimization methods. It can be seen from the experimental results that, after adding SENet, the miss rates of ZF model and VGG16 model were reduced by 0.392% and 0.999%, respectively. After adding the GN layer, the miss rate of VGG16 model was reduced by 0.665%, while the miss rate of ZF model was increased by 2.093%.
Read moreMutual-Supervised Feature Modulation Network for Occluded Pedestrian Detection
State-of-the-art pedestrian detectors have achieved significant progress on non-occluded pedestrians, yet they are still struggling under heavy occlusions. The recent occlusion handling strategy of popular two-stage approaches is to build a two-branch architecture with the help of additional visible body annotations. Nonetheless, these methods still have some weaknesses. Either the two branches are trained independently with only score-level fusion, which cannot guarantee the detectors to learn robust enough pedestrian features. Or the attention mechanisms are exploited to only emphasize on the visible body features. However, the visible body features of heavily occluded pedestrians are concentrated on a relatively small area, which will easily cause missing detections. To address the above issues, we propose in this paper a novel Mutual-Supervised Feature Modulation (MSFM) network, to better handle occluded pedestrian detection. The key MSFM module in our network calculates the similarity loss of full body boxes and visible body boxes corresponding to the same pedestrian so that the full-body detector could learn more complete and robust pedestrian features with the assist of contextual features from the occluding parts. To facilitate the MSFM module, we also propose a novel two-branch architecture, consisting of a standard full body detection branch and an extra visible body classification branch. These two branches are trained in a mutual-supervised way with full body annotations and visible body annotations, respectively. To verify the effectiveness of our proposed method, extensive experiments are conducted on two challenging pedestrian datasets: Caltech and CityPersons, and our approach achieves superior performance compared to other state-of-the-art methods on both datasets, especially in heavy occlusion cases.
Read moreSAFE AND EFFICIENT PEDESTRIAN DETECTION FOR AUTONOMOUS VEHICLES THROUGH ADVANCED 3D CNN-BASED SOLUTIONS
Pedestrian detection is another significant special application of object detection in autonomous vehicles. In contrast to universal object detection, it has similarities and special traits. Nevertheless, there are some difficulties that influence pedestrian detection performance, namely (i) occlusion and deformation (ii) low-quality and multispectral images consisting primarily of lighting conditions, small-scale detection, and target detection extensively, and (iii) true-false pedestrians. Deep learning (DL) methods are a class of artificial intelligence method that can solve the issues mentioned above of pedestrian detection. This paper initially gives an elaborate description of pedestrian detection, difficulties in pedestrian detection, and latest advancements in solving them using the assistance of DL methods with informative discussions, aiming to provide insights to the readers. (2) A new pedestrian detection algorithm (PDA) of true/false pedestrian is suggested here, in which a new YOLO-3D CNN model is applied to reject true/false pedestrian. The primary purpose is to evaluate the performance of the existing 3D CNN taking into consideration the problem of rejecting true false pedestrians based on images captured using the car's onboard cameras and light detection and ranging (LiDAR) sensors. PDA initially utilizes YOLOv3 to capture the entire image for training detector model capable of real-time forecasting. Next, as a feature extractor, it utilizes the MobileNet-SSD that provides great accuracy as well as good trading uptime. PDA then implements the Faster R-CNN method to detect different parts of the object, over the convolutional layer. Lastly, data augmentation techniques are applied in PDA to augment the data coverage by fully exploiting available training data. Simulation results indicate that the proposed pedestrian detection model and PDA improve the accuracy of real and false pedestrians while maintaining real-time requirements.
Read moreResearch on Statistics of Passenger Transfer Flow at Urban Rail Transit Station
With the continuous advancement of urbanization, the urban rail transit network continues to be dense, and the number of transfer stations increases, resulting in an increase in the flow of interchange passengers. This paper introduces pedestrian detection into the statistics to realize real-time statistics of passenger transfer flow. This article first determines the research scenario and simplifies the transfer path. Then, the pedestrian detection model is constructed by using the histogram of gradient directions (HOG) and support vector machine (SVM), and combined with the continuous adaptive mean shift algorithm (CamShift) to design and realize the transfer path passenger flow statistics. Finally, the algorithm was trained and tested through the samples obtained at Xi’an Xiaozhai subway station, and the test results obtained have high accuracy.
Read moreTowards accurate dense pedestrian detection via occlusion-prediction aware label assignment and hierarchical-NMS
Towards accurate dense pedestrian detection via occlusion-prediction aware label assignment and hierarchical-NMS
On Diversity and Complementarity of Pedestrian Detection Models
Pedestrian Detection (PD) is now almost a mature field of signal processing in which numerous application areas exists. Emerging one of these applications is driving autonomy that requires advanced driver assistance systems (ADAS) for PD. Although various benchmark data sets are collected and several methods are already developed, the reported results show that current systems lack generalization capacity due to strong dependence of system performance on PD training set. In other words, the systems trained with a pedestrian data set usually perform considerably well on a test set taken from the same benchmark, but their performance drop drastically when tested on a different benchmark data set. In order to overcome this limitation, one of the approaches might be fusing outcomes of different systems. Here, “different systems” can refer either to train the same model with different data sets or using entirely different models. In both cases, the outcomes of different systems should be diverse enough to cover the entire solution space. Up to our knowledge, the performance of existing well-established models for PD is frequently compared to each other but their diversity is not analysed in detail yet. In this study, the feasibility of PD systems, whose effectiveness on PD is shown by earlier studies, was examined by systematically training and testing them with different benchmark data sets and their combinations. For this purpose, pairwise and non-pairwise measures of diversity have been employed and several conclusions are driven about the complementarity of existing PD models.
Read morePRNet++: Learning towards generalized occluded pedestrian detection via progressive refinement network
PRNet++: Learning towards generalized occluded pedestrian detection via progressive refinement network
Joint Pedestrian and Body Part Detection via Semantic Relationship Learning
While remarkable progress has been made to pedestrian detection in recent years, robust pedestrian detection in the wild e.g., under surveillance scenarios with occlusions, remains a challenging problem. In this paper, we present a novel approach for joint pedestrian and body part detection via semantic relationship learning under unconstrained scenarios. Specifically, we propose a Body Part Indexed Feature (BPIF) representation to encode the semantic relationship between individual body parts (i.e., head, head-shoulder, upper body, and whole body) and highlight per body part features, providing robustness against partial occlusions to the whole body. We also propose an Adaptive Joint Non-Maximum Suppression (AJ-NMS) to replace the original NMS algorithm widely used in object detection, leading to higher precision and recall for detecting overlapped pedestrians. Experimental results on the public-domain CUHK-SYSU Person Search Dataset show that the proposed approach outperforms the state-of-the-art methods for joint pedestrian and body part detection in the wild.
Read moreRETRACTED ARTICLE: Enhanced pedestrian detection using optimized deep convolution neural network for smart building surveillance
Pedestrian detection and tracking is a critical task in the area of smart building surveillance. Due to advancements in sensors, the architects concentrate in construction of smart buildings. Pedestrian detection in smart building is greatly challenged by the image noises by various external environmental parameters. Traditional filter-based techniques for image classification like histogram of oriented gradients filters and machine learning algorithms suffer to perform well for huge volume of pedestrian input images. The advancements in deep learning algorithms perform exponentially good in handling the huge volume of image data. The current study proposes a pedestrian detection model based on deep convolution neural network (CNN) for classification of pedestrians from the input images. Proposed optimized version of VGG-16 architecture is evaluated for pedestrian detection on the INRIA benchmarking dataset consisting of 227 × 227 pixel images. The proposed model achieves an accuracy of 98.5%. It was found that proposed model performs better than the other pretrained CNN architectures and other machine learning models. Pedestrians are reasonably detected and the performance of the proposed algorithm is validated.
Read moreComplex overlapping pedestrian target detection network based on the yolov3 model
This paper proposes a complex overlapping pedestrian target detection model based on yolov3 model by multi-scale feature fusion and context-aware mechanism. The SONY A7R3a camera shot the model on campus, and the data set was obtained after editing and collating. There were 358 high-definition videos with a resolution of 1920*1080, and the frame rate was 50HZ, about 179,000 frames. Through testing, this paper finds that compared with Single Shot Multibox Detector (SSD), the detection accuracy of the newly proposed model is slightly improved, the detection accuracy is the same as that of Faster R-CNN, and the detection accuracy of the newly proposed model is slightly worse than that of RetinaNet. However, the detection speed of Yolov3 is more than twice that of Single Shot Multibox Detector, RetinaNet and Faster R-CNN. The input size of Yolov3 is 320*320, and the processing of a single image only needs 22ms, so the detection speed of the simplified Yolov3 tiny is faster.
Read moreReal-time Pedestrian Detection Algorithm Based on Improved YOLOv3
As a research hotspot in the field of current computer vision, pedestrian detection is widely applied to many fields, such as video surveillance and autonomous driving. However, the accuracy of pedestrian detection under video surveillance is poor, and the miss rate of small target pedestrians is high. In this paper, an improves the YOLOv3 algorithm and a YOLOv3-Multi pedestrian detection model had been proposed. First, referring to the residual structure of DarkNet, the shallow features and deep features had been up-sampled and connected to obtain a multi-scale detection layer. Then, according to different special detection categories, the spatial pyramid pool (SPP) is introduced to strengthen the detection of small targets. The experimental results show that our method improves the average accuracy by 2.54%, 6.43% and 8.99%compared with YOLOv3, SSD and YOLOv2 on the VOC dataset.
Read moreOccluded Pedestrian Detection Through Guided Attention in CNNs
Pedestrian detection has progressed significantly in the last years. However, occluded people are notoriously hard to detect, as their appearance varies substantially depending on a wide range of occlusion patterns. In this paper, we aim to propose a simple and compact method based on the FasterRCNN architecture for occluded pedestrian detection. We start with interpreting CNN channel features of a pedestrian detector, and we find that different channels activate responses for different body parts respectively. These findings motivate us to employ an attention mechanism across channels to represent various occlusion patterns in one single model, as each occlusion pattern can be formulated as some specific combination of body parts. Therefore, an attention network with self or external guidances is proposed as an add-on to the baseline FasterRCNN detector. When evaluating on the heavy occlusion subset, we achieve a significant improvement of 8pp to the baseline FasterRCNN detector on CityPersons and on Caltech we outperform the state-of-the-art method by 4pp.
Read moreYOLOv7-PD: Incorporating DE-ELAN and NWD-CIoU for Advanced Pedestrian Detection Method
In the pedestrian detection task, the excessive depth of the convolutional network in YOLOv7 results in an abundance of background feature information, thereby posing challenges for the model to accurately locate and detect pedestrians, particularly in small-scale or heavily occluded scenarios. To handle this problem, we propose a pedestrian detection model called YOLOv7-PD, to strengthen the accuracy of detecting small-scale pedestrians and occluded pedestrians. First of all, we propose an improved module called DE-ELAN, an improvement on the existing E-ELAN module, which is based on Omni-Dimensional Dynamic Convolution (ODConv). This module leverages four complementary attention types to enhance feature extraction, capturing rich contextual information. Then, we propose a lightweight receptive field enhancement module called light-REFM, which constructs a pyramid structure and acquires fine-grained multi-scale information through dilated convolutions of different sizes. Finally, we propose an improved regression loss function based on the Normalized Wasserstein Distance (NWD) that combines NWD with Complete-IoU (CIoU), enabling precise position and feature capture for small targets. On the Citypersons dataset, YOLOv7-PD outperforms YOLOv7, improving the average precision (AP) by 7% and reducing the miss rate by 2.58%. Experiments on three challenging pedestrian detection datasets demonstrate a balance between precision and speed, achieving excellent performance.
Read moreImproving the Robustness of Pedestrian Detection in Autonomous Driving With Generative Data Augmentation
Pedestrian detection plays a crucial role in autonomous driving by identifying the position, size, orientation, and dynamic features of pedestrians in images or videos, assisting autonomous vehicles in making better decisions and controls. It’s worth noting that the performance of pedestrian detection models largely depends on the quality and diversity of available training data. Current datasets for autonomous driving have limitations in terms of diversity, scale, and quality. In recent years, numerous studies have proposed the use of data augmentation strategies to expand the coverage of datasets, aiming to maximize the utilization of existing training data. However, these data augmentation methods often overlook the diversity of data scenarios. To overcome this challenge, in this paper, we propose a more comprehensive method for data augmentation, based on image descriptions and diffusion models. This method aims to cover a wider range of scene variations, including different weather conditions and lighting situations. We have designed a classifier to select data samples for augmentation, followed by extracting visual features based on image captions and converting them into high-level semantic information as textual descriptions for the corresponding samples. Finally, we utilize diffusion models to generate new variants. Additionally, we have designed three modification patterns to increase diversity in aspects such as weather conditions, lighting, and pedestrian poses within the data. We conducted extensive experiments on the KITTI dataset and in real-world environments, demonstrating that our proposed method significantly enhances the performance of pedestrian detection models in complex scenarios. This meticulous consideration of data augmentation will notably enhance the applicability and robustness of pedestrian detection models in actual autonomous driving scenarios.
Read more