- Research Article
10
- 10.1016/j.eswa.2018.06.043
A Multi-strategy Region Proposal Network
- Jun 28, 2018
- Expert Systems with Applications
- Yu-Peng Chen + 3 more +3
A Multi-strategy Region Proposal Network
Faster Region-based Convolutional Network (Faster R-CNN) is a state-of-the-art object detection method. However, the object detection effect of Faster R-CNN is not good based on the Region Proposal Network (RPN). Inspired by RPN of Faster R-CNN, we propose a novel proposal generation method called Enhanced Region Proposal Network (ERPN). Four improvements are presented in ERPN. Firstly, our proposed deconvolutional feature pyramid network (DFPN) is introduced to improve the quality of region proposals. Secondly, novel anchor boxes are designed with interspersed scales and adaptive aspect ratios. Thereafter, the capability of object localization is increased. Thirdly, a particle swarm optimization (PSO) based support vector machine (SVM), termed PSO-SVM, is developed to distinguish the positive and negative anchor boxes. Fourthly, the classification part of multi-task loss function in RPN is improved. Consequently, the effect of classification loss is strengthened. In this study, our proposed ERPN is compared with five object detection methods on both PASCAL VOC and COCO data sets. For the VGG-16 model, our ERPN obtains 78.6% mAP on VOC 2007 data set, 74.4% mAP on VOC 2012 data set and 31.7% on COCO data set. The performance of ERPN is the best among the comparison object detection methods. Furthermore, the detection speed of ERPN is 5.8 fps. Additionally, ERPN obtains good effect on small object detection.
Loading PDF
A Multi-strategy Region Proposal Network
A Multi-strategy Region Proposal Network
Deep learning for small object detection in images
[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] With the rapid development of deep learning in computer vision, especially deep convolutional neural networks (CNNs), significant advances have been made in recent years on object recognition and detection in images. Highly accurate detection results have been achieved for large objects, whereas detection accuracy on small objects remains to be low. This dissertation focuses on investigating deep learning methods for small object detection in images and proposing new methods with improved performance. First, we conducted a comprehensive review of existing deep learning methods for small object detections, in which we summarized and categorized major techniques and models, identified major challenges, and listed some future research directions. Existing techniques were categorized into using contextual information, combining multiple feature maps, creating sufficient positive examples, and balancing foreground and background examples. Methods developed in four related areas, generic object detection, face detection, object detection in aerial imagery, and segmentation, were summarized and compared. In addition, the performances of several leading deep learning methods for small object detection, including YOLOv3, Faster R-CNN, and SSD, were evaluated based on three large benchmark image datasets of small objects. Experimental results showed that Faster R-CNN performed the best, while YOLOv3 was a close second. Furthermore, a new deep learning method, called Retina-context Net, was proposed and outperformed state-of-the art one-stage deep learning models, including SSD, YOLOv3 and RetinaNet, on the COCO and SUN benchmark datasets. Secondly, we created a new dataset for bird detection, called Little Birds in Aerial Imagery (LBAI), from real-life aerial imagery. LBAI contains birds with sizes ranging from 10 by 10 pixels to 40 by 40 pixels. We adapted and applied several state-of-the-art deep learning models to LBAI, including object detection models such as YOLOv2, SSH, and Tiny Face, and instance segmentation models such as U-Net and Mask R-CNN. Our empirical results illustrated the strength and weakness of these methods, showing that SSH performed the best for easy cases, whereas Tiny Face performed the best for hard cases with cluttered backgrounds. Among small instance segmentation methods, U-Net achieved slightly better performance than Mask R-CNN. Thirdly, we proposed a new graph neural network-based object detection algorithm, called GODM, to take the spatial information of candidate objects into consideration in small object detection. Instead of detecting small objects independently as the existing deep learning methods do, GODM treats the candidate bounding boxes generated by existing object detectors as nodes and creates edges based on the spatial or semantic relationship between the candidate bounding boxes. GODM contains four major components: node feature generation, graph generation, node class labelling, and graph convolutional neural network model. Several graph generation methods were proposed. Experimental results on the LBDA dataset show that GODM outperformed existing state-of-the-art object detector Faster R-CNN significantly, up to 12% better in accuracy. Finally, we proposed a new computer vision-based grass analysis using machine learning. To deal with the variation of lighting condition, a two-stage segmentation strategy is proposed for grass coverage computation based on a blackboard background. On a real world dataset we collected from natural environments, the proposed method was robust to varying environments, lighting, and colors. For grass detection and coverage computation, the error rate was just 3%.
Read moreA PSO and BFO-Based Learning Strategy Applied to Faster R-CNN for Object Detection in Autonomous Driving
Recently outstanding object detection results are achieved by the faster region-based convolutional network (faster R-CNN). Particularly, high-quality detection proposals are obtained by the region proposal network (RPN). Nevertheless, part of the parameters in RPN is assigned by prior knowledge. Therefore the underfitting problem is likely to appear on the training model of RPN. In other words, the generalization ability of RPN is not enough. Increasing parameters is an effective solution to this problem. Thereupon a strengthened RPN (SRPN) is designed to expand the exploring space of RPN. Acquiring the optimal parameter values of SRPN is a non-deterministic polynomial-time hard problem, which can be solved by swarm intelligence algorithms. Thereafter a particle swarm optimization (PSO) and bacterial foraging optimization (BFO)-based learning strategy (PBLS) is introduced to optimize the classifier and loss function of SRPN. In SRPN, a novel multi-level extracting network is created to improve the feature sampling ability. Moreover, the mathematical model of the smooth L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> loss function is improved to boost the fitting ability. Additionally, support vector machine (SVM) method is applied to enhance the classifier learning capability. PBLS is applied to SRPN (PBLS_SRPN). The parameters of SVM and the improved loss function are optimized by the BFO and PSO methods, respectively. Then, the performance of SRPN is further promoted. The excellent results are obtained by our proposed methods on PASCAL VOC 2007, 2012, MS COCO, and KITTI data sets. Consequently, PBLS_SRPN is effective for object detection in autonomous driving.
Read moreSmall object intelligent Detection method based on Adaptive Cascading Context
With the technology advances, deep learning-based object detection has made unprecedented progress. However, the small spatial ratio of object pixels affects the effective extraction of deep details features, resulting in poor detection results in small object detection. To improve the accuracy of small object detection, an adaptive Cascading Context small (ACC) object detection method is proposed based on YOLOv5. Firstly, a separate shallow layer feature was proposed to obtain more detailed information beneficial to small object detection. Secondly, an adaptive cascade method is proposed to fuse the output features of the three layers of the pyramid to adaptively filter negative semantic information, while fusing with shallow features to solve the problem of low classification accuracy caused by insufficient semantic information of shallow features. Finally, an adaptive context model is proposed to use a deformable convolution to obtain spatial context features of shallow small objects, associating the targets with the background, thereby improving the accuracy of small object detection. The experimental results show that the detection accuracy of the proposed method has been improved by 6.12%, 3.35%, 3.33%, and 5.2%, respectively, compared with the source code on the PASCAL VOC, NWPU VHR-10, KITTI, and RSOD datasets, which fully demonstrate the effectiveness of our method in small object detection.
Read moreCascade Object Detection and Remote Sensing Object Detection Method Based on Trainable Activation Function
Object detection is an important process in surveillance system to locate objects and it is considered as major application in computer vision. The Convolution Neural Network (CNN) based models have been developed by many researchers for object detection to achieve higher performance. However, existing models have some limitations such as overfitting problem and lower efficiency in small object detection. Object detection in remote sensing hasthe limitations of low efficiency in detecting small object and the existing methods have poor localization. Cascade Object Detection methods have been applied to increase the learning process of the detection model. In this research, the Additive Activation Function (AAF) is applied in a Faster Region based CNN (RCNN) for object detection. The proposed AAF-Faster RCNN method has the advantage of better convergence and clear bounding variance. The Fourier Series and Linear Combination of activation function are used to update the loss function. The Microsoft (MS) COCO datasets and Pascal VOC 2007/2012 are used to evaluate the performance of the AAF-Faster RCNN model. The proposed AAF-Faster RCNN is also analyzed for small object detection in the benchmark dataset. The analysis shows that the proposed AAF-Faster RCNN model has higher efficiency than state-of-art Pay Attention to Them (PAT) model in object detection. To evaluate the performance of AAF-Faster RCNN method of object detection in remote sensing, the NWPU VHR-10 remote sensing data set is used to test the proposed method. The AAF-Faster RCNN model has mean Average Precision (mAP) of 83.1% and existing PAT-SSD512 method has the 81.7%mAP in Pascal VOC 2007 dataset.
Read moreA survey and performance evaluation of deep learning methods for small object detection
A survey and performance evaluation of deep learning methods for small object detection
Rupiah Banknotes Detection Comparison of The Faster R-CNN Algorithm and YOLOv5
Money is an essential part of human life. Humans are never separated from activities related to money. As time goes by, money is not only a means of transactions between humans but also between humans and machines. Machines can recognize money in various ways, including object detection. Object detection is one of the most popular branches of computer vision. There are many methods for carrying out object detection, such as Faster R-CNN and YOLO. Faster R-CNN has been widely used in various fields to perform object detection tasks. Faster R-CNN has advantages over its predecessor because it uses a Region Proposal Network (RPN) as a substitute for selective search, which requires less compilation time. YOLO (You Only Look Once) is the most frequently used object detection method. This method divides the image into grids; each part of the grid predicts objects and their probabilities. The main advantages of YOLO are its high speed and ability to recognize objects in various conditions and positions with reasonably high accuracy. This research compares the Faster R-CNN algorithm model using the ResNet-50 architecture with YOLOv5 to recognize rupiah banknotes. The dataset used is 1120 images consisting of 8 classes. The YOLOv5 model trained on RGB data had the best results, with calculation accuracy reaching 1. Test results on three images also showed suitable results. The hope is that this research can be applied in other research to build a system for recognizing rupiah banknotes.
Read moreFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [1] and Fast R-CNN [2] have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features-using the recently popular terminology of neural networks with 'attention' mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model [3] , our detection system has a frame rate of 5 fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.
Read moreCollaborative Training Between Region Proposal Localization and Classification for Domain Adaptive Object Detection
Object detectors are usually trained with large amount of labeled data, which is expensive and labor-intensive. Pre-trained detectors applied to unlabeled dataset always suffer from the difference of dataset distribution, also called domain shift. Domain adaptation for object detection tries to adapt the detector from labeled datasets to unlabeled ones for better performance. In this paper, we are the first to reveal that the region proposal network (RPN) and region proposal classifier (RPC) in the endemic two-stage detectors (e.g., Faster RCNN) demonstrate significantly different transferability when facing large domain gap. The region classifier shows preferable performance but is limited without RPN’s high-quality proposals while simple alignment in the backbone network is not effective enough for RPN adaptation. We delve into the consistency and the difference of RPN and RPC, treat them individually and leverage high-confidence output of one as mutual guidance to train the other. Moreover, the samples with low-confidence are used for discrepancy calculation between RPN and RPC and minimax optimization. Extensive experimental results on various scenarios have demonstrated the effectiveness of our proposed method in both domain-adaptive region proposal generation and object detection. Code is available at https://github.com/GanlongZhao/CST_DA_detection.
Read moreRich Features and Precise Localization with Region Proposal Network for Object Detection
Deep Network greatly accelerates the development of object detection. Recent advances in object detection are mainly attributed to the combination of deep network and region proposal methods [1, 2, 3]. However, the accuracy of object detection on the complicated datasets is still not satisfied, especially on small object detection. This is mainly because of the coarseness of the convolution feature maps. In this paper, we design a new strategy for generating region proposals and propose a new localization method for object detection. Compared with previous baseline detectors such as Fast R-CNN [4] and Faster R-CNN [5], Our method makes use of the adjacent-level feature maps at all scales to generate region proposals and also adopts the cascaded region proposal network (RPN) to fine-tune the location of the bounding box. Compared with other state-of-the-art methods, our method achieves the best recall and object detection accuracy.
Read moreFRDet: Few‐shot object detection via feature reconstruction
State‐of‐the‐art object detection models rely on large‐scale datasets for training to achieve good precision. Without sufficient samples, the model can suffer from severe overfitting. Current explorations in few‐shot object detection are mainly divided into meta‐learning‐based methods and fine‐tuning‐based methods. However, existing models do not focus on how feature maps should be processed to present more accurate regions of interest (RoIs), leading to many non‐supporting RoIs. These non‐supporting RoIs can increase the burden of subsequent classification and even lead to misclassification. Additionally, catastrophic forgetting is inevitable in both few‐shot object detection models. Many models classify directly in low‐dimensional spaces due to insufficient resources, but this transformation of the data space can confuse some categories and lead to misclassification. To address these problems, the Feature Reconstruction Detector (FRDet) is proposed, a simple yet effective fine‐tune‐based approach for few‐shot object detection. FRDet includes a region proposal network (RPN) based on channel attention and space attention called Multi‐Attention RPN (MARPN) and a head based on feature reconstruction called Feature Reconstruction Head (FRHead). MARPN utilizes channel attention to suppress non‐supporting classes and spatial attention to enhance support classes based on Attention RPN, resulting in fewer but more accurate RoIs. Meanwhile, FRHead utilizes support features to reconstruct query RoI features through a closed‐form solution, allowing for a comprehensive and fine‐grained comparison. The model was validated on the PASCAL VOC, MS COCO, FSOD, and CUB200 datasets and achieved better results.
Read moreSmall Object Detection with Efficient Multi-Scale Collaborative Attention and Depth Feature Fusion Based on Detection Transformer
Existing DEtection TRansformer-based (DETR) object detection methods have been widely applied to standard object detection tasks, but still face numerous challenges in detecting small objects. These methods frequently miss the fine details of small objects and fail to preserve global context, particularly under scale variation or occlusion. The resulting feature maps lack sufficient spatial and structural information. Moreover, some DETR-based models specifically designed for small object detection often have poor generalization capabilities and are difficult to adapt to datasets with diverse object scales and complex backgrounds. To address these issues, this paper proposes a novel object detection model—small object detection with efficient multi-scale collaborative attention and depth feature fusion based on DETR (ED-DETR)—which consists of three core modules: an efficient multi-scale collaborative attention mechanism (EMCA), DepthPro, a zero-shot metric monocular depth estimation model, and an adaptive feature fusion module for depth maps and feature maps. Specifically, EMCA extends the single-space attention mechanism in efficient multi-scale attention (EMA) to a composite structure of parallel spatial and channel attention, enhancing ED-DETR’s ability to express features collaboratively in both spatial and channel dimensions. DepthPro generates depth maps to extract depth information. The adaptive feature fusion module integrates depth information with RGB visual features, improving ED-DETR’s ability to perceive object position, scale, and occlusion. The experimental results show that ED-DETR achieves the current best 33.6% mAP on the AI-TOD-V2 dataset, which predominantly contains tiny objects, outperforming previous CNN-based and DETR-based methods, and shows excellent generalization performance on the VisDrone and COCO datasets.
Read moreAI-Powered Waste Detection and Localization Using Region Proposal Network and CNN
To reduce environmental pollution caused by improper waste disposal, efficient waste management solutions are essential. Traditional methods rely on manual monitoring, which is time-consuming and ineffective. Advancements in artificial intelligence and deep learning enable automated waste detection and localization, improving efficiency in waste management. Faster R-CNN, an advanced object detection model, enhances precision in recognizing and localizing waste in visual data. By utilizing deep learning techniques, waste detection achieves high accuracy, reducing dependence on manual inspections.The integration of Faster R-CNN with image processing techniques allows for real-time identification of waste objects in complex environments. Localization is achieved by drawing bounding boxes around detected waste, providing precise positional information. The Region Proposal Network (RPN) plays a key role in this process by generating candidate object regions through anchor boxes and refining them based on object scores. Tensor Flow’s Object Detection API facilitates model training and optimization, ensuring accurate recognition of waste materials. High-performance convolutional neural networks enhance feature extraction, distinguishing waste from non-waste objects effectively. The use of deep learning in image-based waste detection supports scalable waste management solutions.Automated waste detection and localization contribute to sustainable urban development by enabling proactive waste management strategies. The ability to accurately identify and mark waste locations enhances the efficiency of cleaning processes and reduces environmental hazards. By analyzing waste distribution patterns, authorities can optimize resource allocation and improve waste disposal planning. The adoption of AI-driven approaches in waste detection minimizes human effort while promoting a cleaner and healthier environment. Key Words: AI-Driven waste detection, Faster R-CNN, Region Proposal Network, TensorFlow object detection API, Image processing and Localization.
Read moreDual-stage feature specialization network for robust visual object detection in autonomous vehicles
Efficient feature representation is critical for accurate visual perception in autonomous vehicles. Existing two-stage object detection methods often suffer from feature interference between candidate region generation and classification regression tasks, leading to suboptimal performance in complex scenes. To address this, we propose a Dual-Stage Feature Specialization Network (DSFSN) that decouples feature extraction: MobileNetV3 is employed for lightweight candidate region generation, while ResNet-FPN enhances multi-scale feature fusion for precise classification. Extensive experiments on PASCAL VOC and MS COCO datasets demonstrate state-of-the-art performance, achieving 81.6% mAP (9.3% higher than Faster R-CNN) and 29.3% AP on MS COCO, with a 14.9% improvement in small object detection. Real-world tests under diverse conditions (e.g., rain, night) validate the robustness of our method for autonomous driving applications. This work provides a novel framework for balancing accuracy and efficiency in visual perception systems.
Read moreA Small Object Real-Time Detection Method for Power Line Inspection in Low-Illuminance Environments
Power inspection in low-illuminance environments is of great significance for ensuring the all-weather stable operation of the power system. However, low visibility at night seriously interferes with the detection performance of small-sized power devices. In response to the issue, we propose a small object real-time detection method for power line inspection in low-illuminance environments. We design an adaptive transformer-ISP (ATISP) module, in which the optimal parameter regression module generates hyperparameters by sensing input image features to guide the image signal processors (ISPs) to perform image enhancement. With the advantage of ISPs, the ATISP has the advantages of fast inference speed and less training cost. Furthermore, the optimal parameter regression module extracts local features and long-distance dependencies through CNN and Transformer to be able to more fully perceive the input image, so that the generated hyperparameters better enhance image defects. In addition, we use lightweight neural network MobileNetv3 to improve YOLOv7, so that the algorithm maintains excellent small object detection performance while significantly increasing the detection speed. Moreover, the integrated model optimisation uses only the object detection loss functions, which allows ATISP to perform image enhancement just according to the object detection needs, improving small object detection effect and shortening the inference time of ATISP. In extensive experiments, compared with 9 state-of-the-art object detection algorithms, our algorithm has the best small-scale insulator faults detection precision (mAP:75.38 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> ) in our DIFE, best small object detection precision (mAP:56.31 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> ) in public dataset Exdark, and faster detection speed (FPS:98.81 and 97.53), which prove our method can achieve fast and accurate low-illuminance insulators detection.
Read more