- Research Article
10
- 10.1016/j.eswa.2018.06.043
A Multi-strategy Region Proposal Network
- Jun 28, 2018
- Expert Systems with Applications
- Yu-Peng Chen + 3 more +3
A Multi-strategy Region Proposal Network
Object detectors are usually trained with large amount of labeled data, which is expensive and labor-intensive. Pre-trained detectors applied to unlabeled dataset always suffer from the difference of dataset distribution, also called domain shift. Domain adaptation for object detection tries to adapt the detector from labeled datasets to unlabeled ones for better performance. In this paper, we are the first to reveal that the region proposal network (RPN) and region proposal classifier (RPC) in the endemic two-stage detectors (e.g., Faster RCNN) demonstrate significantly different transferability when facing large domain gap. The region classifier shows preferable performance but is limited without RPN’s high-quality proposals while simple alignment in the backbone network is not effective enough for RPN adaptation. We delve into the consistency and the difference of RPN and RPC, treat them individually and leverage high-confidence output of one as mutual guidance to train the other. Moreover, the samples with low-confidence are used for discrepancy calculation between RPN and RPC and minimax optimization. Extensive experimental results on various scenarios have demonstrated the effectiveness of our proposed method in both domain-adaptive region proposal generation and object detection. Code is available at https://github.com/GanlongZhao/CST_DA_detection.
A Multi-strategy Region Proposal Network
A Multi-strategy Region Proposal Network
An Efficient and Accurate Cross-Domain Object Detection Method Using One-Level Feature and Domain Adaptation
To improve the accuracy of cross-domain object detection, the existing unsupervised domain adaptation (UDA) object detection methods mostly use Feature Pyramid Network (FPN), multiple Region Proposal Network (RPN), and multiple domain classifier, but these methods lead to complex network structures, slow model convergence, and low detection efficiency. To solve the above problems, this paper proposes an Efficient and Accurate Cross-domain Object Detection Method Using One-level Feature and Domain Adaptation (OFDA). This method realizes one-level feature object detection through feature fusion and divide-and-conquer technology; realizes overfitting feature suppression and unsupervised domain adaptation through domain-specific suppression and domain feature alignment technology; realizes background feature suppression through the Objectness branch, which replaces the time-consuming Region Proposal Network (RPN) structure and improves the efficiency of unsupervised adaptive detection. The paper verifies the feasibility and superiority of the proposed method by the comparative experiments and ablation experiments of multiple datasets. The proposed OFDA method not only improves the efficiency of object detection, but also ensures the accuracy of cross-domain detection.
Read moreAn Enhanced Region Proposal Network for object detection using deep learning method.
Faster Region-based Convolutional Network (Faster R-CNN) is a state-of-the-art object detection method. However, the object detection effect of Faster R-CNN is not good based on the Region Proposal Network (RPN). Inspired by RPN of Faster R-CNN, we propose a novel proposal generation method called Enhanced Region Proposal Network (ERPN). Four improvements are presented in ERPN. Firstly, our proposed deconvolutional feature pyramid network (DFPN) is introduced to improve the quality of region proposals. Secondly, novel anchor boxes are designed with interspersed scales and adaptive aspect ratios. Thereafter, the capability of object localization is increased. Thirdly, a particle swarm optimization (PSO) based support vector machine (SVM), termed PSO-SVM, is developed to distinguish the positive and negative anchor boxes. Fourthly, the classification part of multi-task loss function in RPN is improved. Consequently, the effect of classification loss is strengthened. In this study, our proposed ERPN is compared with five object detection methods on both PASCAL VOC and COCO data sets. For the VGG-16 model, our ERPN obtains 78.6% mAP on VOC 2007 data set, 74.4% mAP on VOC 2012 data set and 31.7% on COCO data set. The performance of ERPN is the best among the comparison object detection methods. Furthermore, the detection speed of ERPN is 5.8 fps. Additionally, ERPN obtains good effect on small object detection.
Read moreGeometry-Insensitive RPN Prototypes for Domain Adaptive 3D Object Detection
The region proposal network (RPN) plays a critical role in object detection for a two-stage domain adaptive 3D object detector. However, current methods usually minimize the disparity between source and target domains by reducing the bias in intrinsic geometric information or by undertaking feature alignment according to the geometric disparity, but ignore the transferability of RPN related features and neglect the discriminability between foreground and background, resulting in generating low-quality RPN proposals. Thus we propose a novel domain adaptation method to distinguish the discriminability between foreground and background. It could implicitly avoid the geometric disparity of objects in feature alignment. Specifically, we first construct learnable and geometry-insensitive foreground RPN prototype and background RPN prototype. Then we enforce the foreground RPN features and background RPN features to align with the foreground RPN prototype and background RPN prototype, respectively. By this way, the distributional discrepancy is effectively decreased and the adaptability is promoted for existing 3D detectors. We demonstrate that our approach achieves promising results compared with other domain adaptation works on multiple cross-domain detection scenarios.
Read moreFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks.
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [1] and Fast R-CNN [2] have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully convolutional network that simultaneously predicts object bounds and objectness scores at each position. The RPN is trained end-to-end to generate high-quality region proposals, which are used by Fast R-CNN for detection. We further merge RPN and Fast R-CNN into a single network by sharing their convolutional features-using the recently popular terminology of neural networks with 'attention' mechanisms, the RPN component tells the unified network where to look. For the very deep VGG-16 model [3] , our detection system has a frame rate of 5 fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007, 2012, and MS COCO datasets with only 300 proposals per image. In ILSVRC and COCO 2015 competitions, Faster R-CNN and RPN are the foundations of the 1st-place winning entries in several tracks. Code has been made publicly available.
Read moreFaster ILOD: Incremental learning for object detectors based on faster RCNN
Faster ILOD: Incremental learning for object detectors based on faster RCNN
Rupiah Banknotes Detection Comparison of The Faster R-CNN Algorithm and YOLOv5
Money is an essential part of human life. Humans are never separated from activities related to money. As time goes by, money is not only a means of transactions between humans but also between humans and machines. Machines can recognize money in various ways, including object detection. Object detection is one of the most popular branches of computer vision. There are many methods for carrying out object detection, such as Faster R-CNN and YOLO. Faster R-CNN has been widely used in various fields to perform object detection tasks. Faster R-CNN has advantages over its predecessor because it uses a Region Proposal Network (RPN) as a substitute for selective search, which requires less compilation time. YOLO (You Only Look Once) is the most frequently used object detection method. This method divides the image into grids; each part of the grid predicts objects and their probabilities. The main advantages of YOLO are its high speed and ability to recognize objects in various conditions and positions with reasonably high accuracy. This research compares the Faster R-CNN algorithm model using the ResNet-50 architecture with YOLOv5 to recognize rupiah banknotes. The dataset used is 1120 images consisting of 8 classes. The YOLOv5 model trained on RGB data had the best results, with calculation accuracy reaching 1. Test results on three images also showed suitable results. The hope is that this research can be applied in other research to build a system for recognizing rupiah banknotes.
Read moreA study on object detection method from manga images using CNN
Japanese comics (manga) are popular content worldwide. In order to acquire metadata from manga images, techniques automatic recognition of manga content have been studied. Recently, Convolutional Neural Network (CNN) has been applied to object detection in manga images. R-CNN and Fast R-CNN generate region proposals by Selective Search. Faster R-CNN generates them using CNN layers called Region Proposal Network (RPN). Single Shot MultiBox Detector (SSD), the latest detection method, performs object classification and box adjustment for small regions in an image. These methods are effective to natural images. However, it is unclear whether such methods work properly to manga images or not, since those image features are different from natural images. In this paper, we examine the effectiveness of manga object detection by comparing Fast R-CNN, Faster R-CNN, and SSD. Here, manga objects are panel layout, speech balloon, character face, and text. Experimental results show that Fast R-CNN is effective for panel layout and speech balloon, whereas Faster R-CNN is effective for character face and text.
Read moreA Feature Decomposition-Based Method for Automatic Ship Detection Crossing Different Satellite SAR Images
In the face of Synthetic Aperture Radar (SAR) image object detection with different distributions of training and test data, traditional supervised learning methods cannot achieve good detection performance. Domain adaptation (DA) method has been shown to have the ability to solve this problem, but existing DA object detection algorithms all use adversarial DA theory for the detection task, which is ineffective in solving object regression localization in the detection task. In this article, to better solve the above problem, an automatic SAR image ship detection method based on feature decomposition crossing different satellites is proposed. The feature extraction layer of backbone network is divided into low level and high level, where domain-invariant feature extractors are designed for the local features extracted from the low level and the global features extracted from the high level, respectively. We argue that the local and global features extracted from source domain and target domain contain domain-specific features (DSF) for adversarial DA and domain-invariant features (DIF) that contribute to object regression localization. Then, we decompose the local features and global features into DSF and DIF via vector decomposition method. For DSF counterpart, we introduce adversarial DA attention for feature alignment. DIF from the local features are fused into the backbone network for high-level global feature extraction. Finally, by using region proposal network and adversarial domain classifier, we can get the accurate bounding box and object class of SAR image objects. Extensive experiments prove that the proposed method outperforms state-of-the-art methods in terms of detection performance.
Read moreBackNet: An Enhanced Backbone Network for Accurate Detection of Objects with Large Scale Variations
Deep Convolutional Neural Networks (CNNs) have induced significant progress in the field of computer vision including object detection and classification. Two-stage detectors like Faster RCNN and its variants are found to be more accurate compared to its one-stage counter-parts. Faster RCNN combines an ImageNet pretrained backbone network (e.g VGG16) and a Region Proposal Network (RPN) for object detection. Although Faster RCNN performs well on medium and large scale objects, detecting smaller ones with high accuracy while maintaining stable performance on larger objects still remains a challenging task. In this work, we focus on designing a robust backbone network for Faster RCNN that is capable of detecting objects with large variations in scale. Considering the difficulties posed by small objects, our aim is to design a backbone network that allows signals extracted from small objects to be propagated right through to the deepest layers of the network. This being our motivation, we propose a robust network: BackNet, which can be integrated as a backbone into any two-stage detector. We evaluate the performance of BackNet-Faster RCNN on MS COCO dataset and show that the proposed method outperforms five contemporary methods.
Read moreObject Detection Based on Faster R-CNN Algorithm with Skip Pooling and Fusion of Contextual Information.
Deep learning is currently the mainstream method of object detection. Faster region-based convolutional neural network (Faster R-CNN) has a pivotal position in deep learning. It has impressive detection effects in ordinary scenes. However, under special conditions, there can still be unsatisfactory detection performance, such as the object having problems like occlusion, deformation, or small size. This paper proposes a novel and improved algorithm based on the Faster R-CNN framework combined with the Faster R-CNN algorithm with skip pooling and fusion of contextual information. This algorithm can improve the detection performance under special conditions on the basis of Faster R-CNN. The improvement mainly has three parts: The first part adds a context information feature extraction model after the conv5_3 of the convolutional layer; the second part adds skip pooling so that the former can fully obtain the contextual information of the object, especially for situations where the object is occluded and deformed; and the third part replaces the region proposal network (RPN) with a more efficient guided anchor RPN (GA-RPN), which can maintain the recall rate while improving the detection performance. The latter can obtain more detailed information from different feature layers of the deep neural network algorithm, and is especially aimed at scenes with small objects. Compared with Faster R-CNN, you only look once series (such as: YOLOv3), single shot detector (such as: SSD512), and other object detection algorithms, the algorithm proposed in this paper has an average improvement of 6.857% on the mean average precision (mAP) evaluation index while maintaining a certain recall rate. This strongly proves that the proposed method has higher detection rate and detection efficiency in this case.
Read moreDomain Adaptive Faster R-CNN for Object Detection in the Wild
Object detection typically assumes that training and test data are drawn from an identical distribution, which, however, does not always hold in practice. Such a distribution mismatch will lead to a significant performance drop. In this work, we aim to improve the cross-domain robustness of object detection. We tackle the domain shift on two levels: (1) the image-level shift, such as image style, illumination, etc., and (2) the instance-level shift, such as object appearance, size, etc. We build our approach based on the recent state-of-the-art Faster R-CNN model, and design two domain adaptation components, on image level and instance level, to reduce the domain discrepancy. The two domain adaptation components are based on <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> -divergence theory, and are implemented by learning a domain classifier in adversarial training manner. The domain classifiers on different levels are further reinforced with a consistency regularization to learn a domain-invariant region proposal network (RPN) in the Faster R-CNN model. We evaluate our newly proposed approach using multiple datasets including Cityscapes, KITTI, SIM10K, etc. The results demonstrate the effectiveness of our proposed approach for robust object detection in various domain shift scenarios.
Read moreAI-Powered Waste Detection and Localization Using Region Proposal Network and CNN
To reduce environmental pollution caused by improper waste disposal, efficient waste management solutions are essential. Traditional methods rely on manual monitoring, which is time-consuming and ineffective. Advancements in artificial intelligence and deep learning enable automated waste detection and localization, improving efficiency in waste management. Faster R-CNN, an advanced object detection model, enhances precision in recognizing and localizing waste in visual data. By utilizing deep learning techniques, waste detection achieves high accuracy, reducing dependence on manual inspections.The integration of Faster R-CNN with image processing techniques allows for real-time identification of waste objects in complex environments. Localization is achieved by drawing bounding boxes around detected waste, providing precise positional information. The Region Proposal Network (RPN) plays a key role in this process by generating candidate object regions through anchor boxes and refining them based on object scores. Tensor Flow’s Object Detection API facilitates model training and optimization, ensuring accurate recognition of waste materials. High-performance convolutional neural networks enhance feature extraction, distinguishing waste from non-waste objects effectively. The use of deep learning in image-based waste detection supports scalable waste management solutions.Automated waste detection and localization contribute to sustainable urban development by enabling proactive waste management strategies. The ability to accurately identify and mark waste locations enhances the efficiency of cleaning processes and reduces environmental hazards. By analyzing waste distribution patterns, authorities can optimize resource allocation and improve waste disposal planning. The adoption of AI-driven approaches in waste detection minimizes human effort while promoting a cleaner and healthier environment. Key Words: AI-Driven waste detection, Faster R-CNN, Region Proposal Network, TensorFlow object detection API, Image processing and Localization.
Read moreRefine pedestrian detections by referring to features in different ways
The performance of object detection has been improved as the success of deep architectures. The main algorithm predominantly used for general detection is Faster R-CNN because of their high accuracy and fast inference time. In pedestrian detection, Region Proposal Network (RPN) itself which is used for region proposals in Faster R-CNN can be used as a pedestrian detector. Also, RPN even shows better performance than Faster R-CNN for pedestrian detection. However, RPN generates severe false positives such as high score backgrounds and double detections because it does not have downstream classifier. From this observations, we made a network to refine results generated from the RPN. Our Refinement Network refers to the feature maps of the RPN and trains the network to rescore severe false positives. Also, we found that different type of feature referencing method is crucial for improving performance. Our network showed better accuracy than RPN with almost same speed on Caltech Pedestrian Detection benchmark.
Read moreDeep learning for small object detection in images
[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] With the rapid development of deep learning in computer vision, especially deep convolutional neural networks (CNNs), significant advances have been made in recent years on object recognition and detection in images. Highly accurate detection results have been achieved for large objects, whereas detection accuracy on small objects remains to be low. This dissertation focuses on investigating deep learning methods for small object detection in images and proposing new methods with improved performance. First, we conducted a comprehensive review of existing deep learning methods for small object detections, in which we summarized and categorized major techniques and models, identified major challenges, and listed some future research directions. Existing techniques were categorized into using contextual information, combining multiple feature maps, creating sufficient positive examples, and balancing foreground and background examples. Methods developed in four related areas, generic object detection, face detection, object detection in aerial imagery, and segmentation, were summarized and compared. In addition, the performances of several leading deep learning methods for small object detection, including YOLOv3, Faster R-CNN, and SSD, were evaluated based on three large benchmark image datasets of small objects. Experimental results showed that Faster R-CNN performed the best, while YOLOv3 was a close second. Furthermore, a new deep learning method, called Retina-context Net, was proposed and outperformed state-of-the art one-stage deep learning models, including SSD, YOLOv3 and RetinaNet, on the COCO and SUN benchmark datasets. Secondly, we created a new dataset for bird detection, called Little Birds in Aerial Imagery (LBAI), from real-life aerial imagery. LBAI contains birds with sizes ranging from 10 by 10 pixels to 40 by 40 pixels. We adapted and applied several state-of-the-art deep learning models to LBAI, including object detection models such as YOLOv2, SSH, and Tiny Face, and instance segmentation models such as U-Net and Mask R-CNN. Our empirical results illustrated the strength and weakness of these methods, showing that SSH performed the best for easy cases, whereas Tiny Face performed the best for hard cases with cluttered backgrounds. Among small instance segmentation methods, U-Net achieved slightly better performance than Mask R-CNN. Thirdly, we proposed a new graph neural network-based object detection algorithm, called GODM, to take the spatial information of candidate objects into consideration in small object detection. Instead of detecting small objects independently as the existing deep learning methods do, GODM treats the candidate bounding boxes generated by existing object detectors as nodes and creates edges based on the spatial or semantic relationship between the candidate bounding boxes. GODM contains four major components: node feature generation, graph generation, node class labelling, and graph convolutional neural network model. Several graph generation methods were proposed. Experimental results on the LBDA dataset show that GODM outperformed existing state-of-the-art object detector Faster R-CNN significantly, up to 12% better in accuracy. Finally, we proposed a new computer vision-based grass analysis using machine learning. To deal with the variation of lighting condition, a two-stage segmentation strategy is proposed for grass coverage computation based on a blackboard background. On a real world dataset we collected from natural environments, the proposed method was robust to varying environments, lighting, and colors. For grass detection and coverage computation, the error rate was just 3%.
Read more