Region proposal and object detection using HoG-based CNN feature map

The convolutional neural networks (CNNs) have recently demonstrated to be a powerful tool for object detection. However, with the complex scenes in remote sensing images, feature extraction of the object in the CNN will be seriously affected by background information. To address this issue, in this article, a region-enhanced CNN (RECNN) is proposed for the object detection of remote sensing images. The RECNN introduces the saliency constraint and multilayer fusion strategy into the CNN model, which can effectively enhance the object regions for better detection. Specifically, the saliency map is extracted and utilized to guide the training of the proposed model to strengthen saliency regions in feature maps. In addition, since different layers can reflect the object regions in varied resolutions, a multilayer fusion strategy is introduced to connect different convolutional layers and explore the context, where the feature maps of object regions are further enhanced. Experimental results on a publicly available ten-class object detection data set demonstrate the superiority of the RECNN over several competitive object detection methods.

One of the most important and challenging research topics in the area of computer vision is visual object tracking, which is relevant to many real-world applications. Recently, discriminative correlation filters (DCF) have been demonstrated to overcome the problems in visual object tracking efficiently. So far, only single-resolution feature maps have been utilized in DCF. Owing to this limitation, the potential of DCF has not been exploited. Moreover, convolutional features have demonstrated a better performance for visual tracking than histogram of oriented gradients (HOG) features and color features. Based on these facts, in this paper, we propose collaborative learning based on multi-resolution feature maps for DCF, employing convolutional features. Further, the confidence score, which represents the location of the target object, is selected from various candidates based on certain rules. In addition, the continuous filters are trained to handle the variations of appearance of the target. The extensive experimental results obtained using VOT2015 and OTB-100 benchmark datasets show that the proposed algorithm performs favorably against state-of-the-art tracking algorithms.

In order to detect noncooperative objects with unknown structures, template based matching approaches are generally adopted. They rely on a large set of manually-selected templates and slide them over the image to determine the potential locations of objects. The process is exhaustive and computationally inefficient. In this paper, we propose a novel object detection algorithm using improved features of histogram of oriented gradients (HOG) to reduce the search region of potential objects regardless of their prior information. Firstly, we improve the HOG descriptor to make it more discriminative. The capability of detecting objects comes from positive and negative features of the training dataset. Then, the cascaded support vector machine is used to train the model, aiming at selecting proposals with higher scores at each scale and aspect ratio. Lastly, the best proposal over all scales is chosen as the object detection region. Further experiments demonstrate that our method improves favorably the detection rate on VOC 2007 and achieves satisfying performance in satellite bracket detection.

DCAM: Disturbed class activation maps for weakly supervised semantic segmentation

Object detection has received significant interest in the research field of computer vision and is widely used in human-centric applications. The occlusion problem is a frequent obstacle that degrades detection quality. In this paper, we propose a novel object detection framework targeting robust object detection in occlusion. The proposed deep learning-based network consists mainly of two parts: 1) object detection framework, which classifies the object categories and localizes the object location and 2) plug-in bounding-box (BB) estimator, which estimates the object and occlusion region from the feature map of the backbone network and the corresponding critic network for evaluating the predicted BB map. The BB estimator and the critic network are the plug-in modules added to the object detection framework and learned competitively with adversarial manner. As the plug-in BB estimator is learned to estimate the BB map containing the object and occlusion pattern information, the backbone network can embed this information to enable robust detection under occlusion in the test phase. The comprehensive experimental results on the PASCAL VOC, MS COCO, and KITTI dataset showed that the performance is improved with the plug-in BB-Critic network by predicting and criticizing object and occlusion in general generic object detection framework.

[ACCESS RESTRICTED TO THE UNIVERSITY OF MISSOURI AT REQUEST OF AUTHOR.] Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple images for each detection, and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of images per study. The task of converting images to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g. camera malfunction or moving vegetation) or pictures of humans. We offer the first widely available computer vision tool for processing camera trap images. Our results show that the tool is accurate and results in substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps. In this dissertation, we have developed new image/video processing and computer vision algorithms for efficient and accurate object detection and sequence-level classiffication from natural scene camera-trap images. This work addresses the following five major tasks: (1) Human-animal detection. We develop a fast and accurate scheme for human-animal detection from highly cluttered camera-trap images using joint background modeling and deep learning classification. Specifically, first, We develop an effective background modeling and subtraction scheme to generate region proposals for the foreground objects. We then develop a cross-frame image patch verification to reduce the number of foreground object proposals. Finally, We perform complexity-accuracy analysis of deep convolutional neural networks (DCNN) to develop a fast deep learning classification scheme to classify these region proposals into three categories: human, animals, and background patches. The optimized DCNN is able to maintain high level of accuracy while reducing the computational complexity by 14 times. Our experimental results demonstrate that the proposed method outperforms existing methods on the camera-trap dataset. (2) Object segmentation from natural scene. We first design and train a fast DCNN for animal-human-background object classification, which is used to analyze the input image to generate multi-layer feature maps, representing the responses of different image regions to the animal-human-background classifier. From these feature maps, we construct the so-called deep objectness graph for accurate animal-human object segmentation with graph cut. The segmented object regions from each image in the sequence are then verfied and fused in the temporal domain using background modeling. Our experimental results demonstrate that our proposed method outperforms existing state-of-the-art methods on the camera-trap dataset with highly cluttered natural scenes. (3) DCNN domain background modeling. We replaced the background model with a new more efficient deep learning based model. The input frames are segmented into regions through the deep objectness graph then the region boundaries of the input frames are multiplied by each other to obtain the regions of movement patches. We construct the background representation using the temporal information of the co-located patches. We propose to fuse the subtraction and foreground/background pixel classiffcation of two representation : a) chromaticity and b) deep pixel information. (4) Sequence-level object classiffcation. We proposed a new method for sequence-level video recognition with application to animal species recognition from camera trap images. First, using background modeling and cross-frame patch verification, we developed a scheme to generate candidate object regions or object proposals in the spatiotemporal domain. Second, we develop a dynamic programming optimization approach to identify the best temporal subset of object proposals. Third, we aggregate and fuse the features of these selected object proposals for efficient sequence-level animal species classification.

Object detection and tracking under complex environment are challenging because of the disturbances induced by background clutter, illumination changes, occlusions and other factors. The bulk of traditional algorithms basically rely on hand‐crafted features, which are not sufficiently robust to a complex environment. Moreover, the processes of detection and tracking are separated, which leads to the overall efficiency not high. In this study, a novel local probability model (LPM)‐based mean shift (MS) algorithm is proposed to integrate object detection and tracking. The main contributions include: (i) a new framework based on the combination of LPM and MS is established for the integration of object tracking and detection. (ii) For object detection, the training and prediction of LPM are built by stacked denoising autoencoders based deep learning. (iii) For object tracking, an MS tracking algorithm leveraging LPM is modified to improve the tracking efficiency under a complex environment. Experimental results demonstrate that the proposed method is superior to the colour histograms based MS and histograms of oriented gradients based MS in terms of robustness and tracking accuracy.

The study of remote sensing image object detection has excellent research value in environmental protection and public safety. However, the performance of the detectors is unsatisfactory due to the large variability of object size and complex background noise in remote sensing images. Therefore, it is essential to improve the detection performance of the detectors. Inspired by the idea of "divide and conquer", we proposed a Multiple Receptive Field Attention(MRFA) to solve these problems and which is a plug-and-play attention method. First, we use the method of multiple receptive field feature map generation to convert the input feature map into four feature maps with different receptive fields. In this way, the small, medium, large, and immense objects in the input feature maps are "seen" in these feature maps, respectively. Then, we used the multiple attention map fusion method to focus objects of different sizes separately, which can effectively suppress noise in the background of remote sensing images. Experiments on remote sensing object detection datasets DIOR and HRRSD demonstrate that the performance of our method is better than other state-of-the-art attention modules. At the same time, the experiments on remote sensing image semantic segmentation dataset WHDLD and classification dataset AID prove the generalization and superiority of our method.

LPCL: Localized prominence contrastive learning for self-supervised dense visual pre-training

Multi-level refinement enriched feature pyramid network for object detection

Proposes an algorithm for segmentation and extracting an object region by using Gabor filters. Gabor filters are exploited to extract spatial frequency in some orientations, and not only the outputs of Gabor filters but also color information is used to construct the features at each image pixel. The criterion is devised so as to consider the similarity, the region size and the region shape factors in order to efficiently merge the features. In general, a complex object may be segmented into multiple regions. However for the purpose of detecting such a complex object, we represent the object region by the normalized cumulative histogram of features. From experimental results, it is found that the proposed algorithm is able to efficiently detect the object regions such as cars in images of usual traffic scenes.

In this paper, we present an adaptive background generation method for automatic selection of initial object region for simultaneous object detection and depth estimation using the MCA camera. We then estimate color shifting vectors (CSVs) of the extracted object's region using simplified elastic registration algorithm in the pyramid-based multi-resolution framework. Using the color shifting property of the MCA camera, the depth of the object's region is estimated from CSVs. The proposed method uses only one camera for object detection and depth estimation. Experimental results show that the MCA camera-based object tracking system can be used in various consumer surveillance systems.

Having profound implications for autonomous systems, intelligent surveillance, and human-computer interaction, object detection and classification of images are one of the principal challenges in computer vision. Using the vast benchmark datasets of PASCAL VOC and ImageNet, this study introduces a deep learning system for object recognition and classification. The model employs a hybrid structure combining Vision Transformers (ViT) and ResNet-50 for extracting global context and spatial features. On the more diversified and complex PASCAL VOC dataset, the pre-trained weights of ImageNet are transferred for the purpose of generalization and speed of convergence. By thorough testing on PASCAL VOC 2007 and 2012 datasets, the model outperformed baseline models like Faster R-CNN and YOLOv5 by 2.7% and 1.9%, respectively, in terms of mean Average Precision (mAP) at 83.4% on VOC 2007 and 82.1% on VOC 2012. With 78.9% top-1 classification accuracy, ImageNet-based training significantly improved the system discriminative ability for 1,000 object categories. Greater robustness to occlusion and class imbalance were achieved with increased data augmentation, optimized focus loss, and improved attention modules.

One-stage object detection methods are usually more computationally efficient than two-stage methods, which makes it more likely to be applied in practice. However, one-stage methods often suffer from lower detection accuracies, especially when the objects to be detected are small. In this paper, we propose a multi-flow sub-network and multiple connections for single shot detection (MSSD), which is built upon a one-stage strategy to inherit the computational efficiency and improve the detection accuracy. The multi-flow sub-network in MSSD aims to extract high quality feature maps with high spatial resolution, sufficient non-linear transformation, and multiple receptive fields, which facilitates detection of small objects in particular. In addition, MSSD uses multiple connections, including up-sampling, down-sampling, and resolution-invariant connections, to combine feature maps of different layers, which helps the model capture fine-grained details and improve feature representation. Extensive experiments on PASCAL VOC and MS COCO demonstrate that MSSD achieves competitive detection accuracy with high computational efficiency compared to state-of-the-art methods. MSSD with input size of 320\(\,\times \,\)320 achieves 80.6% mAP on VOC2007 at 45 FPS and 29.7% mAP on COCO, both with a Nvidia Titan X GPU.

Research in automated object detection has mainly addressed detection in 2-D intensity images. The best performing systems exhibit a high degree of robustness to variation in object scale, rotation and viewpoint. Depending on the context of the detection task, rotation and viewpoint can be controlled for, but estimating object scale is usually not possible without additional sensing apparatus. Objects must be searched for at all scales where they are likely to be present. Range sensors allow for direct distance measurements to be taken and the generation of range maps or point clouds that when co-registered against standard 2-D visual images create 3-D images. Knowledge of depth obviates the need to search over the entire scale space for an object, but this also allows for the object's scale itself to be incorporated into the detection scheme. Most existing object detection schemes use range only to constrain the detection search space or to discard detections that do not match the expected object scale. This paper presents a novel feature extraction and detection method that calculates scale proportionate histograms of oriented gradients (Pro-HOG) by exploiting known depth in an image. Pro-HOG is evaluated against a reference implementation of the original histograms of oriented gradients (HOG) feature extractor for detection of images of cars in the PASCAL Visual Object Classes 2007 dataset. High quality detection results are achieved on the real world Earthmine dataset using a Pro-HOG feature detector trained with the PASCAL VOC 2007 dataset. It is demonstrated that Pro-HOG's detection accuracy is comparable to HOG but at much reduced computational overhead.