SFGNet detecting objects via spatial fine-grained feature and enhanced RPN with spatial context

Zero-shot object detection aims to detect and recognize objects unobserved in training samples from images. Previous studies generally utilized concept names or textual descriptions to build relationships between seen and unseen classes. However, these works rarely exploited the valuable information in textual descriptions for optimizing the network. Actually, textual descriptions contain much valuable information related to categories. Exploiting this information can help training the network and improve the detection performance. Besides, textual descriptions usually contain the names of objects that need to be detected. By using this character, we can narrow the scope of candidate unseen categories, thus can improve the detection accuracy. In this regard, we propose a novel framework that incorporates both images and their text descriptions for zero-shot object detection. In particular, we employ text convolutional neural network (CNN) and Faster R-CNN to extract text features and image features respectively, and combine them to optimize the regions that contain objects in images and to classify those newly detected objects simultaneously. Besides, we try extracting potential object labels directly from textual descriptions and introducing online hard example mining (OHEM) to assist with object classification and network optimization. Our extensive experiments on two public datasets demonstrate the superior performance of our approach to state-of-the-art methods.

Pulmonary nodule detection in low-dose computed tomography (CT) images is essential for early screening and treatment of lung cancer. Previous related researches based on deep convolutional neural networks generally rely on 2D or 2.5D components and only focus on the output feature information under a single receptive field. Considering the 3D nature of lung CT images and the performance limitation of state-of-the-art nodule detection methods, we develop a novel 3D multi-branch region proposal network with an encoder-decoder structure. Specifically, each parallel branch is designed with 3D residual blocks and U-Net-like structure to effectively extract multi-scale fusion features based on 3D spatial information of CT scans, and the strategies of varying receptive fields and sharing weight parameters are used to improve the sensitivity of the detection network to nodules with scale variation and maintain the original parameters. Besides, we propose a multi-scale attentional feature fusion module to better fuse high-resolution and semantically strong features and adaptively learn the inter-dependency information of different feature maps. Finally, we compare a dynamically scaled cross entropy loss and online hard example mining (OHEM) to combat the imbalance of positive and negative samples during training, which is aimed at assisting with network optimization. Our extensive experiments on publicly available CT scans obtained from LUNA16 and TianChi1competition dataset demonstrate that our method outperform state-of-the-art pulmonary nodule detection models.

In recent years, image-based automatic insect target detection technology has been developed in the field of insect target detection. Traditional insect target detection is mainly artificial identification, but in order to avoid the problem of low detection accuracy caused by subjective factors, using convolutional neural network to extract features automatically and using the deep learning model to detect insect targets. In addition, we improve the model from the following two aspects: On the one hand, because most of insect data sets we collected are taken in the field, the background of the data sets is very complex and the image resolution is not high. For this reason, we replace the basic network VGG16 of the model with ResNet50 with a deeper layer of network structure and fewer parameters. On the other hand, we use OHEM (online hard example mining) to solve the imbalance between the target frame and background frame in target detection. The results show that the accuracy of the improved Faster-RCNN model is 89.64, which is 4.31% higher than that of the non improved Faster-RCNN model.

Existing tracking-by-detection approaches using deep features have achieved promising results in recent years. However, these methods mainly exploit feature representations learned from individual static frames, thus paying little attention to the temporal smoothness between frames. This easily leads trackers to drift in the presence of large appearance variations and occlusions. To address this issue, we propose a two-stream network to learn discriminative spatio-temporal feature representations to represent the target objects. The proposed network consists of a Spatial ConvNet module and a Temporal ConvNet module. Specifically, the Spatial ConvNet adopts 2D convolutions to encode the target-specific appearance in static frames, while the Temporal ConvNet models the temporal appearance variations using 3D convolutions and learns consistent temporal patterns in a short video clip. Then we propose a proposal refinement module to adjust the predicted bounding box, which can make the target localizing outputs to be more consistent in video sequences. In addition, to improve the model adaptation during online update, we propose a contrastive online hard example mining (OHEM) strategy, which selects hard negative samples and enforces them to be embedded in a more discriminative feature space. Extensive experiments conducted on the OTB, Temple Color and VOT benchmarks demonstrate that the proposed algorithm performs favorably against the state-of-the-art methods.

In this paper, we design an intelligent student behavior analysis system for recorded classrooms, which automatically detects hand-raising, standing, and sleeping behaviors of students. Detecting these behaviors is quite challenging mainly due to various scale behaviors, low resolution, and imbalanced behavior samples. To overcome the above-mentioned challenges, we first build a large-scale student behavior corpus from thirty schools, labeling these behaviors using bounding boxes frame-by-frame, which changes the behavior recognition problem into object detections. Then, we propose an improved Faster R-CNN, a classical object detection model, for student behavior analysis. Specifically, we first present a novel scale-aware detection head to overcome scale variations. Secondly, we propose a new feature fusion strategy to detect low-resolution behaviors while introduces little computation overhead. Thirdly, we utilize OHEM (Online Hard Example Mining) to alleviate severe class imbalances. Experiment results on our real corpus are increased by 3.4% mAP while maintaining a fast speed.

Intelligent analysis method of dam material gradation for asphalt-core rock-fill dam based on enhanced Cascade Mask R-CNN and GCNet

Freight car target detection plays an important role in railway traffic safety, which typically depends on artificial observation or conventional machine learning, with insufficient accuracy and high demand for an observer's physical strength and image quality. Motivated by the recent advances of the convolutional neural network in object detection, this study investigates how deep neural networks can be applied in freight car target detection to better solve the aforementioned problems. We propose a novel two-training method for freight car target detection; the method includes general training and special training. In addition, online hard example mining and deformable convolutional network are introduced to select hard examples and extract better features for the special training stage to improve the problem of tiny target detection in poor images obtained from freight car target detection. The proposed methods are verified using experimental results based on three aspects, i.e. indexes, visualization, and speed. High accuracy can be achieved with good recall and acceptable speed for freight car target detection applications. Finally, we illustrate the utility of using such a model to test high robustness for changes in image quality and other target detection tasks with slight modification.

C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks

An object detector based on convolutional neural network (CNN) has been widely used in the field of computer vision because of its simplicity and efficiency. The average accuracy of CNN model detection results in the object detector is greatly affected by the loss function. The precision of the localization algorithm in the loss function is the main factor affecting the result. Based on the complete intersection over union (CIoU) loss function, an improved penalty function is proposed to improve the localization accuracy. Specifically, the algorithm more comprehensively considers matching bounding boxes between prediction with ground truth, using the proportional relationship of the aspect ratio from both bounding boxes. Under the same aspect ratio of the two bounding boxes, the influence factors of the prediction box on localization accuracy were considered. In this way, the function of the penalty function is strengthened, and localization accuracy of the network model improved. This loss function is called Improved CIoU (ICIoU). Experiments on the Udacity, PASCAL VOC, and MS COCO datasets have demonstrated the effectiveness of ICIoU in improving localization accuracy of network models by using the one-stage object detector YOLOv4. Compared with CIoU, the proposed ICIoU improved average precision (AP) by 0.57% and AP75 by 0.12% on Udacity, AP by 0.26% and AP75 by 1.28% on PASCAL VOC, and AP by 0.06% and AP75 by 0.65% on MS COCO.

paper, we focus on exploring the relationship between weave patterns and their mechanical properties in woven fiber composites through Machine Learning. Specifically, we explore the interactions between woven architectures and in-plane stiffness properties through Deep Convolutional Neural Network (DCNN) and Generative Adversarial Network (GAN). Our research is important for exploring how woven composite’s pattern is related to its mechanical properties and accelerating woven composite design as well as optimization. We focus on two tasks: (1) Stiffness prediction: Predicting in-plane stiffness properties for given weave patterns. Our DCNN extracts high-level features through several convolutional and fully connected layers to determine the final predictions. (2) Weave pattern prediction: Predicting weave patterns for target stiffness properties, which can be treated as the reverse task of the first one. Due to many-to-one mapping between weave patterns and the composite properties, we utilize a Decoder Neural Network as our baseline model and compare its performance with GAN and Genetic Algorithm. We represent the weave patterns as 2D checkerboard models and use finite element analysis (FEA) to determine in-plane stiffness properties, which serve as input data for our ML framework. We show that: (1) for stiffness prediction, DCNN can predict stiffness values for a given weave pattern with relatively high accuracy (above 93%); (2) for weave pattern prediction, the GAN model gives the best prediction accuracy (above 92%) while Decoder Neural Network has the best time efficiency. HAOTIAN FENG

The recent rapid success of deep convolutional neural networks (CNN) on many computer vision tasks largely benefits from the well-annotated Pascal VOC, ImageNet, and MS COCO datasets. However, it is challenging to get ImageNet-like annotations (1000 classes) in the medical imaging domain due to the lack of clinical training in the lay crowdsourcing community. We address this problem by presenting a semi-supervised training method for neural networks with true-class and pseudo-class (un-annotated class) labels on partially annotated training data. The true-class labels are supervised annotations from clinical professionals. The pseudo-class labels are unsupervised clustering of un-annotated data. Our method rests upon the hypothesis of better coherent annotations with discriminative classes leading to better trained CNN models. We validated our method on extra-coronary calcification detection in low dose CT scans. The CNN trained with true-class and 10 pseudo-classes achieved an 85.0% sensitivity at 10 false positives per patient (0.3 false positive per slice), which significantly outperformed the CNN trained with true-class only (sensitivity =56.0% at 10 false positives per patient).

Image retrieval has made significant advances, fueled mainly by deep convolutional neural networks, but their training procedure is not efficient enough. Because of the large imbalance between easy examples and hard examples, networks lack direct guidance information from hard examples. In this paper, we solve the problem by developing an effective and efficient method, called mixed triplet loss with hard example feedback network (MHEF-TripNet). Since the proportion of hard examples is small, a sample selection probability matrix is introduced to select hard examples, which assists a network to focus more on enlarging the gap between the confusing categories in triplet loss. And it will be adjusted according to the feedback of test results after each training iteration. Furthermore, a mixed triplet loss function is proposed, which combines triplet loss with category loss to take advantage of association information between images and category information. The effectiveness of MHEF-TripNet is confirmed by experimentation on UC Merced Land Use and Kdelab Airplane datasets. Compared with previous image retrieval approaches, our approach obtains superior performance.

In the field of computer vision, object classification and object detection are widely used in many fields. The traditional object detection have two main problems:one is that sliding window of the regional selection strategy is high time complexity and have window redundancy. And the other one is that Robustness of the feature is not well. In order to solve those problems, Regional Proposal Network (RPN) is used to select candidate regions instead of selective search algorithm. Compared with traditional algorithms and selective search algorithms, RPN has higher efficiency and accuracy. We combine HOG feature and convolution neural network (CNN) to extract features. And we use SVM to classify. For TorontoNet, our algorithm's mAP is 1.6 percentage points higher. For OxfordNet, our algorithm's mAP is 1.3 percentage higher.

We aim to model the top-down attention of a Convolutional Neural Network (CNN) classifier for generating task-specific attention maps. Inspired by a top-down human visual attention model, we propose a new backpropagation scheme, called Excitation Backprop, to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process. Furthermore, we introduce the concept of contrastive attention to make the top-down attention maps more discriminative. In experiments, we demonstrate the accuracy and generalizability of our method in weakly supervised localization tasks on the MS COCO, PASCAL VOC07 and ImageNet datasets. The usefulness of our method is further validated in the text-to-region association task. On the Flickr30k Entities dataset, we achieve promising performance in phrase localization by leveraging the top-down attention of a CNN model that has been trained on weakly labeled web images.

Improving the precision of weakly supervised multiscale objects localization is of significant challenge in computer vision, especially when tackling the small objects. in this paper, we propose to integrate the feature pyramid network (FPN) with convolutional neural network (CNN) for weakly supervised object localization, where the FPN is built upon the outputs of different layers of the CNN. Then, we upsample the high-level maps by nearest-neighbor interpolation and fuse with the low-level maps in the FPN to produce multi-scale fused maps which features of both high resolution and strong semantics. Finally, we produce class activation maps by each layer of the FPN and gain multiple prediction scores by wildcat spatial pooling. To acquire more precise localization, we select the class activation map that corresponds to the highest score across all multiscale maps for object localization. In particular, we choose the maximum response regions of the class activation map for pointwise localization and choose the largest connected component above the threshold in the class activation map for bounding box localization. By applying the proposed strategy over PASCAL VOC dataset and MS COCO dataset, it is demonstrated that our strategy is highly effective in improving the precision of weakly supervised object localization as compared with some of the state-of-the-art weakly supervised methods.