CNN Feature Boosted SeqSLAM for Real‐Time Loop Closure Detection

Disaster damage detection through synergistic use of deep learning and 3D point cloud features derived from very high resolution oblique aerial images, and multiple-kernel-learning

Recent works about convolutional neural networks (CNN) show breakthrough performance on various tasks. However, most of them only use the features extracted from the topmost layer of CNN instead of leveraging the features extracted from different layers. As the first group which explicitly addresses utilizing the features from different layers of CNN, we propose cross-layer CNN features which consist of the features extracted from multiple layers of CNN. Our experimental results show that our proposed cross-layer CNN features outperform not only the state-of-the-art results but also the features commonly used in the traditional CNN framework on three tasks — artistic style, artist, and architectural style classification. As shown by the experimental results, our proposed cross-layer CNN features achieve the best known performance on the three tasks in different domains, which makes our proposed cross-layer CNN features promising solutions for generic tasks.

Automatic target recognition has been one of the hottest research in synthetic aperture radar (SAR) data processing. Noticing that popular recognition methods cannot utilize multiple features of SAR complex data, a method fused scattering center feature and deep convolutional neural network (CNN) feature is proposed in this letter. This method contains three key parts, namely, scattering center extraction and reconstruction block, CNN feature extraction block, and final feature fusion and classification block. In this process, the scattering center feature and CNN feature are fused at the level of feature maps, which retain the space information of 2-D feature maps. What is more, the proposed half end-to-end strategy realizes the automatic update of weighting parameters in feature extraction network and subnetwork, which promotes a better recognition efficiency. Experimental results on measured SAR data show that the proposed method can achieve better accuracy than other single feature-based methods and feature fusion methods.

We propose simple and effective models for the image annotation that make use of Convolutional Neural Network (CNN) features extracted from an image and word embedding vectors to represent their associated tags. Our first set of models is based on the Canonical Correlation Analysis (CCA) framework that helps in modeling both views - visual features (CNN feature) and textual features (word embedding vectors) of the data. Results on all three variants of the CCA models, namely linear CCA, kernel CCA and CCA with k-nearest neighbor (CCA-KNN) clustering, are reported. The best results are obtained using CCA-KNN which outperforms previous results on the Corel-5k and the ESP-Game datasets and achieves comparable results on the IAPRTC-12 dataset. In our experiments we evaluate CNN features in the existing models which bring out the advantages of it over dozens of handcrafted features. We also demonstrate that word embedding vectors perform better than binary vectors as a representation of the tags associated with an image. In addition we compare the CCA model to a simple CNN based linear regression model, which allows the CNN layers to be trained using back-propagation.

Robust RGB-D tracking via compact CNN features

In this work, we present an approach based on multilevel local as well as global Convolutional Neural Network (CNN) feature matching to retrieve near duplicate images. CNN features are suitable for visual matching. The CNN features of entire image may not give accuracy in retrieval due to various image editing/capturing operations. Our retrieval task focuses on matching image pairs based on local and global levels. In local matching, an image is segmented into fixed size blocks followed by extracting patches by considering neighboring regions at different levels. Matching local image patches at different levels provides robustness to our retrieval model. In local patch extraction, we select blocks containing SURF feature points instead of selecting all blocks. CNN features are extracted and stored for each image patch and then followed by extraction of global CNN features. Finally, similarity between image pairs is computed by considering all extracted CNN features. Our similarity function is based on correlation and number of blocks found in matching. We implemented our proposed approach on benchmarking Holiday dataset. Retrieval results show remarkable improvement in mean average precision (mAP) on the dataset.

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI.

Convolution Neural Network (CNN) features have been widely used in visual tracking due to their powerful representation. As an important component of CNN, the pooling layer plays a critical role, but the max/average/min operation only explores the first-order information, which limits the discrimination ability of the CNN features in some complex situations. In this paper, a high-order pooling layer is integrated into the VGG16 network for visual tracking. In detail, a high-order covariance pooling layer is employed to replace the last maxpooling layer to learn discrimination features and is trained on the ImageNet and CUB200-2011 data sets. In tracking stage, the multiple levels of feature maps are extracted as the appearance representation of the target. After that, the extracted CNN features are integrated into the correlation filters framework when tracking is on-the-fly. The experimental results show that the proposed algorithm achieves excellent performance in both success rate and tracking accuracy.

Can classification of graphic novel illustrators be achieved by convolutional neural network (CNN) features evolved for classifying concepts on photographs? Assuming that basic features at lower network levels generically represent invariants of our environment, they should be reusable. However, features at what level of abstraction are characteristic of illustrator style? We tested transfer learning by classifying roughly 50,000 digitized pages from about 200 comic books of the Graphic Narrative Corpus (GNC, [6]) by illustrator. For comparison, we also classified Manga109 [18] by book. We tested the predictability of visual features by experimentally varying which of the mixed layers of Inception V3 [29] was used to train classifiers. Overall, the top-1 test-set classification accuracy in the artist attribution analysis increased from 92% for mixed-layer 0 to over 97% when adding mixed-layers higher in the hierarchy. Above mixed-layer 5, there were signs of overfitting, suggesting that texture-like mid-level vision features were sufficient. Experiments varying input material show that page layout and coloring scheme are important contributors. Thus, stylistic classification of comics artists is possible re-using pre-trained CNN features, given only a limited amount of additional training material. We propose that CNN features are general enough to provide the foundation of a visual stylometry, potentially useful for comparative art history.

Robust fusion for RGB-D tracking using CNN features

Deep convolutional neural networks (CNNs) have been widely and successfully applied in many computer vision tasks, such as classification, detection, semantic segmentation, and so on. As for image retrieval, while off-the-shelf CNN features from models trained for classification task are demonstrated promising, it remains a challenge to learn specific features oriented for instance retrieval. Witnessing the great success of low-level SIFT feature in image retrieval and its complementary nature to the semantic-aware CNN feature, in this paper, we propose to embed the SIFT feature into the CNN feature with a Siamese structure in a learning-based paradigm. The learning objective consists of two kinds of loss, i.e., similarity loss and fidelity loss. The first loss embeds the image-level nearest neighborhood structure with the SIFT feature into CNN feature learning, while the second loss imposes that the CNN feature with the updated CNN model preserves the fidelity of that from the original CNN model solely trained for classification. After the learning, the generated CNN feature inherits the property of the SIFT feature, which is well oriented for image retrieval. We evaluate our approach on the public data sets, and comprehensive experiments demonstrate the effectiveness of the proposed method.

Perceptual metrics based on features of deep Convolutional Neural Networks (CNNs) have shown remarkable success when used as loss functions in a range of computer vision problems and significantly outperform classical losses such as L1 or L2 in pixel space. The source of this success remains somewhat mysterious, especially since a good loss does not require a particular CNN architecture nor a particular training method. In this paper we show that similar success can be achieved even with losses based on features of a deep CNN with random filters. We use the tool of infinite CNNs to derive an analytical form for perceptual similarity in such CNNs, and prove that the perceptual distance between two images is equivalent to the maximum mean discrepancy (MMD) distance between local distributions of small patches in the two images. We use this equivalence to propose a simple metric for comparing two images which directly computes the MMD between local distributions of patches in the two images. Our proposed metric is simple to understand, requires no deep networks, and gives comparable performance to perceptual metrics in a range of computer vision tasks.

The Neural Network (NN) with Rectified Linear Units (ReLU), has achieved a big success for image classification with large number of labelled training samples. The performance however is unclear when the number of labelled training samples is limited and the size of samples is large. Usually, the Convolutional Neural Network (CNN) is used to process the large-size images, but the unsupervised pre-training method for deep CNN is still progressing slowly. Therefore, in this paper, we first explore the ability of denoising auto-encoder with ReLU for pre-training CNN layer-by-layer, and then investigate the performance of CNN with weight initialized by the pre-trained features for image classification tasks, where the number of training samples is limited and the size of samples is large. Experiments on Caltech-101 benchmark demonstrate the effectiveness of our method.

Face verification for on line application is a difficult problem and many researchers have tried to solve it by convolutional neural network. Among of them, most works used the last-hidden layer as the feature of face, and abandoned the features in the lower layers which indicate local information. To remedy this, we extract features of all layers in the convolutional neural network, and fuse these features together after dimensionality reduction with PCA. Then these features are utilized for face verification with neural network classifier. Experiment results show that complete features can improve the verification rate effectively than using the last-hidden layer only.