Correlated Topic Vector for Scene Classification.

While deep learning (DL) has proven to be a powerful paradigm for the classification of the large-scale big image data set. Deep learning network such as CNN requires a large number of labeled samples for training the network. However, often times the labeled data are difficult, expensive, and time-consuming. In this paper, we propose a semi-supervised approach to fused fuzzy-rough C-mean clustering with convolutional neural networks (CNNs) to knowledge learning from simultaneously intra-model and inter-model relationships forming final data representation to be classified, which can be achieved by better performance. Conception behind this is to reduce the uncertainty in terms of vagueness and indiscernibility by using fuzzy-rough C-mean clustering and specifically removing noise samples by using CNN from raw data. The framework of our proposed semi-supervised approach uses all the training data with abundant unlabeled data with few labeled data to train FRCNN model. To show the effectiveness of our model, we used four benchmark large-scale image datasets and also compared it with the state-of-the-art supervised, unsupervised, and semi-supervised learning methods for image classification.

As the number of internet users increases so does the number of malicious attacks using malware. The detection of malicious code is becoming critical, and the existing approaches need to be improved. Here, we propose a feature fusion method to combine the features extracted from pre-trained AlexNet and Inception-v3 deep neural networks with features attained using segmentation-based fractal texture analysis (SFTA) of images representing the malware code. In this work, we use distinctive pre-trained models (AlexNet and Inception-V3) for feature extraction. The purpose of deep convolutional neural network (CNN) feature extraction from two models is to improve the malware classifier accuracy, because both models have characteristics and qualities to extract different features. This technique produces a fusion of features to build a multimodal representation of malicious code that can be used to classify the grayscale images, separating the malware into 25 malware classes. The features that are extracted from malware images are then classified using different variants of support vector machine (SVM), k-nearest neighbor (KNN), decision tree (DT), and other classifiers. To improve the classification results, we also adopted data augmentation based on affine image transforms. The presented method is evaluated on a Malimg malware image dataset, achieving an accuracy of 99.3%, which makes it the best among the competing approaches.

Automatic target recognition has been one of the hottest research in synthetic aperture radar (SAR) data processing. Noticing that popular recognition methods cannot utilize multiple features of SAR complex data, a method fused scattering center feature and deep convolutional neural network (CNN) feature is proposed in this letter. This method contains three key parts, namely, scattering center extraction and reconstruction block, CNN feature extraction block, and final feature fusion and classification block. In this process, the scattering center feature and CNN feature are fused at the level of feature maps, which retain the space information of 2-D feature maps. What is more, the proposed half end-to-end strategy realizes the automatic update of weighting parameters in feature extraction network and subnetwork, which promotes a better recognition efficiency. Experimental results on measured SAR data show that the proposed method can achieve better accuracy than other single feature-based methods and feature fusion methods.

We present a method to combine the Fisher vector representation and the Deep Convolutional Neural Network (DCNN) features to generate a rerpesentation, called the Fisher vector encoded DCNN (FV-DCNN) features, for unconstrained face verification. One of the key features of our method is that spatial and appearance information are simultaneously processed when learning the Gaussian mixture model to encode the DCNN features. Evaluations on two challenging verification datasets show that the proposed FV-DCNN method is able to capture the salient local features and also performs well when compared to many state-of-the-art face verification methods.

In view of the scene’s complexity and diversity in scene classification, this paper makes full use of the contextual semantic relationships between the objects to describe the visual attention regions of the scenes and combines with the deep convolution neural networks, so that a scene classification model using visual attention and deep networks is constructed. Firstly, the visual attention regions in the scene image are marked by using the context-based saliency detection algorithm. Then, the original image and the visual attention region detection image are superimposed to obtain a visual attention region enhancement image. Furthermore, the deep convolution features of the original image, the visual attention region detection image, and the visual attention region enhancement image are extracted by using the deep convolution neural networks pretrained on the large-scale scene image dataset Places. Finally, the deep visual attention features are constructed by using the multilayer deep convolution features of the deep convolution networks, and a classification model is constructed. In order to verify the effectiveness of the proposed model, the experiments are carried out on four standard scene datasets LabelMe, UIUC-Sports, Scene-15, and MIT67. The results show that the proposed model improves the performance of the classification well and has good adaptability.

Fine-grained visual categorization aims to distinguish objects in subordinate classes instead of basic class, and is a challenge visual task due to the high correlation between subordinated classes and large intra-class variation (e.g. different object poses). Although, deep convolutional neural network (DCNN) has brought dramatic success on generic object classification, detection and segmentation with the availability of the large-scale training samples, direct application of DCNN on fine-grained visual categorization, where only decades or at most hundreds of training samples for each subordinate class are available in most public finegrained image datasets, cannot lead to satisfactory classification results due to small number of training samples. This study explores the transfer learning strategy for finegrained dog breed categorization based on the learned CNN models with the large-scale image dataset: ImageNet, and prove promising performance with two DCNN models: AlexNet and VGG-16. Furthermore, we argue that different DCNN architecture may extract the representation of different image aspects due to the previously defined CNN kernel sizes, number and various operations in the model learning procedure, and thus result in different performance for visual categorization. This study proposes to fusion multiple CNN architectures for combining different aspect representations to give more accurate performance. We compressively study the fusion of different layers such as Fc6 and Fc7 in AlexNet and VGG-16, and manifest 2.88% improvement of the fusion architecture over the best performance of the only one DCNN model: VGG-16 from 81.2% to 84.08%.

In China, a variety of non-motor vehicles (NM-Vehicle) exist on the road in recent years, some are similar to pedestrian, others are similar to motor vehicle (M-vehicle) in monitor video, which brings a great number of trouble to policeman when case tracking. In this paper, we proposed a traffic object classification pipeline based on state-of-the-art deep convolutional neural networks (CNNs).Unfortunately, there is no traffic object image database which typically includes motor vehicle (M-vehicle), non-motor vehicle (NM-vehicle) and pedestrian images as we know, and CNNs will be suffered over-fitting without enough training images. Therefore, we first construct a large-scale traffic object image dataset with more than 50000 training images. Then histogram equalization was used to compensate the poor illumination caused by the diversity in image scene. Considering the computing time, we finally choose the 3-convlayer network to our cooperative engagement information system (CEIS), and the value of mAP is 93.3%. With the model fusion, the result is improved to 96%.

Pepper is a common vegetable, with a wide range of applications and market demand. Pepper recognition is an important task when harvesting peppers by a robot. Due to the fact that chili peppers are usually planted very densely, the fruits often grow in clusters. As a result, identifying and locating peppers that are obscured by other peppers, branches, and leaves becomes a challenging task. In this task, the robot needs to overcome visual obstacles in order to accurately detect and locate the target pepper. This paper proposes a pepper recognition method based on the improved You Only Look Once version 3 (YOLOv3) convolutional neural network. By adding Res2Net backbone into the feature extraction network of YOLOv3, the extraction performance is improved, and the accuracy and efficiency of pepper identification are also improved. In this study, large-scale pepper image data sets were used for training and testing, and the generalizability and robustness of the model were improved through data enhancement and transfer learning techniques. For the test samples, the recognition rate of the traditional Backpropagation Neural Network (BPNN) algorithm is 66.25%, while the recognition rate of the improved YOLOv3 convolutional network for the same test samples is 91.25%. The experimental results show that this method has good performance in the pepper recognition task, with high accuracy and fast processing speed.

With the development of the Internet, more and more people like to share their daily food, which makes the Internet rich in food image data. Many researchers collect food images on the Internet, label them, and combine machine learning and other methods to achieve food recognition systems. These food recognition systems can quickly help users record their daily diet and analyze their eating habits. The advanced food identification system can currently accurately identify hundreds of foods. At the same time, it can explore the food nutrition content and give food reference suggestions by accessing the food nutrition database. In this paper, we collected and evaluated large-scale food image data sets and food recognition systems.

The objective of this work is to visually search large-scale video datasets for semantic entities specified by a text query. The paradigm we explore is constructing visual models for such semantic entities on-the-fly, i.e. at run time, by using an image search engine to source visual training data for the text query. The approach combines fast and accurate learning and retrieval, and enables videos to be returned within seconds of specifying a query. We describe three classes of queries, each with its associated visual search method: object instances (using a bag of visual words approach for matching); object categories (using a discriminative classifier for ranking key frames); and faces (using a discriminative classifier for ranking face tracks). We discuss the features suitable for each class of query, for example Fisher vectors or features derived from convolutional neural networks (CNNs), and how these choices impact on the trade-off between three important performance measures for a real-time system of this kind, namely: (1) accuracy, (2) memory footprint, and (3) speed. We also discuss and compare a number of important implementation issues, such as how to remove ‘outliers’ in the downloaded images efficiently, and how to best obtain a single descriptor for a face track. We also sketch the architecture of the real-time on-the-fly system. Quantitative results are given on a number of large-scale image and video benchmarks (e.g. TRECVID INS, MIRFLICKR-1M), and we further demonstrate the performance and real-world applicability of our methods over a dataset sourced from 10,000 h of unedited footage from BBC News, comprising 5M+ key frames.

Co-trained convolutional neural networks for automated detection of prostate cancer in multi-parametric MRI.

Sun orientation estimation from a single image using short-cuts in DCNN

This paper investigates two-types of shape representations for individual Maya codical glyphs: traditional bag-of-words built on knowledge-driven local shape descriptors (HOOSC), and Convolutional Neural Networks (CNN) based representations, learned from data. For CNN representations, first, we evaluate the activations of typical CNNs that are pretrained on large-scale image datasets; second, we train a CNN from scratch with all the available individual segments. One of the main challenges while training CNNs is the limited amount of available data (and handling data imbalance issue). Here, we attempt to solve this imbalance issue by introducing class-weights into the loss computation during training. Another possibility is oversampling the minority class samples during batch selection. We show that deep representations outperform the other, but CNN training requires special care for small-scale unbalanced data, that is usually the case in the cultural heritage domain.

Scene classification of high resolution remote sensing images plays an important role for a wide range of applications. While significant efforts have been made in developing various methods for scene classification, most of them are based on handcrafted or shallow learning-based features. In this paper, we investigate the use of deep convolutional neural network (CNN) for scene classification. To this end, we first adopt two simple and effective strategies to extract CNN features: (1) using pre-trained CNN models as universal feature extractors, and (2) domain-specifically fine-tuning pre-trained CNN models on our scene classification dataset. Then, scene classification is carried out by using simple classifiers such as linear support vector machine (SVM). In our work, three off-the-shelf CNN models including AlexNet [1], VGGNet [2], and GoogleNet [3] are investigated. Comprehensive evaluations on a publicly available 21 classes land use dataset and comparisons with several state-of-the-art approaches demonstrate that deep CNN features are effective for scene classification of high resolution remote sensing images.

The illumination effect is essential for the realistic results in images which are created by inserting virtual objects into real scene. For outdoor scenes, automatic estimation of sun orientation condition from a single outdoor image is fundamental for inserting 3D models to a single image. Traditional methods for outdoor sun orientation estimation often use handcraft illumination features or cues. These cues heavily rely on the experiences of human and pre-processing progresses using other image understanding technologies such as shadow and sky detection, geometry recovery and intrinsic image decomposition, which limit their performances. We propose an end to end way of outdoor sun orientation estimation via a novel deep convolutional neural network (DCNN), which directly outputs the orientation of the sun from an outdoor image. Our proposed SunOriNet contains a contact layer that directly contacts the intermediate feature maps to the high-level ones and learns hierarchical features automatically from a large-scale image dataset with annotated sun orientations. The experiments reveal that our DCNN can well estimate sun orientation from a single outdoor image. The estimation accuracy of our method outperforms model state-of-the-art DCNN based methods.