Active Learning for Deep Visual Tracking.

Deep neural networks, albeit their great success on feature learning in various computer vision tasks, are usually considered as impractical for online visual tracking, because they require very long training time and a large number of training samples. In this paper, we present an efficient and very robust tracking algorithm using a single convolutional neural network (CNN) for learning effective feature representations of the target object in a purely online manner. Our contributions are multifold. First, we introduce a novel truncated structural loss function that maintains as many training samples as possible and reduces the risk of tracking error accumulation. Second, we enhance the ordinary stochastic gradient descent approach in CNN training with a robust sample selection mechanism. The sampling mechanism randomly generates positive and negative samples from different temporal distributions, which are generated by taking the temporal relations and label noise into account. Finally, a lazy yet effective updating scheme is designed for CNN training. Equipped with this novel updating algorithm, the CNN model is robust to some long-existing difficulties in visual tracking, such as occlusion or incorrect detections, without loss of the effective adaption for significant appearance changes. In the experiment, our CNN tracker outperforms all compared state-of-the-art methods on two recently proposed benchmarks, which in total involve over 60 video sequences. The remarkable performance improvement over the existing trackers illustrates the superiority of the feature representations, which are learned purely online via the proposed deep learning framework.

Based on margin sampling (MS) strategy, an active learning approach was introduced for proposed sample selection from large quantities of labeled samples using a Landsat-7 ETM+ image to solve remote sensing image classification problems for large number of training samples. As a breakthrough from conventional random sampling and stratified systematic sampling methods, this approach ensures classification of only using a few hundred training samples to be as effective as that of using several thousand and even tens of thousands of samples by conventional methods, thereby avoiding enormous calculations, substantially reducing operating time and improving training efficiency. The test results of the proposed approach was compared with those of random sampling and stratified systematic sampling, and the effects of training samples on classification under optimized and non-optimized selection conditions was analyzed.

Convolutional neural networks (CNNs) have shown results superior to most traditional image understanding approaches in many fields, incl. crop classification from satellite time series images. However, CNNs require a large number of training samples to properly train the network. The process of collecting and labeling such samples using traditional methods can be both, time-consuming and costly. To address this issue and improve classification accuracy, generating virtual training labels (VTL) from existing ones is a promising solution. To this end, this study proposes a novel method for generating VTL based on sub-dividing the training samples of each crop using self-organizing maps (SOM), and then assigning labels to a set of unlabeled pixels based on the distance to these sub-classes. We apply the new method to crop classification from Sentinel images. A three-dimensional (3D) CNN is utilized for extracting features from the fusion of optical and radar time series. The results of the evaluation show that the proposed method is effective in generating VTL, as demonstrated by the achieved overall accuracy (OA) of 95.3% and kappa coefficient (KC) of 94.5%, compared to 91.3% and 89.9% for a solution without VTL. The results suggest that the proposed method has the potential to enhance the classification accuracy of crops using VTL.

Recently, Convolutional Neural Network (CNN) has achieved some success in synthetic aperture radar (SAR) image classification. This outstanding performance mainly depends on a large number of training samples, but achieved satisfactory classification results with limited training samples remains a challenge. To address this problem, we propose a dual-scale Siamese densely connected network with Markov Random Fields (DS-SDCNet-MRF) for single-polarization SAR image classification. First, the Siamese densely connected network (SDCNet) is proposed to fully extract discriminative features under limited samples. Then, the proposed DS-SDCNet is constructed with two SDCNets of different scales to produce complementary classification results. Among them, the large-scale SDCNet has a better classification result in the homogeneous region, while the small-scale SDCNet tends to provide good detail preservation. Finally, an improved MRF model which combined the category probability information is proposed to further improve the classification performance. Experimental results on simulated and real single-polarization SAR data demonstrate that the proposed method achieves more encouraging classification performance than the current state-of-the-art classification methods with limited samples.

Neural networks (NNs) are efficient tools for rapidly obtaining geoelectric models to solve magnetotelluric (MT) inversion problems. Training an NN with strong predictive power requires numerous training samples to prevent underfitting. To reduce the computational burden of generating a large number of training samples, this work analyzes the influence of the sample features and distribution on the training effect for an NN and proposes an efficient method of sample generation. This innovative method consists of three steps: 1) geoelectrically simplifying the features; 2) removing unnecessary features on the basis of realistic geological characteristics; and 3) mapping the samples to a higher-dimensional space. Numerical examples based on simple stratified models show that the number of samples can be reduced to below one-millionth of the original number while improving the predictive effect of the NN. The performance and effectiveness for processing more complex structures are verified by the inversion results obtained for a public data set, COPROD2. We conclude that this advanced method can generate high-quality training samples at a greatly reduced computational cost. The analysis of the sample features and distribution not only advances the state of research on the use of machine learning in geophysical inversion but also is a forward-looking study on the mechanisms of underfitting, tracing the source of these phenomena back to the training samples used.

At present, the clear license plate recognition algorithm has been mature, but for the blurred license plate which cannot be recognized by the human eye, the recognition rate of the traditional license plate recognition algorithm is too low. In view of this, a license plate character recognition algorithm based on convolutional neural network structure is proposed. In this study, a large number of training samples of fuzzy characters are constructed, and a convolutional neural network is trained for blind segmentation of fuzzy license plate characters. By calling the trained convolutional neural network, we can recognize the characters after blind segmentation. The experimental results show that the training set accuracy is close to 100%, and the accuracy of the test set is more than 93%. It can recognize the license plate characters which cannot be recognized by the human eye.

Learning-enabled spectroscopic analysis, promising for automated real-time analysis of chemicals, is facing several challenges. First, a typical machine learning model requires a large number of training samples that physical systems can not provide. Second, it requires the testing samples to be in range with the training samples, which often is not the case in the real world. Further, a spectroscopy device is limited by its memory size, computing power, and battery capacity. That requires highly efficient learning models for on-site analysis. In this paper, by analyzing multi-gas mixtures and multi-molecule suspensions, we first show that orders of magnitude reduction of data dimension can be achieved as the number of principal components that need to be retained is the same as the independent constituents in the mixture. From this principle, we designed highly compact models in which the essential principal components can be directly extracted from the interrelations between the individual chemical properties and principal components; and only a few training samples are required. Our model can predict the constituent concentrations that have not been seen in the training dataset and provide estimations of measurement noises. This approach can be extended as an effectively standardized method for principle component extraction.

Availability of limited training remote sensing datasets is one of the problems in deep learning, as deep architectures require a large number of training samples for proper training. In this paper, we present a technique for data augmentation based on a spectral indexed generative adversarial network to train deep convolutional neural networks. This technique uses the spectral characteristic of multispectral (MS) images to support data augmentation in order to generate realistic training samples with respect to each land-use and land-cover class. The impact of multispectral remote sensing data generated through the spectral indexed GAN are evaluated through classification experiments. Experimental results obtained on the classification of the Sentinel-2 Eurosatallband datasets show that data augmentation through spectral indexed GAN enhances the main accuracy metrics.

Invariant object recognition (IOR) has been one of the most active research areas in computer vision. However, there is no technique able to achieve the best performance in all possible domains. Out of many techniques, convolutional network (CN) is proven to be a good candidate in this area. Given large numbers of training samples of objects under various variation aspects such as lighting, pose, background, etc., convolutional network can learn to extract invariant features by itself. This conies with the price of lengthy training time. Hence, we propose a circular pairwise classification technique to shorten the training time. We compared the recognition accuracy and training time complexity between our approach and a benchmark generic object recognizer LeNet7 which is a monolithic convolutional network.

Melanoma, most threatening type of skin cancer, is on the rise. In this paper an implementation of a deep-learning system on a computer server, equipped with graphic processing unit (GPU), is proposed for detection of melanoma lesions. Clinical (non-dermoscopic) images are used in the proposed system, which could assist a dermatologist in early diagnosis of this type of skin cancer. In the proposed system, input clinical images, which could contain illumination and noise effects, are preprocessed in order to reduce such artifacts. Afterward, the enhanced images are fed to a pre-trained convolutional neural network (CNN) which is a member of deep learning models. The CNN classifier, which is trained by large number of training samples, distinguishes between melanoma and benign cases. Experimental results show that the proposed method is superior in terms of diagnostic accuracy in comparison with the state-of-the-art methods.

Convolutional Neural Networks have been widely used in remote sensing scene classification. Since this kind of model needs a large number of training samples containing data category information, a Generative Adversarial Network (GAN) is usually used to address the problem of lack of samples. However, GAN mainly generates scene data samples that do not contain category information. To address this problem, a novel supervised adversarial Lie Group feature learning network is proposed. In the case of limited data samples, the model can effectively generate data samples with category information. There are two main differences between our method and the traditional GAN. First, our model takes category information and data samples as the input of the model and optimizes the constraint of category information in the loss function, so that data samples containing category information can be generated. Secondly, the object scale sample generation strategy is introduced, which can generate data samples of different scales and ensure that the generated data samples contain richer feature information. After large-scale experiments on two publicly available and challenging datasets, it is found that our method can achieve better scene classification accuracy even with limited data samples.

In recent years, person re-identification (re-id) catches great attention in both computer vision community and industry. In this paper, we propose a new framework for person re-identification with a triplet-based deep similarity learning using convolutional neural networks (CNNs). The network is trained with triplet input: two of them have the same class labels and the other one is different. It aims to learn the deep feature representation, with which the distance within the same class is decreased, while the distance between the different classes is increased as much as possible. Moreover, we trained the model jointly on six different datasets, which differs from common practice - one model is just trained on one dataset and tested also on the same one. However, the enormous number of possible triplet data among the large number of training samples makes the training impossible. To address this challenge, a double-sampling scheme is proposed to generate triplets of images as effective as possible. The proposed framework is evaluated on several benchmark datasets. The experimental results show that, our method is effective for the task of person re-identification and it is comparable or even outperforms the state-of-the-art methods.

GAR: Graph adversarial representation for adverse drug event detection on Twitter

Layer-by-layer description of the core is performed to understand the regularities of the structure of the geological section, predict the development of reservoirs, clarify stratigraphic boundaries and obtain calculation parameters for assessing hydrocarbon reserves. In this case, the name of the breed is one of the key parameters determined in the layer-by-layer description. This paper presents a comparative analysis of two approaches to determining the breed using machine learning methods: based on graphical identifiers and convolutional neural networks. The original sample contained photographs of core samples from the Tyumenskaya suite fields (8 fields, 15 wells, more than 2 km of core) under daylight. For the analysis, 4 main classes of rocks (siltstones, mudstones, sandstones, coals) were selected. For these rocks, windows of 5 × 5 cm were formed and compressed to 299 × 299 pixels. The total sample exceeded 90,000 windows: 70% — training sample (60,359 windows) and 30% — test (31,140 windows). The training and test samples contain photographs of core samples from different fields. The comparison was made between convolutional neural networks (ResNet, ResNeXt, Inception, etc.) and a classifier (such as XGBoost) based on graphic identifiers of two types: color (average color, dominant colors) and texture (entropy, Euler’s number, contrast, dissimilarity, uniformity, energy, correlation). According to the results of the experiments, the model based on convolutional neural networks turned out to be more sensitive to implicit features and made it possible to reduce the error in the weighted average f1-measure with respect to the ensemble of weak classifiers by 12.5% on the test sample even without optimization of hyperparameters. Thus, we can conclude that the model based on convolutional neural networks is more sensitive to implicit features that are difficult to extract using known graphic identifiers. On the other hand, the approach based on graphic identifiers and an ensemble of weak classifiers can be used without specialized computing power (video cards).

The main core purpose of artificial emotional intelligence is to recognize human emotions. Technologies such as facial, semantic, or brainwave recognition applications have been widely proposed. However, the abovementioned recognition techniques for emotional features require a large number of training samples to obtain high accuracy. Human behaviour pattern can be trained and recognized by the continuous movement of the Spatial Temporal Graph Convolution Network (ST-GCN). However, this technology does not distinguish between the speed of delicate emotions, and the speed of human behaviour and delicate changes of emotions cannot be effectively distinguished. This research paper proposes Spatial Temporal Variation Convolutional Network training for human emotion recognition, using skeleton detection technology to calculate the degree of skeleton point change between consecutive actions and using the nearest neighbour algorithm to classify speed levels and train the ST-GCN recognition model to obtain the emotional state. Application of the speed change recognition ability of the Spatial Temporal Variation Graph Convolution Network (STV-GCN) to artificial emotional intelligence calculation makes it possible to efficiently recognize the delicate actions of happy, sad, fear, and angry in human behaviour. The STV-GCN technology proposed in this paper is compared with ST-GCN and can effectively improve the recognition accuracy by more than 50%.