Feature Fusion Method based on Discriminant Correlation Analysis for Land Use Classification with Few-Shot

In the neck part of a two-stage object detection network, feature fusion is generally carried out in either a top-down or bottom-up manner. However, two types of imbalance may exist: feature imbalance in the neck of the model and gradient imbalance in the region of interest extraction layer due to the scale changes of objects. The deeper the network is, the more abstract the learned features are, that is to say, more semantic information can be extracted. However, the extracted image background, spatial location, and other resolution information are less. In contrast, the shallow part can learn little semantic information, but a lot of spatial location information. We propose the Both Ends to Centre to Multiple Layers (BEtM) feature fusion method to solve the feature imbalance problem in the neck and a Multi-level Region of Interest Feature Extraction (MRoIE) layer to solve the gradient imbalance problem. In combination with the Region-based Convolutional Neural Network (R-CNN) framework, our Balanced Feature Fusion (BFF) method offers significantly improved network performance compared with the Faster R-CNN architecture. On the MS COCO 2017 dataset, it achieves an average precision (AP) that is 1.9 points and 3.2 points higher than those of the Feature Pyramid Network (FPN) Faster R-CNN framework and the Generic Region of Interest Extractor (GRoIE) framework, respectively.

In this paper we explore the benefits of synthetically generated data for the task of company logo detection with deep-learned features in the absence of a large training set. We use pre-trained deep convolutional neural networks for feature extraction and use a set of support vector machines for classifying those features. In order to generate sufficient training examples we synthesize artificial training images. Using a bootstrapping process, we iteratively add new synthesized examples from an unlabeled dataset to the training set. Using this setup we are able to obtain a performance which is close to the performance of the full training set.

Part-based visual tracking via structural support correlation filter

Malignant melanoma is the deadliest form of skin cancer and, in recent years, is rapidly growing in terms of the incidence worldwide rate. The most effective approach to targeted treatment is early diagnosis. Deep learning algorithms, specifically convolutional neural networks, represent a methodology for the image analysis and representation. They optimize the features design task, essential for an automatic approach on different types of images, including medical. In this paper, we adopted pretrained deep convolutional neural networks architectures for the image representation with purpose to predict skin lesion melanoma. Firstly, we applied a transfer learning approach to extract image features. Secondly, we adopted the transferred learning features inside an ensemble classification context. Specifically, the framework trains individual classifiers on balanced subspaces and combines the provided predictions through statistical measures. Experimental phase on datasets of skin lesion images is performed and results obtained show the effectiveness of the proposed approach with respect to state-of-the-art competitors.

In this paper, a novel image classification method is proposed with a Bag-of-Words framework. For its simplified characteristics, the Bag-of-Words method is widely used in fields such as image classification and object recognition. Nevertheless, it still has some challenges, e.g., one single feature cannot always describe the information of a whole image, and the visual words allocation mechanism cannot represent the image better. Accordingly, two methods are proposed in this paper. For the first problem, a feature-fusion method is used by combining SIFT and HOG features. For the second, a soft threshold visual word allocation mechanism is put forward, in which k-means method is adopted to cluster the syncretic feature and form a “Visual Dictionary“, where each of the visual words with different weights is assigned to. Meanwhile, to add the image's spatial information, SPM theory is combined within the Bag-of-Words framework by using an SVM classifier to classify the image. Experiments on the Caltech-101 and Scene-15 datasets show that the proposed method outperforms the traditional methods.

Leaves are the primary interface where energy, water and carbon exchanges occur between the forest ecosystems and the atmosphere. Leaf area index (LAI) is a measure of the amount of leaf area in a stand, and the tree crown size characterizes how leaves are clumped in the canopy. Both LAI and tree crown size are of essential ecological and management value. There is a lot of interest in extracting both canopy structural parameters from remote sensing. The LAI is generally estimated with spectral information from remotely sensed images at relatively coarse spatial resolution. There has been much less success in estimating tree crown size with remote sensing. The recent availability of abundant high spatial resolution imagery from space offers new potential for extracting LAI and tree crown size, particularly in the spatial domain. This study found that the spatial information in Ikonos imagery is highly valuable in estimating both tree crown size and LAI. When the conifer‐ and hardwood‐dominated stands are pooled, tree crown sizes of conifer stands relate best to the ratio of image variance at 2×2 m spatial resolution to that at 3×3 m spatial resolution, while LAI relates best to image variance at 4×4 m spatial resolution. When the conifer‐ and hardwood‐dominated stands are separated, image spatial information estimates tree crown size much better for conifer‐dominated stands than for the hardwood‐dominated stands, while the relationship between image spatial information and LAI is strengthened after the two types of stands are combined. Tree crown size is more sensitive to image spatial resolution than LAI. Image variance is more useful in estimating LAI than normalized difference vegetation index (NDVI) and simple ratio vegetation index (SRVI). Combining both spatial and spectral information provides some improvement in estimating LAI compared with using spatial information alone. Therefore, future efforts to estimate canopy structure with high resolution imagery should also use image spatial information.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

Geographic information systems (GISs) are widely used for representation, management, and analysis of spatial data in many disciplines. In particular, geoscientists increasingly use these tools for data integration and management purposes in many environmental applications, ranging from water resources management to the study of global warming. Beyond these capabilities, geoscientists need to model and simulate three-dimensional (3D) dynamic fields and readily integrate those results with other relevant spatial information in order to have a better understanding of the environmental problems. However, GISs are very limited for the modeling and simulation of spatial fields, which are mostly 3D and dynamic. These limitations are mainly related to the existing GIS spatial data structures that are static and limited to 2D space. In order to overcome these limitations, we develop and implement a new kinetic 3D spatial data structure based on Delaunay tetrahedralization and a 3D Voronoi diagram to support a 3D dynamic field simulation within GISs. In this article, we describe in detail the different steps from discretization of a 3D continuous field to its numerical integration, based on an event-driven method. For validation of the proposed spatial data structure itself and its potential for the simulation of a dynamic field, two case studies are presented in the article. According to our observations, during the simulation process, the data structure is maintained and the 3D spatial information is managed adequately. Furthermore, the results obtained from both experiments are very satisfactory and are comparable with the results obtained from other existing methods for the simulation of the same dynamic field. To conclude, we discuss the current challenges related to the development of the 3D kinetic data structure itself and its adaptation to 3D dynamic field simulation and suggest some solutions for its improvement.

This paper proposes an efficient and high-fidelity image fusion method based on adaptive smoothing filtering for panchromatic (PAN) and multispectral (MS) image fusion. The scale ratio reflects the ratio of spatial resolution between the panchromatic image and the multispectral image. When facing a multiscale fusion task, traditional methods are unable to simultaneously handle the problems of spectral resolution loss resulting from high scale ratios and the issue of reduced spatial resolution due to low scale ratios. To adapt to the fusion of panchromatic and multispectral satellite images of different scales, this paper improves the problem of the insufficient filtering of high-frequency information of remote sensing images of different scales by the classic filter-based intensity modulation (SFIM) model. It uses Gaussian convolution kernels instead of traditional mean convolution kernels and builds a Gaussian pyramid to adaptively construct convolution kernels of different scales to filter out high-frequency information of high-resolution images. It can adaptively process panchromatic multispectral images of different scales, iteratively filter the spatial information in panchromatic images, and ensure that the scale transformation is consistent with the definition of multispectral images. Using 15 common fusion methods, this paper compares the experimental results of ZY-3 with scale ratio 2.7 and SV-1 with scale ratio 4 data. The results show that the method proposed in this paper retains good spatial information for image fusion at different scales and has good spectral preservation.

A fundamental issue in differential motion analysis is the compromise between the flexibility of the matching criterion for image regions and the ability of recovering the motion. Localized matching criteria, e.g., pixel-based SSD, may enable the recovery of all motion parameters, but it does not tolerate much appearance changes. On the other hand, global criteria, e.g., matching histograms, can accommodate dramatic appearance changes, but may be blind to some motion parameters, e.g., scaling and rotation. This paper presents a novel differential approach that integrates the advantages of both in a principled way based on a spatial-appearance model (SAM) that combines local appearances variations and global spatial structures. This model can capture a large variety of appearance variations that are attributed to the local non-rigidity. At the same time, this model enables efficient recovery of all motion parameters. A maximum likelihood matching criterion is defined and rigorous analytical results are obtained that lead to a closed form solution to motion tracking. Very encouraging results demonstrate the effectiveness and efficiency of the proposed method for tracking non-rigid objects that exhibit dramatic appearance deformations, large object scale changes and partial occlusions.

Discovering class-specific informative regions for a given concept with a few images is an interesting but very challenging task, due to occlusion, scale changes of objects, as well as different views under varying lighting conditions. This paper proposes a new perspective to discover the informative regions by using several images. To achieve this, we introduce a new representation of image: Ordered-BoW Image (BoWI), whose elements summarizes information of the patch centered at the element in original image. Because of its “structured pixels”, BoWI is robust and informative enough for an object class representation. Histogram-based Multi-Ranking Amalgamation Strategy (MRAS) is adopted to explore the most informative patches for an object in BoWI. Experiments on Landmark-National Icon data set that our approach is robust to occlusion, scale and illumination, and achieves promising performance in discovering class-specific informative regions.KeywordsBoW ImageMulti-Ranking Amalgamation StrategyInformative Patch

Water Accounting Plus for Water Resources Reporting and River Basin Planning

Dimensionality reduction is of great importance which aims at reducing the spectral dimensionality while keeping the desirable intrinsic structure information of hyperspectral images. Tensor analysis which can retain both spatial and spectral information of hyperspectral images has caused more and more concern in the field of hyperspectral images processing. In general, a desirable low dimensionality feature representation should be discriminative and compact. To achieve this, a tensor discriminant analysis model via compact feature representation (TDA-CFR) was proposed in this paper. In TDA-CFR, the traditional linear discriminant analysis was extended to tensor space to make the resulting feature representation more informative and discriminative. Furthermore, TDA-CFR redefines the feature representation of each spectral band by employing the tensor low rank decomposition framework which leads to a more compact representation.

The application of a visual data mining framework to determine soil, climate and land use relationships

We propose a novel scale and rotation invariant method to track a human subject's body part regions in cluttered videos. The proposed method optimizes the assembly of body part region proposals with the spatial and temporal constraints of a human body plan. This approach is invariant to the object scale and rotation changes. To enable scale and rotation invariance, the human body part graph of the proposed method has to be loopy, efficiently optimizing the body part region assembly is a great challenge. We propose a dynamic programming method to solve the problem. We devise a method that finds N-best whole body configurations from loopy structures in each video frame using dynamic programming. The N-best configurations are then used to construct trellises with which we track human body part regions by finding shortest paths on the trellises. Our experiments on a variety of videos show that the proposed method is efficient, accurate and robust against object appearance variations, scale and rotation changes and background clutter.