Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification

One of the challenging problems in understanding high-resolution remote sensing images is aerial scene classification. A well-designed feature representation method and classifier can improve classification accuracy. In this paper, we construct a new two-stream deep architecture for aerial scene classification. First, we use two pretrained convolutional neural networks (CNNs) as feature extractor to learn deep features from the original aerial image and the processed aerial image through saliency detection, respectively. Second, two feature fusion strategies are adopted to fuse the two different types of deep convolutional features extracted by the original RGB stream and the saliency stream. Finally, we use the extreme learning machine (ELM) classifier for final classification with the fused features. The effectiveness of the proposed architecture is tested on four challenging datasets: UC-Merced dataset with 21 scene categories, WHU-RS dataset with 19 scene categories, AID dataset with 30 scene categories, and NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that our architecture gets a significant classification accuracy improvement over all state-of-the-art references.

Many predictive tasks in recommender systems model from categorical variables. Different from continuous features extracted from images and videos, categorical data is discrete and of multi-field while their dependencies are little known, which brings the problem of heavy computation on a large-scale sparse feature space. Deep learning methods have strong feature extraction capabilities and now have been more and more widely applied to recommender systems, but they do not perform well on discrete data. To tackle these two problems, in this paper we propose Deep Feature Fusion Model(DFFM) over sparse multi-field categorical data. DFFM utilizes categorical features as inputs and applies the Stacked Denoising AutoEncoder to obtain a dense representation. We construct a full feature connection layer and adopt a multi-layer convolution neural network to further extract deeper features and convert rating prediction to a classification problem. The extensive experiments on real world datasets show that our proposed method outperforms other state-of-the-art approaches.

Extracting disease characteristics from large-scale Electronic Medical Records and achieving disease-assisted diagnoses have significant research value. Due to the complex multi-feature items and unbalanced data distribution of Electronic Medical Records, feature representation and disease diagnosis are difficult. Our study proposes a deep feature fusion (DFF) model based on the feature partition and deep feature extraction. First, the feature partition is performed, and different feature representation algorithms are adopted for different types of data. The discrete feature items are directly mapped into real-valued vectors, and the continuous feature items are represented by GCNN-based VAE. Then, the two parts are fused. Finally, the assisted diagnosis results are output through a supervised learning classification method based on the XGBoost framework. The dataset of our study is from the 18,590 real and effective clinical Electronic Medical Record of Huangshi Central Hospital. The experimental results show that the method can perform clinical Assisted diagnosis accurately and efficiently, which are superior to some other state-of-the-art approaches, can better meet the needs of practical clinical diagnosis applications.

The performance of visual object tracking depends largely on the target appearance model. Benefited from the success of CNN in feature extraction, recent studies have paid much attention to CNN representation learning and feature fusion model. However, the existing feature fusion models ignore the relation between the features of different layers. In this paper, we propose a deep feature fusion model based on the siamese network by considering the connection between feature maps of CNN. To tackle the limitation of different feature map sizes in CNN, we propose to fuse different resolution feature maps by introducing de-convolutional layers in the offline training stage. Specifically, a top-down modulation is adopted for feature fusion. In the tracking stage, a simple matching operation between the fused feature of the examplar and search region is conducted with the learned model, which can maintain the real-time tracking speed. Experimental results show that, the proposed method obtains favorable tracking accuracy against the state-of-the-art trackers with a real-time tracking speed.

Wind speed forecasting is important for high‐efficiency utilisation of wind energy and management of grid‐connected power systems. Due to the noise, instability and irregularity of atmosphere system, the current models based on raw historical data have encountered many problems. In this study, a deep novel feature extraction approach is developed based on stacked denoising autoencoders and batch normalisation. Then the deep features extracted from raw historical data are fed to long short‐term memory (LSTM) neural networks for prediction. Meanwhile, density‐based spatial clustering of applications with noise is employed to process the numerical weather prediction data. By picking out the abnormal samples, the representative training samples are selected to improve the efficiency of the model. For illustration and verification purposes, the proposed model is used to predict the wind speed of Wind Atlas for South Africa (WASA). Empirical results show that deep feature extraction can improve the forecasting accuracy of LSTM 49% than feature selection, indicating that proper feature extraction is crucial to wind speed forecasting. And the proposed model outperforms other benchmark methods at least 17%. Hence, the proposed model is promising for wind speed forecasting.

Vehicle classification is a challenging task in the area of image processing. It involves the classification of various vehicles based on their color, model, and make. A distinctive variety of vehicles belonging to various model categories have been developed in the automobile industry, which has made it necessary to establish a compact system that can classify vehicles within a complex model group. A well-established vehicle classification system has applications in security, vehicle monitoring in traffic cameras, route analysis in autonomous vehicles, and traffic control systems. In this paper, a hybrid model based on the integration of a pre-trained Convolutional Neural Network (CNN) and an evolutionary feature selection model is proposed for vehicle classification. The proposed model performs classification of eight different vehicle categories including sports cars, luxury cars and hybrid power-house SUVs. The used in this work is derived from Stanford car dataset that contains almost 196 cars and vehicle classes. After performing appropriate data preparation and preprocessing steps, feature learning and extraction is carried out using pre-trained VGG16 first that learns and extracts deep features from the set of input images. These features are then taken out of the last fully connected layer of VGG16, and feature optimization phase is carried out using evolution-based nature-inspired optimization model Genetic Algorithm (GA). The classification is performed using numerous SVM kernels where Cubic SVM achieves an accuracy of 99.7% and outperforms other kernels as well as excels in terns of performance as compared to the existing works.

Remote sensing image retrieval (RSIR) is the process of ranking database images depending on the degree of similarity compared to the query image. As the complexity of RSIR increases due to the diversity in shooting range, angle, and location of remote sensors, there is an increasing demand for methods to address these issues and improve retrieval performance. In this work, we introduce a novel method for retrieving aerial images by merging group convolution with attention mechanism and metric learning, resulting in robustness to rotational variations. For refinement and emphasis on important features, we applied channel attention in each group convolution stage. By utilizing the characteristics of group convolution and channel-wise attention, it is possible to acknowledge the equality among rotated but identically located images. The training procedure has two main steps: (i) training the network with Aerial Image Dataset (AID) for classification, (ii) fine-tuning the network with triplet-loss for retrieval with Google Earth South Korea and NWPU-RESISC45 datasets. Results show that the proposed method performance exceeds other state-of-the-art retrieval methods in both rotated and original environments. Furthermore, we utilize class activation maps (CAM) to visualize the distinct difference of main features between our method and baseline, resulting in better adaptability in rotated environments.

Temporal lobe injury (TLI), a form of nervous system damage in the brain, is a major neurological complication after radiation therapy (RT). TLI must be highly valued because of the irreversible brain injury. This article aims to develop a predictive pipeline, called deep longitudinal feature representations (DLFR), to detect TLI at the presymptomatic stage accurately via the learning of effective deep longitudinal feature representations. DLFR characterizes high-level information and developmental changes within and across subjects The DLFR consists of four components: (i) extraction of deep features from a pretrained ResNet50 model; (ii) compression of learned highly representative features by the global max pooling; (iii) fusion of deep longitudinal features for the fully use of all follow-up data; (iv) random forest-based prediction of the diagnostic status. In total, 244 nasopharyngeal carcinoma patients before and after RT with a follow-up period of 0 ~ 9 years were included for analysis. All patients were divided into four different latency groups, and the current latency was used for training to predict the diagnostic status of the next latency. The AUCs of the predicted three different latency groups using DLFR were 0.64 ± 0.11, 0.76 ± 0.10, and 0.88 ± 0.05, while those of radiomics features were 0.56 ± 0.06, 0.63 ± 0.03, and 0.53 ± 0.04, and those of histogram of oriented gradients features were 0.60 ± 0.09, 0.52 ± 0.03, and 0.58 ± 0.06. Most importantly, the AUCs of the predicted three different latency groups for white matter regions were 0.66 ± 0.10, 0.80 ± 0.09, and 0.78 ± 0.09. Our proposed method can dynamically detect TLI at the presymptomatic stage, which can enable the administration of preventive neurological intervention.

Recently, convolutional neural networks (CNNs) have been successfully utilized in polarimetric synthetic aperture radar (PolSAR) image classification and obtained promising results. However, most CNN-based classification methods require a large number of labeled samples and it is difficult to obtain sufficient labeled samples. For this reason, an ensemble dual-branch CNN (EDb-CNN) is proposed for PolSAR image classification with small samples. First, to solve the problem of the small sample in PolSAR image classification, a new data enhancement method based on the superpixel algorithm is proposed to expand the number of labeled samples. Second, to obtain different scales of features from PolSAR images, a Db-CNN model is proposed. This model contains two parallel CNN structures. One CNN branch is used to extract the polarization features from the complex coherency matrix. The other branch is utilized to extract the spatial features based on weighted spatial neighborhood. On the top of these two branches, a feature fusion model is adopted to combine these two deep features, and a weighted loss function is employed to improve the learning procedure. Then, the ensemble learning algorithm is used for each CNN branch and Db-CNN network to obtain the better classification results. Finally, a postprocess algorithm based on the superpixel algorithm is proposed to improve the consistency of classification results. Experiments on two PolSAR datasets show that the proposed method achieves a much better performance than other classification methods, especially when only a few labeled samples are available.

Hyperspectral image classification based on combining texture and multiscale guided filter feature of principal component

A novel SRC fusion method using hierarchical multi-scale LBP and greedy search strategy

More and more attention has been paid to the invariant texture analysis, because the training and testing samples generally have not identical or similar orientations, or are not acquired from the same viewpoint in many practical applications, which often has negative influences on texture analysis. Local binary pattern (LBP) has been widely applied to texture classification due to its simplicity, efficiency, and rotation invariant property. In this paper, an integrated local binary pattern (ILBP) scheme including original rotation invariant LBP, improved contrast rotation invariant LBP, and direction rotation invariant LBP is proposed which can effectively overcome the deficiency of original LBP that is ignoring contrast and direction information. In addition, for surmounting another major drawback of LBP such as locality which can result in the lack of shape and space expression of the holistic texture image, Zernike moment features are fused into the improved LBP texture features in the proposed method because they comprise orthogonal and rotation invariant property and can be easily and rapidly calculated to an arbitrary high order. Experimental results show that the proposed method can be remarkably superior to the other state-of-the-art methods when rotation invariant texture features are extracted and classified.

Rationale and ObjectivesAccurate prediction of microvascular invasion (MVI) in hepatocellular carcinoma (HCC) is crucial for guiding treatment. This study evaluates and compares the performance of clinicoradiologic, traditional radiomics, deep learning radiomics, feature fusion, and decision fusion models based on multi-region MR habitat imaging using six machine learning classifiers. Materials and MethodsWe retrospectively included 300 HCC patients. The intratumoral and peritumoral regions were segmented into distinct habitats, from which radiomics and deep learning features were extracted using arterial phase MR images. To reduce feature dimensionality, we applied intra-class correlation coefficient (ICC) analysis, Pearson correlation coefficient (PCC) filtering, and recursive feature elimination (RFE). Based on the selected optimal features, prediction models were constructed using decision tree (DT), K-nearest neighbors (KNN), logistic regression (LR), random forest (RF), support vector machine (SVM), and XGBoost (XGB) classifiers. Additionally, fusion models were developed utilizing both feature fusion and decision fusion strategies. The performance of these models was validated using the area under the receiver operating characteristic curve (ROC AUC), calibration curves, and decision curve analysis. ResultsThe decision fusion model (VOI-Peri10-1) using LR and integrating clinicoradiologic, radiomics, and deep learning features achieved the highest AUC of 0.808 (95% confidence interval [CI]: 0.807–0.912) in the test cohort, with good calibration (Hosmer-Lemeshow test, P > 0.050) and clinical net benefit. ConclusionThe LR-based decision fusion model integrating clinicoradiologic, radiomics, and deep learning features shows promise for preoperative prediction of MVI in HCC, aiding in patient outcome predictions and personalized treatment planning.

Convolutional neural network (CNN) based methods have dominated the field of aerial scene classification for the past few years. While achieving remarkable success, CNN-based methods suffer from excessive parameters and notoriously rely on large amounts of training data. In this work, we introduce few-shot learning to the aerial scene classification problem. Few-shot learning aims to learn a model on base-set that can quickly adapt to unseen categories in novel-set, using only a few labeled samples. To this end, we proposed a meta-learning method for few-shot classification of aerial scene images. First, we train a feature extractor on all base categories to learn a representation of inputs. Then in the meta-training stage, the classifier is optimized in the metric space by cosine distance with a learnable scale parameter. At last, in the meta-testing stage, the query sample in the unseen category is predicted by the adapted classifier given a few support samples. We conduct extensive experiments on two challenging datasets: NWPU-RESISC45 and RSD46-WHU. The experimental results show that our method yields state-of-the-art performance. Furthermore, several ablation experiments are conducted to investigate the effects of dataset scale, the impact of different metrics and the number of support shots; the experiment results confirm that our model is specifically effective in few-shot settings.

Despite deep neural networks (DNNs) have been widely applied in remote sensing (RS) scene classification and achieved satisfying performance, the vulnerability of DNNs towards adversarial examples significantly degrades their performance. Moreover, the relatively limited labeled samples of RS scene classification make DNNs more likely to overfit, leading to weak generalizability and noise sensitivity. This may result in DNNs being more vulnerable to adversarial examples. Consequently, the defense of adversarial examples is of crucial importance to improve both the generalizability and robustness of DNNs in the RS scene classification task. However, few studies have been conducted on defense for RS scene classification, especially ignoring the intrinsic characteristics of RS images. In this paper, an effective defense framework for RS scene classification, named reconstruction-assisted and distance-optimized adversarial training (RDAT), is proposed to defend adversarial examples. In order to solve the problems caused by high interclass similarity, a distance-optimized (DO) strategy is designed for adversarial training to strengthen the learning of underfitting content, increase the interclass distance, and improve the robustness of the networks. Furthermore, in order to generate high quality samples for adversarial training, a reconstruction-assisted (RA) block is proposed to eliminate adversarial perturbations in adversarial examples. Specifically, in this block, by swin transformer (SwinT) block and multi-scale convolution (MSC) block, SwinT-MSC-UNet (SMUNet) is constructed to fully extract global and multi-scale local features to adapt to the characteristics of RS images with large variance of ground object scales. Extensive experiments on the benchmark datasets, i.e., UC Merced (UCM) and Aerial Image Dataset (AID), have demonstrate that the proposed RDAT can effectively resist multiple adversarial attacks and yield superior results than other defense methods for RS scene classification.