Vehicle Re-Identification by Separating Representative Spatial Features

Vehicle re-identification aims to retrieve images of the same vehicle under different viewpoints, camera perspectives, and environmental conditions from a large-scale image database. However, significant challenges arise due to intra-class variations and inter-class similarities. Moreover, the difficulty in effectively utilizing multi-view information substantially limits further improvements in recognition accuracy. This paper proposes a novel multi-branch network named MGF-Net that enhances vehicle feature representation by jointly optimizing global and fine-grained feature extraction modules. In the global feature extraction module, a dual-dimensional channel–spatial adaptive feature re-weighting mechanism is implemented to amplify responses in vehicle-centered regions, suppress background interference, and improve the robustness of global features. In the fine-grained feature extraction module, a multi-stage feature interaction mechanism is constructed to capture detailed features and spatial semantic associations of key vehicle parts, such as the front grille and hood emblem, thereby avoiding the feature fragmentation issue caused by manual part partitioning. To further improve recognition performance in cross-view scenarios, a multi-view fusion strategy is introduced to dynamically aggregate feature representations from different viewpoints, thereby enhancing the model’s generalization ability. Additionally, both branches are optimized using classification loss and metric loss functions to refine the model from multiple perspectives. Extensive experiments on the challenging VeRi-776 and VehicleID datasets demonstrate that our approach achieves state-of-the-art performance and significantly improves vehicle re-identification accuracy.

Creating accurate land use and land cover maps using remote sensing images is one of the most important applications of remotely sensed data. Abundant spectral information in hyperspectral images (HSI) makes it possible to distinguish materials that would not be distinguishable by multi-spectral sensors. Spectral and spatial information from HSI is of primary importance for image classification. In this study, a hybrid stacked autoencoder (SAE) architecture and support vector machine (SVM) classifier was constructed to classify the HSI. The SAE architecture is constituted by stacking a multiple autoencoder (AE) deep learning network that consists in the encoder and decoder process. Spatial features in a neighbor region extracted from the principal component analysis (PCA) and the texture feature extracted from the gray-level cooccurrence matrix (GLCM) were fed into the classifier. It was found that the best result was from the combination of GLCM texture feature, PCA spatial feature, and spectral feature. Meanwhile, the representative features derived from SAE deep learning network were better than the original features. It reminded us that extracting the representative features from hyperspectral images is a key step of improving classification accuracy.

Hyperspectral image classification has been widely applied in many fields, but it also faces challenges because of small number of labeled samples. In this paper, we propose the Multiscale Multistage Spectral-Spatial Feature Fusion Framework (M^2 S^2 F^2 ) for hyperspectral image classification using small training samples. The Framework is the combination of two deep convolutional neural networks, which can extract more representative and discriminative features by combining the following operations. Firstly, two different scale 3-D cubes are the inputs for the spectral and spatial feature extraction respectively. Secondly, by fusing strong complementary information between different layers, we form multistage spectral and spatial features by fusion primary, intermediate and advanced features of the spectral and spatial features respectively. Spectral and spatial features are extracted by spectral and spatial skipped residual blocks, which can effectively alleviate the problems of gradient degradation. Thirdly, the fusion of complementary multistage spectral and spatial features can improve the classification accuracy. Experimental results on the IN, UP and KSC datasets show the effectiveness of the proposed method using small training samples.

Hyperspectral images have the characteristics of high spectral resolution and low spatial resolution, which will make the extracted features insufficient and lack detailed information about ground objects, thus affecting the accuracy of classification. The numerous spectral bands of hyperspectral images contain rich spectral features but also bring issues of noise and redundancy. To improve the spatial resolution and fully extract spatial and spectral features, this article proposes an improved feature enhancement and extraction model (IFEE) using spatial feature enhancement and attention-guided bidirectional sequential spectral feature extraction for hyperspectral image classification. The adaptive guided filtering is introduced to highlight details and edge features in hyperspectral images. Then, an image enhancement module composed of two-dimensional convolutional neural networks is used to improve the resolution of the image after adaptive guidance filtering and provide a high-resolution image with key features emphasized for the subsequent feature extraction module. The proposed spectral attention mechanism helps to extract more representative spectral features, emphasizing useful information while suppressing the interference of noise. Experimental results show that our method outperforms other comparative methods even with very few training samples.

PSTFormer: A novel parallel spatial-temporal transformer for remaining useful life prediction of aeroengine

The fusion of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data is widely used for land cover classification. However, due to different imaging mechanisms, HSI and LiDAR data always present significant image differences, and the dimensions and feature distributions of HSI and LiDAR are highly dissimilar. This makes it challenging to represent and correlate semantic information from multimodal data. Current methods for classifying pixel-by-pixel features, which rely on cascaded or attention-based fusion, cannot effectively use multimodal features. To achieve accurate classification results, extracting and fusing similar high-order semantic information and complementary discriminative information contained in multimodal data is vital. In this paper, we propose a cross-modal semantic enhancement network (CMSE) for multimodal semantic information mining and fusion. Our proposed CMSE framework extracts features from the image on multiple scales, capturing more representative local sparse features with different sizes of convolution kernels. To represent high-level semantic features related to land cover, we establish a Gaussian-weighted matrix and semantically transform the spatial and spectral features of distinct branches. Finally, we build a multi-level residual fusion module to incrementally fuse spectral features from HSI and elevation features from LiDAR. Additionally, we introduce a cross-modal semantically constrained loss to guide multimodal semantic feature alignment. We evaluate our approach on three multimodal remote sensing datasets, namely the Houston2013, Trento, and MUUFL datasets. The experimental results demonstrate that our proposed CMSE model achieves superior performance in terms of accuracy and robustness compared to other related deep networks.

Graph neural networks (GNNs) have demonstrated strong performance in handling graph-structured data in recent years, particularly in capturing complex inter-node relationships among data samples, showcasing advantages over traditional neural networks. However, challenges persist, including difficulties in cross-modal information fusion, inadequate modeling of modal relationships, and high computational costs. To address these limitations, this paper proposes GGMEN, a novel model that integrates the local neighborhood aggregation capability of graph convolutional networks with the global structural expressiveness of graph isomorphic networks (GINs). Leveraging shallow feature extraction via time-frequency joint analysis, the paper extracts 14 representative physiological statistical features. Simultaneously, the Transformer model captures spatial features from individual facial expression video frames, enabling spatio-temporal modeling of facial expressions. The graph convolutional network (GCN) layer models temporal dependencies in physiological signals and spatial relationships of facial key points, while the GIN layer enhances modeling of complex higher-order relationships. Multi-modal emotion perception is achieved through attention-based modality fusion. Experiments on multiple datasets validate the model’s effectiveness across multiple emotion perception benchmarks, achieving an emotion recognition accuracy of 81.25% on dataset DEAP. Comparative analyses with existing models confirm the accuracy improvement of the proposed framework.

Data augmentation technology can effectively solve the problem of few-shot image classification, but many approaches based on data augmentation generated sample have poor discriminability, which negatively affects the classifier and classification accuracy needs to be improved. In this regard, based on the TFFCN, a mixture loss function-based classification network for few-shot learning (MLFCN) is proposed. First of all, for the prototype tensor features with a simple mean representation, ignoring the different contributions of the sample features of support set for per class to the class prototype tensor features, so that the generated prototype tensor features can not well represent sample features of per class, affecting the discriminability of the generated samples. Introduce the improved adaptively spatial feature fusion module, it uses the convolution, considers the support set of different sample features on the prototype tensor, adaptively assign weights, then weighted and summed to obtain a more representative prototype tensor feature; secondly, aiming at the blurred classification boundary of similar categories in the generated sample features, a mixture loss function is proposed. The loss function is to add the inter-class mean squared error term to the mean square error loss, and stretches the inter-class distance of the generated tensor feature, so that the classification boundary is clearer, so as to train a better classifier. Experimental results show that the improved adaptively spatial feature fusion and the mixing loss function can augment the discriminability of the generated tensor features and further improve the classification accuracy.

Learning representative and discriminative feature that make full use of spectral-spatial information is of cardinal significance for hyperspectral imagery (HSI) interpretation. In this paper, we present a novel unsupervised feature extraction method, deep multiscale spectral-spatial feature fusion (DMS $^3$ F $^2$ ), for hyperspectral images classification (HSIC). First, pretrained filter banks in VGG16 are transferred to extract deep multiscale spatial structural information about HSI. Second, we propose a new unsupervised cooperative sparse autoencoder method to fuse together the deep spatial feature and the raw spectral information. It achieves both dimensionality reduction and feature fusion, and cooperates on different features to fulfil self-supervised projective learning, which is beneficial in detecting important structure and retaining significant amount of information about the input. In the end, fused spectral-spatial features from multiple scales in VGG16 are further weighted fused and upsampled to obtain the final discriminative feature. Qualitative classification experiments by support vector machines are reported to show the effectiveness of the proposed algorithm. Compared with existing feature extraction techniques, the proposed method is much more effective for HSIC, especially for datasets with less training samples and scenes with complex semantic information and great heterogeneity in terms of geometrical shape.

Hyperspectral image (HSI) classification remains a challenging problem due to unique characteristics of HSI data (such as numerous bands and strong correlations in the spectral and spatial domains) and small sample size. To address such concerns, we propose a novel spectral–spatial feature extraction method for HSI classification by employing graph embedding and deep learning (DL) models. Since the conventional graph cannot capture the complex manifold relationship of HSI data, and there exist the observations of within-class variation as well as the similarity between different classes in the spectral domain, we construct the supervised within-class/between-class hypergraph (SWBH) to extract the spectral features of HSI. Since it is difficult for DL models to learn representative features for HSI data when the labeled training samples are limited, we propose the random zero settings to newly generate a large amount of labeled HSI samples for the training of convolutional neural network (CNN). The designed sample expanded CNN (SECNN) is used to extract the HSI spatial features. Thus, the spectral–spatial features of HSI can be learned by integrating the features extracted from SWBH and SECNN, respectively. Experiments on three real HSI datasets demonstrate higher classification accuracy of the proposed SWBH–SECNN method.

We propose a new fusion-based classification technique for optical multi-source remote sensing images called OptFus. OptFus is developed to merge and process optical imagery having different spatial and spectral resolutions. The spatial features are extracted using morphological filters from the RGB data containing high spatial resolution. A feature fusion technique is developed to combine all the sensor data in a subspace using a common set of representative features. Finally, the fused features are classified using a Support Vector Machine to ensure a robust supervised spectral classification. The proposed method is designed to allocate varying weights to the data from various imaging sensors in the fusion process. OptFus is applied to two multi-source optical datasets captured from geological drill-core samples. The classification accuracy demonstrates considerable improvements compared to the state-of-the-art. A Matlab implementation of OptFus is available online: https://github.com/BehnoodRasti/OptFus.

Hyperspectral change detection (CD) can be effectively performed using deep-learning networks. Although these approaches require qualified training samples, it is difficult to obtain ground-truth data in the real world. Preserving spatial information during training is difficult due to structural limitations. To solve such problems, our study proposed a novel CD method for hyperspectral images (HSIs), including sample generation and a deep-learning network, called the recurrent three-dimensional (3D) fully convolutional network (Re3FCN), which merged the advantages of a 3D fully convolutional network (FCN) and a convolutional long short-term memory (ConvLSTM). Principal component analysis (PCA) and the spectral correlation angle (SCA) were used to generate training samples with high probabilities of being changed or unchanged. The strategy assisted in training fewer samples of representative feature expression. The Re3FCN was mainly comprised of spectral–spatial and temporal modules. Particularly, a spectral–spatial module with a 3D convolutional layer extracts the spectral–spatial features from the HSIs simultaneously, whilst a temporal module with ConvLSTM records and analyzes the multi-temporal HSI change information. The study first proposed a simple and effective method to generate samples for network training. This method can be applied effectively to cases with no training samples. Re3FCN can perform end-to-end detection for binary and multiple changes. Moreover, Re3FCN can receive multi-temporal HSIs directly as input without learning the characteristics of multiple changes. Finally, the network could extract joint spectral–spatial–temporal features and it preserved the spatial structure during the learning process through the fully convolutional structure. This study was the first to use a 3D FCN and a ConvLSTM for the remote-sensing CD. To demonstrate the effectiveness of the proposed CD method, we performed binary and multi-class CD experiments. Results revealed that the Re3FCN outperformed the other conventional methods, such as change vector analysis, iteratively reweighted multivariate alteration detection, PCA-SCA, FCN, and the combination of 2D convolutional layers-fully connected LSTM.

A Multi-scale feature modulation network for efficient underwater image enhancement