An adaptive cross‐scale transformer based on graph signal processing for person re‐identification

Two-phase self-supervised pretraining for object re-identification

Abstract. Person Re-Identification (Re-ID) is the task of matching the same person in multiple images captured by different cameras. Recently deep learning-based Re-ID algorithms demonstrated an exciting progress for terrestrial-based cameras, still, person Re-ID in aerial images poses multiple challenges including occlusion of human feature parts, image distortion, and dynamic camera location. In this paper, we propose a new Person Aerial Re-ID framework Robust to Occlusion and Thermal imagery (ParrotGAN). Our model is focused on cross-modality person Re-ID in aerial images. Furthermore, we collected a new large-scale synthetic multimodal AerialReID dataset with 30k images and 137 person ID. Our ParrotGAN model leverages two strategies to achieve robust performance in the task of person Re-ID in thermal and visible range. Firstly, we use a latent space of StyleGAN2 model to estimate the distance between two images of a person. Specifically, we project each real image into the latent space with a correspondent latent vector z. We use the distance between latent vectors to provide a Re-ID similarity metric. Secondly, we use a generative-adversarial network to translate a color image to a synthetic thermal image. We use synthetic image for a cross-modality Re-ID. We evaluate our ParrotGAN model and basleines on our AerialReID and PRAI-1581 datasets. The results of the evaluation are encouraging and demonstrate that our ParrotGAN model competes with baselines in visible range aerial person Re-ID and outperforms them in the cross-modality setting. We made our code and dataset publicly available.

A review on video person re-identification based on deep learning

Compact feature representation of person image is important for person re-identification (Re-ID) task. Recently, part-based representation models have been widely studied for extracting the more compact and robust feature representation for person image to improve person Re-ID results. However, existing part-based representation models mostly extract the features of different parts independently which ignore the spatial relationship information among different parts. To address this issue, in this paper we propose a novel deep learning framework, named Part-based Hierarchical Graph Convolutional Network (PH-GCN) for person Re-ID problem. Given a person image, PH-GCN first constructs a hierarchical graph to represent the spatial relationships among different parts. Then, both local and global feature learning is achieved by the feature information passing in PH-GCN, which takes the information of other parts into account for part feature representation. Finally, a perceptron layer is adopted for the final person part label prediction and re-identification. The proposed framework provides a general solution that integrates <i>local</i>, <i>global</i> and <i>structural</i> feature learning simultaneously in a unified end-to-end network representation and learning. Extensive experiments on several widely used benchmark datasets demonstrate the effectiveness and benefits of the proposed PH-GCN approach for person Re-ID task.

Graph is a great model that can efficiently deal with structural data and has excellent research value and application value in our life. Graph theory and signal processing have been perfectly combined to form graph signal processing theory, which provides a theory for studying high-dimensional and structural signals. The wireless sensor network (WSN) is a typical data transmission network based on a graph structure. Because of redundant sensor nodes in WSN, it results in large energy consumption and slow transmission rate. To solve this problem, we propose a new algorithm to optimize the number of nodes by using graph signals processing and compressed sensing (CS) theory. The new node optimization algorithm can not only reduce energy consumption but also increase the transmission rate.

Prevalent nighttime person re-identification (ReID) methods typically combine image relighting and ReID networks in a sequential manner. However, their performance (recognition accuracy) is limited by the quality of relighting images and insufficient collaboration between image relighting and ReID tasks. To handle these problems, we propose a novel Collaborative Enhancement Network called CENet, which performs the multilevel feature interactions in a parallel framework, for nighttime person ReID. In particular, the designed parallel structure of CENet can not only avoid the impact of the quality of relighting images on ReID performance, but also allow us to mine the collaborative relations between image relighting and person ReID tasks. To this end, we integrate the multilevel feature interactions in CENet, where we first share the Transformer encoder to build the low-level feature interaction, and then perform the feature distillation that transfers the high-level features from image relighting to ReID, thereby alleviating the severe image degradation issue caused by the nighttime scenario while avoiding the impact of relighting images. In addition, the sizes of existing real-world nighttime person ReID datasets are limited, and large-scale synthetic ones exhibit substantial domain gaps with real-world data. To leverage both small-scale real-world and large-scale synthetic training data, we develop a multi-domain learning algorithm, which alternately utilizes both kinds of data to reduce the inter-domain difference in training procedure. Extensive experiments on two real nighttime datasets, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Night600</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RGBNT201_rgb</i> , and a synthetic nighttime ReID dataset are conducted to validate the effectiveness of CENet. We release the code and synthetic dataset at: https://github.com/Alexadlu/CENet.

Medical image segmentation is a fundamental task in medical image analysis, playing a crucial role in disease diagnosis, treatment planning, and clinical decision-making. Accurate segmentation of anatomical structures, such as blood vessels, organs, and lesions, is essential for reliable medical interpretations. However, existing segmentation models often face challenges in effectively capturing both local and global features within medical images. Traditional convolutional neural networks (CNNs) excel at extracting fine-grained local features but struggle to model long-range dependencies due to their limited receptive fields. Conversely, transformer-based models can capture global contextual relationships but often fail to preserve detailed local structures, leading to suboptimal segmentation performance. The purpose of this study is to develop SEFormer, a novel hybrid network that integrates SENet, ResNet, and Transformer to enhance medical image segmentation. SEFormer aims to effectively capture both local and global features, addressing the limitations of traditional CNNs and transformer-based models.SENet is incorporated to recalibrate feature maps by adaptively emphasizing informative features, thereby compensating for the global representation limitations of CNNs. Meanwhile, the ResNet backbone ensures deep feature extraction while maintaining computational efficiency. Additionally, the Transformer module is introduced to capture long-range dependencies and enhance contextual awareness, complementing the local feature extraction capabilities of CNNs. This hybrid approach allows SEFormer to balance fine-grained spatial details with broader contextual information, leading to more precise segmentation results.Furthermore, to mitigate information loss during feature extraction and ensure a more complete and hierarchical representation of image information, we introduce a feature pyramid structure inspired by multi-scale image pyramid models. By progressively increasing the receptive field across different scales and employing skip connections, SEFormer effectively fuses multi-scale local and global features at each stage of the pyramid. This hierarchical fusion mechanism ensures a richer and more robust feature representation, which is particularly beneficial for segmenting complex medical structures such as blood vessels. We evaluate SEFormer on the CHASE_DB1 dataset, a widely used benchmark for retinal vessel segmentation. Experimental results demonstrate that SEFormer outperforms existing state-of-the-art segmentation methods, achieving a 3.25% improvement in segmentation accuracy. Additionally, we conduct ablation studies to verify the contributions of different components within SEFormer. The results show that incorporating SENet enhances feature recalibration and channel attention, while the Transformer module significantly improves global context awareness. The feature pyramid structure further contributes to performance gains by ensuring a multi-scale representation of vascular structures. Compared to conventional CNN-based methods, our approach achieves better segmentation accuracy with improved robustness to variations in vessel thickness, noise, and image contrast. Furthermore, SEFormer maintains computational efficiency, making it suitable for real-world clinical applications where both accuracy and processing speed are critical. SEFormer provides an efficient and accurate approach for medical image segmentation by leveraging the complementary strengths of CNNs, SENet, and Transformers while integrating a feature pyramid structure to maximize feature representation. Future research directions include extending SEFormer to other medical imaging modalities, such as CT and MRI scans, as well as exploring further optimization techniques to reduce computational costs while maintaining high segmentation performance. Additionally, integrating SEFormer with active learning frameworks could further improve segmentation performance by leveraging expert-annotated data more efficiently.

This paper investigates the relation between graph signal processing and consensus of multi-agent systems. The graph signal processing is a technique to process graph signals, that is, signals whose values are on the vertices of graphs and whose structures are specified by the edges. By considering the combination of the states of agents and the graph describing the network structure between them as a graph signal, we show that the multi-agent consensus corresponds to low-pass filtering of the graph signal. This reveals a connection between the two distinct areas, i.e., the graph signal processing and the control of multi-agent systems. In addition, we provide a design method of consensus controllers based on the graph signal processing. In the proposed method, the controllers of agents are designed so that the spatial frequency of the states of the agents becomes a desired one. This enables us to construct controllers of multi-agent systems in the spatial frequency domain.

At present, occlusion and appearance similarity pose severe challenges to person re-identification tasks. Although many robust deep convolutional neural networks alleviate these problems, convolutional layers with limited receptive fields cannot model global semantic information well. In addition, in the person re-identification model, many metric losses ignore or destroy the intra-class structure of the sample, which makes the model difficult to be optimized. Therefore, we design a discriminative Re-identification model with global-local attention and adaptive weighted rank list loss (GLWR). Specifically, our global-local attention (GL-Attention) learns the semantic context in the channel and spatial dimensions. By learning the dependencies between features, GL-Attention integrates global semantic information into local features to extract discriminative features. Unlike rank list loss, our adaptive weighted rank list loss (WRLL) adaptively assigns weights according to the metric distance between the negative sample and the input image, which further improves the performance of the model. Experimental studies on three public datasets (Market-1501, DukeMTMC-ReID and CUHK03) indicate that the performance of our GLWR is significantly superior to many of the latest algorithms.

State-of-the-art data analysis tools have to deal with high-dimensional data. Fortunately, the inherent dimensionality of data is often much smaller, as it has an internal structure limiting its degrees of freedom. In most cases, this structure can be approximated using a graph, i.e, a set of nodes connected by edges. Based on this idea, graphs have been largely used in semi-supervised and unsupervised learning. The canonical assumption when incorporating a graph prior is to assume that the signal is smooth with respect to the graph, i.e, that connected pieces of data have similar values. Nevertheless, this hypothesis is insufficient to characterize more complex relations between the data and its structure, the graph. To this end, the field of Graph Signal Processing (GSP) extends the graph smoothness hypothesis to a more general assumption. Its key idea is to think of the signal as a sum of harmonic modes, which are obtained from the eigen-decomposition of the Laplacian, an operator representing the second order derivative. Thus, the essence of GSP resides in the Fourier transform defined by the projection on the Laplacian eigenvectors basis, which allows us to filter graph signals. However, GSP suffers from a counterintuitive irregularity. Contrarily to the classical Fourier basis, the graph Laplacian eigenvectors are not spreading uniformly on the vertex set and can be highly concentrated in a region of the graph. This non-uniformity results in a lack of any intuitive translation operator and consequently poses challenges to generalize some of the classical signal processing theory, e.g., stationarity, uncertainty principles, or sampling. In this thesis, we answer these difficulties by accepting that each node is unique and should be treated specially according to its surrounding structure. This is achieved using an operator called localization that shifts a kernel defined in the spectral domain around a specific node while adapting to the local graph structure. In the first part of the thesis, we focus on harmonic analysis in GSP. We start by making use of the norm of localized kernels to capture the spectral local features of the graph vertices. To illustrate the relevance of this analysis we then introduce structural clustering, an algorithm that groups nodes according to the role they play in the graph. Then, we bound the vertex-frequency concentration of graph filter banks atoms. Because of the irregular structure of graphs, we construct a local uncertainty principle that is, again, driven by the norm of a localized kernel. In the second part of the thesis, we tackle GSP problems from a machine learning perspective. We use the localization operator to extend the notion of stationarity to graph signals. It consists of a graph signal probabilistic framework allowing us to optimally solve inverse problems. Finally, we show how the graph total variation can be used in semi-supervised learning and prove its consistency with respect to the underlying graph manifold.

Person re-identification (Re-ID) has advanced significantly through the integration of deep learning techniques, with face recognition serving as a foundational component. This study presents a comprehensive analysis of state-of-the-art approaches spanning face detection, alignment, recognition, and cross-camera person Re-ID. Deep Convolutional Neural Networks (CNNs), attention mechanisms, and generative models (GANs) drive progress in robust feature extraction, occlusion handling, and domain adaptation. Landmark techniques such as DeepFace, DeepID2+, and center loss have achieved near-human face verification accuracy, while cascaded CNNs and Kalman-filter-based tracking enhance detection and temporal consistency in video surveillance. Emerging trends include transformer-based models, multi-modal biometric fusion, and edge-cloud optimization for scalable deployment. However, challenges remain in cross-domain generalization, fairness, and full-body Re-ID integration. This synthesis identifies critical research gaps and underscores the need for holistic, real-time, and ethically sound Re-ID systems capable of operating under diverse real-world conditions.

The emerging field of graph signal processing (GSP) allows one to transpose classical signal processing operations (e.g., filtering) to signals on graphs. The GSP framework is generally built upon the graph Laplacian, which plays a crucial role in studying graph properties and measuring graph signal smoothness. Here, instead, we propose the graph modularity matrix as the centerpiece of GSP to incorporate knowledge about graph community structure when processing signals on the graph but without the need for community detection. We study this approach in several generic settings, such as filtering, optimal sampling and reconstruction, surrogate data generation, and denoising. Feasibility is illustrated by a small-scale example and a transportation network data set as well as one application in human neuroimaging where community-aware GSP reveals relationships between behavior and brain features that are not shown by Laplacian-based GSP. This work demonstrates how concepts from network science can lead to new, meaningful operations on graph signals.

The necessity to process signals living in non-Euclidean domains, such as signals defined on the top of a graph, has led to the extension of signal processing techniques to the graph setting. Among different approaches, graph signal processing distinguishes itself by providing a Fourier analysis of these signals. Analogously to the Fourier transform for time and image signals, the graph Fourier transform decomposes the graph signals decomposes in terms of the harmonics provided by the underlying topology. For instance, a graph signal characterized by a slow variation between adjacent nodes has a low frequency content. Along with the graph Fourier transform, graph filters are the key tool to alter the graph frequency content of a graph signal. This thesis focuses on graph filters that are performed distributively in the node domain–that is, each node needs to exchange information only within its neighbor to perform a given filtering operation. Similarly to the classical filters, we propose ways to design and implement distributed finite impulse response and infinite impulse response graph filters. One of the key contributions of this thesis is to bring the temporal dimension to graph signal processing and build upon a graph-time signal processing framework. This is done in different ways. First, we analyze the effects that the temporal variations on the graph signal and graph topology have on the filtering output. Second, we introduce the notion of joint graph-time filtering. Third, we presentpr a statistical analysis of the distributed graph filtering when the graph signal and the graph topology change randomly in time. Finally, we extend the sampling framework from the reconstruction of graph signals to the observation and tracking of time-varying graph processes. We characterize the behavior of the distributed autoregressivemoving average (ARMA) graph filters when the graph signal and the graph topology are time-varying. The latter analysis is exploited in two ways: i ) to quantify the limitations of graph filters in a dynamic environment, such as a moving sensors processing a time-varying signal in a sensor network; and i i ) to provide ways for filtering with low computation and communication complexity time-varying graph signals. We develop the notion of distributed graph-time filtering, which is an operation that jointly processes the graph frequencies of a time-varying graph signal on one hand and its temporal frequencies on the other hand. We propose distributed finite impulse response and infinite impulse response recursions to implement a two-dimensional graphtime filtering operation. Finally, we propose design strategies to find the filter coefficients that approximate a desired two-dimensional frequency response. We extend the analysis of graph filters to a stochastic environment, i.e., when the graph topology and the graph signal change randomly over time. By characterizing the first and second order moments of the filter output, we quantify the impact of the graph signal and the graph topology randomness into the distributed filtering operation. The latter allows us to develop the notion of graph filtering in the mean, which is also used to ease the computational burden of classical graph filters. Finally, we propose a sampling framework for time-varying graph signals. Particularly, when the graph signal changes over time following a state-space model, we extend the graph signal sampling theory to the tasks of observing and tracking the time-varying graph signal froma few relevant nodes. The latter theory considers the graph signal sampling as a particular case and shows that tools from sparse sensing and sensor selection can be used for sampling.

Person re-identification (Re-ID) aims to retrieve a person of interest across multiple nonoverlapping cameras. In the last few years, the construction of aerial person Re-ID datasets has been appealing because visual surveillance through unmanned aerial vehicle (UAV) platforms has become very valuable in real-world scenarios. However, great differences are exhibited between the pedestrian images captured by ground cameras and those captured by UAVs. Person Re-ID methods based on ground person images have difficulty when performing Re-ID based on aerial person images. In this paper, we first propose a novel meta-transfer learning method for person Re-ID in aerial imagery; this approach trains a generalisable Re-ID model to learn discriminative feature representations for aerial person images. Specifically, a meta-learning strategy is introduced to study a feature extractor, and a transfer learning strategy is introduced to utilise and further improve the acquired meta-knowledge. To overcome the convergence speed and recognition accuracy reductions caused by the presence of difficult categories in the given dataset, we propose a learning strategy based on curriculum sampling that is harmonised with our meta-transfer learning framework. In addition, a new metric formulation of sample similarity is introduced based on Mahalanobis distance to improve the optimisation of the model. Extensive comparative evaluation experiments are conducted on the large-scale aerial Re-ID dataset, and the results obtained show that our method achieves a Rank-1 accuracy of 63.63 $$\%$$ and a mean average precision (mAP) of 38.02 $$\%$$ , demonstrating its potential for completing person Re-ID in aerial images. Ablation studies also validate that each component contributes to improving the performance of the model.

Towards fast and kernelized orthogonal discriminant analysis on person re-identification