SiamDMCF: a dynamic multi-order context fusion siamese network for robust visual tracking

Objective. Accurate target localization is essential for effective radiation therapy, yet respiratory motion introduces considerable uncertainty. Ultrasound-based motion tracking offers a non-invasive solution, but the presence of similar anatomical structures and significant changes in appearance severely hinders robust and accurate tracking. Robust tracking is essential to ensure accuracy and improve treatment outcomes.Approach. We propose SiamAT, a tracking framework comprising a Siamese-like feature extraction network with a Swin-Transformer backbone, an attention-based target-aware module, and multi-task prediction heads. The target-aware module adaptively integrate template and search features to generate target-aware representations for precise localization.Main results. The proposed SiamAT is evaluated on a public dataset, i.e. MICCAI 2015 challenge on liver US tracking, and our clinical dataset provided by the Chinese People's Liberation Army General Hospital. Experimental results demonstrate that the method achieves accurate and robust tracking(0.60±0.30 mm and 0.47±0.31 mm tacking errors in two datasets) and outperforms existing methods. The proposed method runs at approximatively 36 fps on GPU.Significance. The proposed SiamAT provides clinicians with an effective tool for analyzing 2D motion in US images in clinical settings, assisting in the planning of optimal treatment strategies while reducing diagnostic and therapeutic risks.

In order to solve the problems of unbalanced sample data and the lack of consideration of temporal information in existing Siamese-based trackers, this paper proposes a Siamese recurrent neural network and region proposal network (Siamese R-RPN), which can be trained in an end-to-end manner. Siamese R-RPN is consisted of Siamese network, recurrent neural network and region proposal network. Image features extracted by the Siamese network are strengthened by the channel and spatial attention mechanisms, and are sent to the RPN for classification and regression. Temporal information is processed by a recurrent neural network-based Long Short-Term Memory (LSTM) to predict the rough location of the target, it is mapped to the anchor feature map of the RPN for anchor selection. This makes the positive and negative samples participating in the training procedure to become more balanced and representative. Because of the collaborative use of temporal and spatial information, the tracker proposed in this paper has achieved state-of-the-art performance on three large tracking benchmarks—OTB 2015, VOT2016 and VOT 2018—where this verifies its effectiveness.

News recommendation systems must simultaneously address multiple challenges, including dynamic user interest modeling, nonlinear popularity patterns, and diversity recommendation in cold-start scenarios. We present a Popularity-Aware Dynamic Multi-Perspective Fusion Network (PAD-MPFN) that innovatively integrates three key components: adaptive subspace projection for multi-source interest fusion, logarithmic time-decay factors for popularity bias mitigation, and dynamic gating mechanisms for personalized recommendation weighting. The framework uniquely combines sequential behavior analysis, social graph propagation, and temporal popularity modeling through a unified architecture. Experimental results on the MIND dataset, an open-source version of MSN News, demonstrate that PAD-MPFN outperforms existing methods in terms of recommendation performance and cold-start scenarios while effectively alleviating information overload. This study offers a new solution for dynamic interest modeling and diverse recommendation.

In recent years, the use of Convolutional Neural Networks in medical imaging applications has been increasing drastically. Siamese Networks are among the top architectures that are being borrowed for motion tracking. In this work, we propose an improved Siamese network for robust and accurate landmark tracking in ultrasound image sequences. Despite the staggering efficacy of Siamese networks, the inherent assumptions in the original architecture lead to certain limitations. The assumption of the constant position model and not incorporating any motion model remain unaddressed. In our proposed model, we employ a reference template update module to resolve the assumption of the constant position model and incorporate a Linear Kalman Filter to address the issue of the missing motion model. The proposed model was evaluated for robustness on the training set of the Liver Ultrasound Tracking (CLUST) 2D data set. The robustness of the architecture was evaluated by inducing synthetic occlusions around the landmark. The performance was then compared to the original architecture. Experiments proved that the proposed model outperformed the original model by a significant margin.

Visual object tracking is a hot topic in recent years. In the meanwhile, Siamese networks have attracted extensive attention in this field because of its balanced precision and speed. However, most of the Siamese network methods can only distinguish foreground from the non-semantic background. The fine-tuning and retraining of fully-convolutional Siamese networks for object tracking(SiamFC) can achieve higher precision under interferences, but the tracking accuracy is still not ideal, especially in the environment with more target interferences, dim light, and shadows. In this paper, we propose criss-cross attentional Siamese networks for object tracking (SiamCC). To solve the imbalance between foreground and non-semantic background, we use the feature enhancement module of criss-cross attention to greatly improve the accuracy of video object tracking in dim light and shadow environments. Experimental results show that the maximum running speed of SiamCC in the object tracking benchmark dataset is 90 frames/second. In terms of detection accuracy, the accuracy of shadow sequences is greatly improved, especially the accuracy score of sequence HUMAN8 is improved from 0.09 to 0.89 compared with the original SiamFC, and the success rate score is improved from 0.07 to 0.55.

Siamese networks have been successfully introduced into visual tracking, which match the best candidate and a target template via a couple of networks with shared parameters. However, most Siamese network-based trackers (SNTs) are tailored to best match the canonical posture of the template and the search-region images, resulting in inferior performance when the target objects have large-scale pose variations. Besides, SNTs fail to discriminate distractors well because they only leverage high-level semantic features as target representations that cannot well tell from different targets of the same category. To address these issues, this paper presents an efficient and effective SNT that is based on feature alignment and aggregation networks. Specifically, we first design an effective feature alignment network module to calibrate the search-region image. This module results in a more reliable matching response that is robust to severe target pose variations. Then, we develop an effective shallow-level and high-level feature aggregation network module to complement the feature characteristics, making the learned feature representation not only well differentiate the target from distractors, but also robust to target appearance variations. Afterwards, we employ a channel-attention mechanism to further strengthen the discriminative capability of the aggregated feature representation. Finally, both the alignment and the aggregation modules are seamlessly integrated into the Siamese networks for robust tracking. Meanwhile, we offline learn the network parameters end-to-end without time-consuming fine-tuning. Extensive evaluations on a variety of benchmarks including VOT-2017, OTB-100, UAV123 and GOT-10k demonstrate favorable performance of our tracker against state-of-the-art ones with a speed of 60 fps.

Two-stage aware attentional Siamese network for visual tracking

In home-based care, reliable contextual information of remotely monitored patients should be generated to recognize activities and to identify hazardous situations of the patient. This is difficult for several reasons. First, low level data obtained from multisensor have different degrees of uncertainty. Second, generated contexts can be conflicting even if they are acquired by simultaneous operations. And last, context reasoning over time is difficult for temporal changes in sensory information. In this paper, we propose the dynamic evidential fusion approach as a context reasoning method in home-based care. The proposed approach processes the generated contexts with Dynamic Evidential Network (DEN), which is composed of the combination of Dezert-Smarandache Theory (DSmT) and Markov Chain (MC). The DSmT reduces ambiguous or conflicting contextual information and the MC processes the association and correlation of sensory information that may change based on time series. Finally, we compare the dynamic evidential fusion approach with the static evidential fusion approach for analyzing the improvement of the dynamic evidential fusion approach.

Dynamic evidence fusion neural networks with uncertainty theory and its application in brain network analysis

For face recognition systems, liveness detection can effectively avoid illegal fraud and improve the safety of face recognition systems. Common face attacks include photo printing and video replay attacks. This paper studied the differences between photos, videos, and real faces in static texture and motion information and proposed a living detection structure based on feature fusion and attention mechanism, Dynamic and Texture Fusion Attention Network (DTFA-Net). We proposed a dynamic information fusion structure of an interchannel attention block to fuse the magnitude and direction of optical flow to extract facial motion features. In addition, for the face detection failure of HOG algorithm under complex illumination, we proposed an improved Gamma image preprocessing algorithm, which effectively improved the face detection ability. We conducted experiments on the CASIA-MFSD and Replay Attack Databases. According to experiments, the DTFA-Net proposed in this paper achieved 6.9% EER on CASIA and 2.2% HTER on Replay Attack that was comparable to other methods.

Target-Cognisant Siamese Network for Robust Visual Object Tracking

Convolutional neural networks are potent models that yield hierarchies of features and have drawn increasing interest in the visual tracking field. In the paper, we design an end-to-end trainable tracking framework based on Siamese network, which proposes to learn the low-level fine-grained and high-level semantic representations simultaneously with the aim of mutual benefit. Due to the distinct and complementary characteristics of the feature hierarchies, different tracking mechanisms are adopted for different feature layers. The low-level features are exploited and updated with a correlation filter layer for adaptive tracking and the high-level features are compared through cross-correlation directly for robust tracking. The two-level features are jointly trained with a multi-task loss function end-to-end. The proposed tracker takes full advantage of the adaptability of the low-level features and the generalization ability of the high-level features. Extensive experimental tracking results on the widely used OTB and TC128 benchmarks demonstrate the superiority of our tracker. Meanwhile, our proposed tracker can achieve a real-time tracking speed.

Background:Many mental disorders show strong genetic influence. In parallel, dynamic functional network connectivity (dFNC) has shown high sensitivity to brain changes related to mental disorders. However, previous studies linking dFNC to genetics largely follow a paradigm to identify associations between one set of genetic factors and multiple sets of connectivity features from different dFNC states, ignoring the potential variability in genetic correlates across states.Methods:We propose a novel joint ICA (jICA)-based “dynamic fusion” framework to identify dynamically-tuned genetic manifolds. A sliding window approach was utilized to estimate four dFNC states and compute subject-level state-average dFNC (sa-dFNC) features. The sa-dFNC features of each state were combined with schizophrenia risk SNPs within a jICA fusion framework, resulting in four parallel fusions in 32861 individuals of the UK Biobank cohort. The extracted four sets of joint SNP-dFNC components were further validated for clinical relevance in a combined schizophrenia cohort of 1237 individuals (528 patients). The similarity of SNP-dFNC components across four parallel fusions was evaluated as a measure of state variability.Results:We observed a mixture of “state-invariant” and “state-variant” components for SNP and dFNC modalities. Particularly, the schizophrenia-related state-variant SNP components, or manifolds, complemented each other by capturing different genes involved in the same biological functions, revealing a partition of genomic risk particularly elicited by the dynamics of brain function.Conclusions:By augmenting the SNP factors to state-variant manifolds, this dynamic fusion framework promises additional insights into underlying genetic risk of disease-related alterations in dynamic brain function.

Short-term passenger flow prediction is crucial for urban traffic management and crowd control. However, traditional graph neural network methods usually use static adjacency matrices in urban rail transit prediction, ignoring the dynamic changes of inter-station correlations throughout the day. To address this challenge, we propose a new model called Dynamic Multigraph Fusion Spatio-Temporal Graph Convolutional Network (DMSTGCN). This model uses Tucker decomposition to construct a time-varying dynamic graph, capturing the spatial correlations of passenger flow at different time intervals and overcoming the limitations of static matrices in previous studies while effectively mitigating overfitting. Three predefined graphs, namely the physical structure graph, the passenger flow similarity graph and the OD correlation graph, are also integrated to form the fusion graph convolution module, which further enriches the multivariate spatial correlation of passenger flows. Meanwhile, the causal temporal multi-head self-attention mechanism extracts temporal correlations, working with the fusion graph convolution module in a spatio-temporal learner for multilayer spatio-temporal feature extraction. Finally, experiments on three real-world datasets prove the excellence of DMSTGCN and offer interpretable dynamic spatial relationships.

Multi-source data classification is a critical yet challenging task for remote sensing image interpretation. Existing methods lack adaptability to diverse land cover types when modeling frequency domain features. To this end, we propose a Dynamic Frequency Feature Fusion Network (DFFNet) for hyperspectral image (HSI) and Synthetic Aperture Radar (SAR) / Light Detection and Ranging (LiDAR) data joint classification. Specifically, we design a dynamic filter block to dynamically learn the filter kernels in the frequency domain by aggregating the input features. The frequency contextual knowledge is injected into frequency filter kernels. Additionally, we propose spectral-spatial adaptive fusion block for cross-modal feature fusion. It enhances the spectral and spatial attention weight interactions via channel shuffle operation, thereby providing comprehensive cross-modal feature fusion. Experiments on two benchmark datasets show that our DFFNet outperforms state-of-the-art methods in multi-source data classification. The codes will be made publicly available at https://github.com/oucailab/DFFNet.