Multi-scale local features based on anisotropic heat diffusion and global eigen-structure

Multi-scale geometric processing has been a popular and powerful tool in graphics, which typically employs isotropic diffusion across scales. This paper proposes a novel method of multi-scale anisotropic heat diffusion on manifold, based on the new normal-driven shape representation and Edge-weighted Heat Kernels (EHK). The new shape representation, named as Normal-Controlled Coordinates (NCC), can encode local geometric details of a vertex along its normal direction and rapidly reconstruct surface geometry. Moreover, the inner product of NCC and its corresponding vertex normal, called Normal Signature (NS), defines a scalar/heat field over curved surface. The anisotropic heat diffusion is conducted using the weighted heat kernel convolution governed by local geometry. The convolution is computed iteratively based on the semigroup property of heat kernels toward accelerated performance. This diffusion is an efficient multi-scale procedure that rigorously conserves the total heat. We apply our new method to multi-scale feature detection, scalar field smoothing and mesh denoising, and hierarchical shape decomposition. We conduct various experiments to demonstrate the effectiveness of our method. Our method can be generalized to handle any scalar field defined over manifold.

Chinese agricultural diseases and pests named entity recognition with multi-scale local context features and self-attention mechanism

Medical image segmentation is a fundamental task in medical image analysis, playing a crucial role in disease diagnosis, treatment planning, and clinical decision-making. Accurate segmentation of anatomical structures, such as blood vessels, organs, and lesions, is essential for reliable medical interpretations. However, existing segmentation models often face challenges in effectively capturing both local and global features within medical images. Traditional convolutional neural networks (CNNs) excel at extracting fine-grained local features but struggle to model long-range dependencies due to their limited receptive fields. Conversely, transformer-based models can capture global contextual relationships but often fail to preserve detailed local structures, leading to suboptimal segmentation performance. The purpose of this study is to develop SEFormer, a novel hybrid network that integrates SENet, ResNet, and Transformer to enhance medical image segmentation. SEFormer aims to effectively capture both local and global features, addressing the limitations of traditional CNNs and transformer-based models.SENet is incorporated to recalibrate feature maps by adaptively emphasizing informative features, thereby compensating for the global representation limitations of CNNs. Meanwhile, the ResNet backbone ensures deep feature extraction while maintaining computational efficiency. Additionally, the Transformer module is introduced to capture long-range dependencies and enhance contextual awareness, complementing the local feature extraction capabilities of CNNs. This hybrid approach allows SEFormer to balance fine-grained spatial details with broader contextual information, leading to more precise segmentation results.Furthermore, to mitigate information loss during feature extraction and ensure a more complete and hierarchical representation of image information, we introduce a feature pyramid structure inspired by multi-scale image pyramid models. By progressively increasing the receptive field across different scales and employing skip connections, SEFormer effectively fuses multi-scale local and global features at each stage of the pyramid. This hierarchical fusion mechanism ensures a richer and more robust feature representation, which is particularly beneficial for segmenting complex medical structures such as blood vessels. We evaluate SEFormer on the CHASE_DB1 dataset, a widely used benchmark for retinal vessel segmentation. Experimental results demonstrate that SEFormer outperforms existing state-of-the-art segmentation methods, achieving a 3.25% improvement in segmentation accuracy. Additionally, we conduct ablation studies to verify the contributions of different components within SEFormer. The results show that incorporating SENet enhances feature recalibration and channel attention, while the Transformer module significantly improves global context awareness. The feature pyramid structure further contributes to performance gains by ensuring a multi-scale representation of vascular structures. Compared to conventional CNN-based methods, our approach achieves better segmentation accuracy with improved robustness to variations in vessel thickness, noise, and image contrast. Furthermore, SEFormer maintains computational efficiency, making it suitable for real-world clinical applications where both accuracy and processing speed are critical. SEFormer provides an efficient and accurate approach for medical image segmentation by leveraging the complementary strengths of CNNs, SENet, and Transformers while integrating a feature pyramid structure to maximize feature representation. Future research directions include extending SEFormer to other medical imaging modalities, such as CT and MRI scans, as well as exploring further optimization techniques to reduce computational costs while maintaining high segmentation performance. Additionally, integrating SEFormer with active learning frameworks could further improve segmentation performance by leveraging expert-annotated data more efficiently.

Image enhancement plays a crucial role in computer vision by improving visual quality while minimizing distortion. Traditional methods enhance images through pixel value transformations, yet they often introduce new distortions. Recent advancements in deep learning-based techniques promise better results but challenge the preservation of image fidelity. Therefore, it is essential to evaluate the visual quality of enhanced images. However, existing quality assessment methods frequently encounter difficulties due to the unique distortions introduced by these enhancements, thereby restricting their effectiveness. To address these challenges, this paper proposes a novel blind image quality assessment (BIQA) method for enhanced natural images, termed multi-scale local feature fusion and global feature representation-based quality assessment (MLGQA). This model integrates three key components: a multi-scale Feature Attention Mechanism (FAM) for local feature extraction, a Local Feature Fusion (LFF) module for cross-scale feature synthesis, and a Global Feature Representation (GFR) module using Vision Transformers to capture global perceptual attributes. This synergistic framework effectively captures both fine-grained local distortions and broader global features that collectively define the visual quality of enhanced images. Furthermore, in the absence of a dedicated benchmark for enhanced natural images, we design the Natural Image Enhancement Database (NIED), a large-scale dataset consisting of 8,581 original images and 102,972 enhanced natural images generated through a wide array of traditional and deep learning-based enhancement techniques. Extensive experiments on NIED demonstrate that the proposed MLGQA model significantly outperforms current state-of-the-art BIQA methods in terms of both prediction accuracy and robustness.

Medical image classification is essential for contemporary clinical diagnosis and decision support systems. However, medical images generally have similar inter-class features and complex structure patterns, making it a challenging task. While both local and global features are critical for noise reduction and discriminative pattern extraction in medical images, conventional approaches exhibit limitations. Specifically, convolutional neural networks (CNNs) focus on local features extraction but lack a comprehensive understanding of global semantic. Conversely, vision transformers (ViTs) can model long-range feature dependencies but may cause disruption to local features. To address these limitations, we propose Hierarchical Enhanced Multi-attention Feature (HEMF), an adaptive hierarchical enhanced multi-attention feature fusion framework to synergistically extract and fuse multi-scale local and global features. It comprises two core components: (1) the enhanced local and global feature extraction modules to extract multi-scale local and global features in parallel; (2) the hierarchical enhanced feature fusion module integrating a novel attention mechanism named Mixed Attention (MA) and a novel inverted residual block named Squeezed Inverted Residual Multi-Layer Perceptron (SIRMLP) to effectively fuse multi-scale features. Experimental results demonstrate that with nearly minimal model parameters compared to other advanced models, HEMF achieves the accuracy and F1-score of 87.34% and 78.89% on the ISIC2018 dataset, 87.03% and 87.02% on the Kvasir dataset, and 82.26% and 82.20% on the COVID-19 CT dataset, which are the state-of-the-art performance. Our code is open source and available from https://github.com/Esgjgd/HEMF.

Smart healthcare has undergone new opportunities and challenges with the arrival of the Industry 4.0 era. The intelligent imaging diagnosis system is a staple part of smart healthcare, helping doctors make clinical decisions. Nevertheless, intelligent diagnosis analysis is still confronted with the issue that it is challenging to extract effective features from the limited and high‐dimensional data, particularly in resting‐state data of amnesic mild cognitive impairment (aMCI). Furthermore, the intelligent imaging diagnosis system for aMCI is conductive to make timely predicting groups that may convert to Alzheimer’s disease (AD). To improve the system’s detection performance and reduce its data redundancy, we first develop an adaptive structure feature generation strategy (ASFGS) based on the Laplacian matrix and sparse autoencoder to obtain the structural features of brain functional network (BFN). Concurrently, we present a multiscale local feature detection strategy (MLFDS) to overcome the low utilization of local features of BFN. And finally, multiscale features, including structural features and multiscale local features, are fused by concatenation method to further improve the detection performance of aMCI system. Support vector machine based on radial basis function (RBF‐SVM) for small data learning is adopted to evaluate the effectiveness of the proposed features. Besides, we employ leave‐one‐out cross‐validation strategy to avoid the overfitting problem of classifier training process. The experiment results elucidate that the accuracy (ACC) and the area under the curve (AUC) in this work provide 86.57% and 86.36%, respectively, which outperforms the traditional methods and offers new insights for accuracy requirements of the aMCI system.

While some image textures can be changed with scale, others cannot. We focus on a multi-scale features of determing the sensitivity of the texture intensity to change. This paper presents a new method of texture structure classification and depth estimation using multi-scale features extracted from a higher order of the local autocorrelation functions. Multi-scale features consist of the meansand variances of distributions, which are extracted from theautocorrelation feature vectors according to multi-level scale. In order to reduce dimensional feature vectors, we employ the Principal Component Analysis (PCA) in the autocorrelation feature space. Each training image texture makes its own multi-scale model in a reduced PCA feature space, and the test of the texture image is projected in the homogeneous PCA space of the training data. The experimental results show that the proposed multi-scale feature can be utilized notonly for texture classification, but also depth estimation in two dimensional images with texture gradients.

There are problems needed to be solved when extracting centerline of structured light strip on multi-pass seam. The problems include complex shape, non-uniform intensity distribution, disconnected caused by welding beans or occlusion and environmental noise etc. To solve these problems, a novel centerline extraction algorithm is proposed based on anisotropic heat diffusion. Putting pixels gray values as the temperature, the algorithm could get centerline composed by highest energy pixels, through heat diffusion along same temperature curve normal directions. The higher accuracy and efficiency of our algorithm is proved by experiments.

Road detection is a basic functionality for driver assistance system such as vehicle and pedestrian detection. The main challenge is to deal with the presence of different objects (pedestrians and vehicles), the continuously varying background, different imaging conditions (different viewpoints, varying weather conditions and changing illumination), different road types (color, shape) and different environments (off-road, urban, highways). Accordingly, we propose a novel road detection approach based on superpixels and anisotropic heat diffusion. The idea is to previously denoise and deblur road images using directional filters. And then to oversegment the enhanced image into small homogeneous regions which are called superpixels with the superpixel segmentation method. According to the thermodynamic energy diffusion theory, we cluster superpixels into several region blocks. Finally, the algorithm extracts the road region combining the prior knowledge with the largest and most coherent principle of cosegmentation. Quantitative and qualitative experiments show that the approach is robust to lighting variations, heavy traffic and shadows. Moreover, the proposed method provides the better performance when compared with several state-of-the-art methods.

In the present work, we propose the multi-scale Local Difference Features (LDF) for improving handwritten signature verification. This descriptor is similar to local binary patterns, since it highlights intensity differences between a pixel and its neighbors. Presently, we propose a multi-scale implementation of LDF in which, neighboring pixels are taken on several radiuses. This helps to adapt LDF to variations of handwritten strokes within signatures. The verification stage is carried out by using Support Vector Machines classifier. The performance assessment is elaborated on two benchmark datasets. Obtained error rates highlight the effectiveness of the proposed features, since they outperform several state of the art results.

MultiTrans: Multi-branch transformer network for medical image segmentation

Infrared and visible image fusion improves the visual representation of scenes. Current deep learning-based fusion methods typically rely on either convolution operations for local feature extraction or Transformers for global feature extraction, often neglecting the contribution of multi-scale features to fusion performance. To address this limitation, we propose MRMAFusion, a nested connection model that relies on the multi-scale restoration-Transformer (Restormer) and multi-dimensional attention. We construct an encoder–decoder architecture on UNet++ network with multi-scale local and global feature extraction using convolution blocks and Restormer. Restormer can provide global dependency and more comprehensive attention to texture details of the target region along the vertical dimension, compared to extracting features by convolution operations. Along the horizontal dimension, we enhance MRMAFusion’s multi-scale feature extraction and reconstruction capability by incorporating multi-dimensional attention into the encoder’s convolutional blocks. We perform extensive experiments on the public datasets TNO, NIR and RoadScene and compare with other state-of-the-art methods for both objective and subjective evaluation.

摘要: 提出了一种基于颜色空间的Local特征和Regional特征的自然图像显著性检测方法. 该方法将图像分成8×8的子块, 计算多个尺度下每一个子块的Local特征和Regional特征, 并将其加权组合来确定子块的显著程度, 从而得到整个图像的显著特征. 此外, 通过计算4个颜色通道上的色度对比度, 获得显著物体的边缘. 将图像的显著特征与显著物体的边缘综合后得到图像中的显著目标. 实验结果显示, 本文提出的方法能够快速、清晰而准确地提取出图像中的显著性目标. 关键词: 显著性检测 / Local特征 / Regional特征 / 多尺度

This paper proposes a hybrid method for face recognition using local features and statistical feature extraction methods. First, a dense set of local feature points are extracted in order to represent a facial image. Each local feature point is described by the keypoint descriptor defined by SIFT feature. Then, the statistical feature extraction methods, PCA and LDA, are applied to the set of local feature descriptors in order to find low dimensional features. With the obtained low dimensional feature vectors, we can conduct face recognition task efficiently using a simple classifier. Through computational experiments on benchmark data sets, we show that the proposed method is superior to the conventional PCA and LDA in the classification performance. In addition, we also show that the proposed method can achieve remarkable improvement in the processing time compared to the conventional keypoint matching methods proposed for local features.KeywordsFace RecognitionLocal FeatureSift FeatureGray Level IntensityConventional Statistical MethodThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

With the significant success of transformers in natural language processing, an increasing number of researchers are introducing them into the field of computer vision, particularly for action recognition. As a crucial task in video understanding, action recognition has significant applications in live broadcasting, autonomous driving, and medical diagnostics. The attention mechanisms in transformers mimicking human visual attention allocation, thereby enhancing the processing capabilities and comprehension of long video sequences. However, they often overlook the aggregation of multiscale detail features and the hierarchical representations of early visual information. Additionally, attention networks are computationally intensive and parameter-heavy, complicating model training and extending inference times, rendering them unsuitable for real-time applications. To crack these nuts, we propose a lightweight multiscale action recognition model based on convolutional enhancement block (ConvEB) and multiscale average pooling encoder. The ConvEB aims to establish long-range dependencies among multiscale local features in the early stages of the network, providing effective inductive biases for the attention network to compensate for the loss of detailed information. Moreover, we introduce a parallel pooling mixer to replace the original attention mixer, ensuring model lightweight while maintaining recognition accuracy. Finally, we deploy this model in the construction of a virtual panorama live broadcasting system. Experimental results demonstrate that our action recognition algorithm achieves competitive performance, and the constructed panoramic system basically meets the needs of daily live broadcasting.