Texture Structure Classification and Depth Estimation using Multi-Scale Local Autocorrelation Features

MSST-ResNet: Deep multi-scale spatiotemporal features for robust visual object tracking

Smart healthcare has undergone new opportunities and challenges with the arrival of the Industry 4.0 era. The intelligent imaging diagnosis system is a staple part of smart healthcare, helping doctors make clinical decisions. Nevertheless, intelligent diagnosis analysis is still confronted with the issue that it is challenging to extract effective features from the limited and high‐dimensional data, particularly in resting‐state data of amnesic mild cognitive impairment (aMCI). Furthermore, the intelligent imaging diagnosis system for aMCI is conductive to make timely predicting groups that may convert to Alzheimer’s disease (AD). To improve the system’s detection performance and reduce its data redundancy, we first develop an adaptive structure feature generation strategy (ASFGS) based on the Laplacian matrix and sparse autoencoder to obtain the structural features of brain functional network (BFN). Concurrently, we present a multiscale local feature detection strategy (MLFDS) to overcome the low utilization of local features of BFN. And finally, multiscale features, including structural features and multiscale local features, are fused by concatenation method to further improve the detection performance of aMCI system. Support vector machine based on radial basis function (RBF‐SVM) for small data learning is adopted to evaluate the effectiveness of the proposed features. Besides, we employ leave‐one‐out cross‐validation strategy to avoid the overfitting problem of classifier training process. The experiment results elucidate that the accuracy (ACC) and the area under the curve (AUC) in this work provide 86.57% and 86.36%, respectively, which outperforms the traditional methods and offers new insights for accuracy requirements of the aMCI system.

A multi-scale semantic feature fusion method for remote sensing crop classification

Scene classification is an important tool for remote sensing image interpretation, and it has fundamental applications in research and industry. However, given complex backgrounds and scale variations, remote sensing images have large intraclass diversity and interclass similarity, which bring challenges to accurate classification of remote sensing images. We proposed a scene classification method using joint learning and multiscale attention to alleviate the aforementioned problems. To fully utilize the multiscale information of the image and improve the adaptability of the proposed method to objects with various sizes, different from general methods that fuse different scales of features for classification, joint learning using multiscale features is developed to optimize the whole network. Specifically, we leverage a pretrained deep convolutional neural network as the feature extractor to extract low-level, medium-level, and high-level feature maps from the images. Then, due to the poor semantics of low-level and medium-level feature maps compared with the high-level feature maps, we design a multiscale attention module to enhance the semantic information and suppress the noise information. Finally, the global mean pooling is used to obtain the feature vectors and different classifiers are used for different feature vectors. And the decision-level fusion is adopted to obtain more reliable predictions. The experimental results on the AID and NWPU-RESISC45 datasets show that the proposed method makes a significant improvement in terms of overall accuracies compared with the baselines. And the overall accuracies of our method on the two datasets are 97.49% and 95.20%, respectively, which achieves state-of-the-art performance. The code will be public at a Github repository available at https://github.com/Cbanyungong/JLMSAF.

In this study, we present a method for all-around depth estimation from multiple omnidirectional images for indoor environments. In particular, we focus on plane-sweeping stereo as the method for depth estimation from the images. We propose a new icosahedron-based representation and ConvNets for omnidirectional images, which we name "CrownConv" because the representation resembles a crown made of origami. CrownConv can be applied to both fisheye images and equirectangular images to extract features. Furthermore, we propose icosahedron-based spherical sweeping for generating the cost volume on an icosahedron from the extracted features. The cost volume is regularized using the three-dimensional CrownConv, and the final depth is obtained by depth regression from the cost volume. Our proposed method is robust to camera alignments by using the extrinsic camera parameters; therefore, it can achieve precise depth estimation even when the camera alignment differs from that in the training dataset. We evaluate the proposed model on synthetic datasets and demonstrate its effectiveness. As our proposed method is computationally efficient, the depth is estimated from four fisheye images in less than a second using a laptop with a GPU. Therefore, it is suitable for real-world robotics applications. Our source code is available at https://github.com/matsuren/crownconv360depth.

Chinese agricultural diseases and pests named entity recognition with multi-scale local context features and self-attention mechanism

With the significant success of transformers in natural language processing, an increasing number of researchers are introducing them into the field of computer vision, particularly for action recognition. As a crucial task in video understanding, action recognition has significant applications in live broadcasting, autonomous driving, and medical diagnostics. The attention mechanisms in transformers mimicking human visual attention allocation, thereby enhancing the processing capabilities and comprehension of long video sequences. However, they often overlook the aggregation of multiscale detail features and the hierarchical representations of early visual information. Additionally, attention networks are computationally intensive and parameter-heavy, complicating model training and extending inference times, rendering them unsuitable for real-time applications. To crack these nuts, we propose a lightweight multiscale action recognition model based on convolutional enhancement block (ConvEB) and multiscale average pooling encoder. The ConvEB aims to establish long-range dependencies among multiscale local features in the early stages of the network, providing effective inductive biases for the attention network to compensate for the loss of detailed information. Moreover, we introduce a parallel pooling mixer to replace the original attention mixer, ensuring model lightweight while maintaining recognition accuracy. Finally, we deploy this model in the construction of a virtual panorama live broadcasting system. Experimental results demonstrate that our action recognition algorithm achieves competitive performance, and the constructed panoramic system basically meets the needs of daily live broadcasting.

Multi-scale local feature detection enables downstream registration and recognition tasks in medical image analysis. This paper articulates a novel robust method for multi-scale local feature extraction on volumetric data. The central idea is the elegant unification of local/global eigen-structures within the powerful framework of anisotropic heat diffusion. First, the local vector field is constructed by way of Hessian matrix and its eigenvectors/eigenvalues. Second, anisotropic heat kernels are computed using the vector field’s global graph Laplacian. Robust local features are manifested as extrema across multiple time scales, serving as volumetric heat kernel signature. To tackle the computational challenge for massive volumetric data, we propose a multiresolution strategy for hierarchical feature extraction based on our feature-preserving down-sampling approach. As a result, heat kernels and local feature identification can be approximated at a coarser level first, and then are pinpointed in a localized region at a finer resolution. Another novelty of this work lies at the initial heat design directly using local eigenvalue for anisotropic heat diffusion across the volumetric domain. We conduct experiments on various medical datasets, and draw comparisons with 3D SIFT method. The diffusion property of our local features, which can be interpreted as random walks in statistics, makes our method robust to noise, and gives rise to intrinsic multi-scale characteristics.

Currently available thermal image depth estimation methods are difficult to efficiently extract fine multi-scale feature information from thermal images and suffer from the problem of blurring details at the edges of the estimated depth map. To address these challenges, this paper proposes MSDFNet, a multi-scale detail feature fusion encoder–decoder network, for self-supervised monocular thermal image depth estimation. The model is based on a channel expansion hourglass residual lightweight feature encoder, which can capture rich and fine-grained multi-scale feature information with low computational effort. MSDFNet utilizes a detail feature weight evaluation decoder to fuse cross-scale features and reevaluate the importance of each feature, thereby emphasizing critical edge information at multiple scales. Additionally, MSDFNet incorporates a depth consistency loss function, which provides self-supervised signals for the detailed features of thermal images and improves the optimization of network performance. The method is applied to the ViViD++ and MS2 datasets and achieves state-of-the-art depth estimation performance compared to existing state-of-the-art algorithms. In the Indoor Dark scenario of the ViViD++ dataset, the Abs Rel, Sq Rel, RMSE, and RMSE log error metric values of MSDFNet are reduced by 6.71%, 11.92%, 9.09%, and 5.73%, respectively, while the accuracy metric values δ < 1.25 i , i = 1 , 2 , 3 were improved by 4.18%, 1.13%, and 0.2%, respectively. In addition, MSDFNet proves its excellent generalization ability on the MS2 dataset. The Abs Rel and RMSE error values in the night scene are reduced by 45.6% and 30.09%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 20.95% and 1.33%, respectively. The Abs Rel and RMSE values in the rainy day scenario are reduced by 1.33% and 1.21%, respectively, and the accuracy δ < 1.25 i , i = 1 , 3 is improved by 0.24% and 0.83%, respectively.

The quality control and evaluation of traffic detector data are a prerequisite for subsequent applications. Considering that the PCA method is not ideal when detecting fault information with time-varying and multi-scale features, an improved MSPCA model is proposed in this paper. In combination with wavelet packet energy analysis and principal component analysis, data fault identification for traffic detectors is realized. On the basis of traditional multi-scale principal component analysis, detailed information is obtained by wavelet packet multi-scale decomposition, and a principal component analysis model is established in different scale matrices; fault data is separated by wavelet packet energy difference; according to the time characteristics and space of the detector data Correlation fixes fault data. Through case analysis, the feasibility of the method was verified.

Self-supervised depth estimation methods can achieve competitive performance using only unlabeled monocular videos, but they suffer from the uncertainty of jointly learning depth and pose without any ground truths of both tasks. Supervised framework provides robust and superior performance but is limited by the scope of the labeled data. In this paper, we introduce SENSE, a novel learning paradigm for self-supervised monocular depth estimation that progressively evolves the prediction result using supervised learning, but without requiring labeled data. The key contribution of our approach stems from the novel use of the pseudo labels - the noisy depth estimation from the self-supervised methods. We surprisingly find that a fully supervised depth estimation network trained using the pseudo labels can produce even better results than its "ground truth". To push the envelope further, we then evolve the self-supervised backbone by replacing its depth estimation branch with that fully supervised network. Based on this idea, we devise a comprehensive training pipeline that alternatively enhances the two key branches (depth and pose estimation) of the self-supervised backbone network. Our proposed approach can effectively ease the difficulty of multi-task training in self-supervised depth estimation. Experimental results have shown that our proposed approach achieves state-of-the-art results on the KITTI dataset.

A support vector machine (SVM) is an important method in the detection and classification of the color difference on a polysilicon wafer. However, the accuracy of a SVM is affected by its feature vector and parameters. Owing to the complex color information and random texture features on the wafer surface, the feature design is extremely complicated. Meanwhile, a SVM optimized using a popular intelligent algorithm easily falls into a local optimum, and the convergence of the algorithm needs to be improved. Therefore, a classification method is proposed for detecting the color difference from multi-scale features in polysilicon wafer images. First, to extract the features, an image segmentation method is devised based on the maximum region contrast, which effectively applies a threshold segmentation of the wafer images. Second, the multi-scale features and color representations in different color spaces are used to construct a nine-dimensional feature vector that sufficiently describes the surface characteristics of the wafer. An approach to optimize the SVM is finally proposed using a magnetic bacteria optimization algorithm based on an elitist strategy for parameter optimization. The optimum individual of each generation is used to adjust the magnetic moment such that the solution approaches the optimal direction and enhances the global search ability. A fitness function is also introduced to improve the diversity of the solutions through a cross-validation method. The experiment results show that the proposed algorithm achieves an accuracy of 98.3% with a better classification performance than the other methods and that the color difference of polysilicon wafers can be effectively detected.

In the present work, we propose the multi-scale Local Difference Features (LDF) for improving handwritten signature verification. This descriptor is similar to local binary patterns, since it highlights intensity differences between a pixel and its neighbors. Presently, we propose a multi-scale implementation of LDF in which, neighboring pixels are taken on several radiuses. This helps to adapt LDF to variations of handwritten strokes within signatures. The verification stage is carried out by using Support Vector Machines classifier. The performance assessment is elaborated on two benchmark datasets. Obtained error rates highlight the effectiveness of the proposed features, since they outperform several state of the art results.

Research on binocular stereo vision phenotyping measurement for leafy vegetable based on 3DGS supervision

Image enhancement plays a crucial role in computer vision by improving visual quality while minimizing distortion. Traditional methods enhance images through pixel value transformations, yet they often introduce new distortions. Recent advancements in deep learning-based techniques promise better results but challenge the preservation of image fidelity. Therefore, it is essential to evaluate the visual quality of enhanced images. However, existing quality assessment methods frequently encounter difficulties due to the unique distortions introduced by these enhancements, thereby restricting their effectiveness. To address these challenges, this paper proposes a novel blind image quality assessment (BIQA) method for enhanced natural images, termed multi-scale local feature fusion and global feature representation-based quality assessment (MLGQA). This model integrates three key components: a multi-scale Feature Attention Mechanism (FAM) for local feature extraction, a Local Feature Fusion (LFF) module for cross-scale feature synthesis, and a Global Feature Representation (GFR) module using Vision Transformers to capture global perceptual attributes. This synergistic framework effectively captures both fine-grained local distortions and broader global features that collectively define the visual quality of enhanced images. Furthermore, in the absence of a dedicated benchmark for enhanced natural images, we design the Natural Image Enhancement Database (NIED), a large-scale dataset consisting of 8,581 original images and 102,972 enhanced natural images generated through a wide array of traditional and deep learning-based enhancement techniques. Extensive experiments on NIED demonstrate that the proposed MLGQA model significantly outperforms current state-of-the-art BIQA methods in terms of both prediction accuracy and robustness.