Semi-supervised process fault classification based on convolutional ladder network with local and global feature fusion

In order to retrieve the image quickly and accurately in the massive image library, this paper proposes a fast retrieval method for massive face images based on global feature and local feature. Firstly, we use the local binary model feature (LBP) to extract the local face features, such as the tip of the nose, the mouth, and eye pupil etc. Then, the image global features are extracted and are integrated with the local features as our retrieval features. The principal component analysis (PCA) is used to reduce the dimensions of the features to 64, and the reduced dimension is encoded to generate an image signature, whose inverted index table is constructed for the image library and used for efficient retrieval. By testing on the 110,000 experimental datasets, the method can accurately retrieve the desired image within 0.3s using single-thread program.

Image enhancement plays a crucial role in computer vision by improving visual quality while minimizing distortion. Traditional methods enhance images through pixel value transformations, yet they often introduce new distortions. Recent advancements in deep learning-based techniques promise better results but challenge the preservation of image fidelity. Therefore, it is essential to evaluate the visual quality of enhanced images. However, existing quality assessment methods frequently encounter difficulties due to the unique distortions introduced by these enhancements, thereby restricting their effectiveness. To address these challenges, this paper proposes a novel blind image quality assessment (BIQA) method for enhanced natural images, termed multi-scale local feature fusion and global feature representation-based quality assessment (MLGQA). This model integrates three key components: a multi-scale Feature Attention Mechanism (FAM) for local feature extraction, a Local Feature Fusion (LFF) module for cross-scale feature synthesis, and a Global Feature Representation (GFR) module using Vision Transformers to capture global perceptual attributes. This synergistic framework effectively captures both fine-grained local distortions and broader global features that collectively define the visual quality of enhanced images. Furthermore, in the absence of a dedicated benchmark for enhanced natural images, we design the Natural Image Enhancement Database (NIED), a large-scale dataset consisting of 8,581 original images and 102,972 enhanced natural images generated through a wide array of traditional and deep learning-based enhancement techniques. Extensive experiments on NIED demonstrate that the proposed MLGQA model significantly outperforms current state-of-the-art BIQA methods in terms of both prediction accuracy and robustness.

The fully convolutional networks (FCNs) have been widely applied in numerous medical image segmentation tasks. However, tissue regions usually have large variations of shape and scale, so the ability of neural networks to learn multi-scale features is important to the segmentation performance. In this paper, we improve the network for multi-scale feature fusion, in the medical image segmentation by introducing two feature fusion modules: i) global attention multi-scale feature fusion module (GMF); ii) local dense multi-scale feature fusion module (LMF). GMF aims to use global context information to guide the recalibration of low-level features from both spatial and channel aspects, so as to enhance the utilization of effective multi-scale features and suppress the noise of low-level features. LMF adopts bottom-up top-down structure to capture context information, to generate semantic features, and to fuse feature information at different scales. LMF can integrate local dense multi-scale context features layer by layer in the network, thus improving the ability of network to encode interdependent relationships among boundary pixels. Based on the above two modules, we propose a novel medical image segmentation framework (GLF-Net). We evaluated the proposed network and modules on challenging brain tumor segmentation and pancreas segmentation datasets, and very competitive performance has been achieved.

Human perception of the face involves the observation of both coarse (global) and detailed (local) features of the face to identify and categorize a person. Face categorization involves finding common visual cues, such as gender, race and age, which could be used as a precursor to a face recognition system to improve recognition rates. In this paper, we investigate the fusion of both global and local features for gender classification. Global features are obtained using the principal component analysis (PCA) and discrete cosine transformation (DCT) approaches. A spatial local binary pattern (LBP) approach augmented with a two-dimensional DCT approach has been used to find the local features. The performance of the proposed approach has been investigated through extensive experiments performed on FERET database. The proposed approach gives a recognition accuracy of 98.16% on FERET database. Comparisons with some of the existing techniques have shown a marked reduction in number of features used per image to produce results more efficiently and without loss of accuracy for gender classification.

Accurate and interpretable AI models play a critical role in medical image analysis. However, despite advancements in explainable AI (XAI), existing methods struggle with inconsistent interpretability. To overcome this limitation, we introduce the Integrated Latent and Attention Mapping (ILAM) framework, which enhances both classification accuracy and explainability by fusing local and global feature representations. ILAM integrates a custom-designed Autoencoder (AE) with a Vision Transformer (ViT), where the AE learns fine-grained local features through unsupervised patchwise image reconstruction in the latent space. These local features are then fused with global representations extracted by ViT, creating a hybrid model that improves both performance and post hoc interpretability. To refine explainability, ILAM incorporates a modified attention rollout mechanism, which recursively aggregates latent feature representations and attention weights to produce precise and stable activation maps. We evaluate ILAM on three publicly available medical imaging datasets-BreakHis, Chest X-Ray, and Retinal, demonstrating its superior performance over transformer-based models such as ViT, DeiT, CvT, and SwinT. ILAM consistently generates detailed and reliable activation maps, providing clearer visualizations of critical image regions influencing model decisions. By effectively combining local and global feature fusion, ILAM establishes itself as a robust and interpretable framework for medical image classification.

Human activity recognition is an important and challenging topic for computer vision research community. Action representation using deep learning models are currently the dominant technique compared with other methods. However, supervised convolutional neural networks require large computational and memory resources to optimize their parameters. Recently, a simple unsupervised deep learning architecture Principal Component Analysis Network (PCANet) has emerged as an alternative of Convolutional Neural Networks (CNNs) and has significant accomplishments in various vision applications. Meanwhile, encoding and representation techniques using Bag of Words (BoW) and Vector of Locally Aggregated Descriptors (VLAD) have demonstrated great success for several visual tasks specifically in activity recognition. This work presents a novel human activity recognition technique by combining global and local features of PCANet with BoW and VLAD encoding schemes. Both global and local features are learned by PCANet utilizing selected frames from each action video. After that the dimensionality of these features is decreased via Whitening PCA (WPCA). Then encoding schemes are applied on both features to represent the final descriptors for each action. Ultimately, Support Vector Machines classifier (SVM) is trained for recognition process. Several experiments are conducted on UCF sports dataset to evaluate our method. All experimental results utilizing leave-one-out-cross validation (LOOCV) strategy are satisfactorv and comparable.

A wavelet-based local and global feature fusion network (LAGN) is proposed for low-light image enhancement, aiming to enhance image details and restore colors in dark areas. This study focuses on addressing three key issues in low-light image enhancement: Enhancing low-light images using LAGN to preserve image details and colors; extracting image edge information via wavelet transform to enhance image details; and extracting local and global features of images through convolutional neural networks and Transformer to improve image contrast. Comparisons with state-of-the-art methods on two datasets verify that LAGN achieves the best performance in terms of details, brightness, and contrast.

Palmprint is the region between wrist and fingers. In this paper, a palmprint personal identification system is proposed based on the local and global information fusion. The local and global information is critical for the image observation based on the results of the relationship between physical stimuli and perceptions. The local features of the enhanced palmprint are extracted using discrete orthonormal Stockwell transform. The global feature is obtained by reducing the scale of discrete orthonormal Stockwell transform to infinity. The local and global matching distances of the two palmprint images are fused to get the final matching distance of the proposed scheme. The results show that the fusion of local and global features outperforms the existing works on the available three datasets.

This paper proposes an effective method to improve the spatial resolution of FengYun-2 (FY-2) infrared cloud images via deep convolutional neural networks. The proposed model consists of four parts: shallow feature representation block, stacked multi-scale fusion blocks, global feature fusion block, and feature reconstruction block. The multi-scale fusion block combines dilated convolution, local feature fusion and local residual learning to extract multi-scale local features from the original low-resolution image directly. Then these local features are all merged by the global feature fusion block to reconstruct the residual representations in high-resolution space. For training and testing, we have specially built a dataset of infrared cloud images. We evaluated the proposed method both on simulated and real data. Experimental results demonstrate that the proposed approach achieves improved reconstruction accuracy than the state-of-the-art methods. Besides, the concise structure of the proposed model enables it to be applicable in practice.

The appearance of pedestrians can vary greatly from image to image, and different pedestrians may look similar in a given image. Such similarities and variabilities in the appearance and clothing of individuals make the task of pedestrian re-identification very challenging. Here, a pedestrian re-identification method based on the fusion of local features and gait energy image (GEI) features is proposed. In this method, the human body is divided into four regions according to joint points. The color and texture of each region of the human body are extracted as local features, and GEI features of the pedestrian gait are also obtained. These features are then fused with the local and GEI features of the person. Independent distance measure learning using the cross-view quadratic discriminant analysis (XQDA) method is used to obtain the similarity of the metric function of the image pairs, and the final similarity is acquired by weight matching. Evaluation of experimental results by cumulative matching characteristic (CMC) curves reveals that, after fusion of local and GEI features, the pedestrian reidentification effect is improved compared with existing methods and is notably better than the recognition rate of pedestrian re-identification with a single feature.

MFFNet: Single facial depth map refinement using multi-level feature fusion

Human beings can easily estimate the age or age group of a person from a facial image where as this capability is not prominent in machines. This problem becomes more complex due to presence of different facial expressions and age progression. In this paper, we introduced a novel method for age prediction using combination of local and global features. After detecting the face from image, we partition the facial image in 16 * 16 non-overlapping blocks and apply grey-level co-occurrence matrix (GLCM) on these blocks. After calculating four facial parts (eyes, forehead, left cheek and right cheek) from facial image, features from second local feature Gabor filter are obtained. Histogram of oriented gradients has been used as a global feature for feature extraction from complete face image. Experimental results show that fusion of local and global features perform better than existing approaches and reported 6.31 years mean absolute error (MAE) on PAL dataset.

The amount of multimedia data, such as images and videos, has been increasing rapidly with the development of various imaging devices and the Internet, bringing more stress and challenges to information storage and transmission. The redundancy in images can be reduced to decrease data size via lossy compression, such as the most widely used standard Joint Photographic Experts Group (JPEG). However, the decompressed images generally suffer from various artifacts (e.g., blocking, banding, ringing, and blurring) due to the loss of information, especially at high compression ratios. This article presents a feature-enriched deep convolutional neural network for compression artifacts reduction (FeCarNet, for short). Taking the dense network as the backbone, FeCarNet enriches features to gain valuable information via introducing multi-scale dilated convolutions, along with the efficient 1 ×1 convolution for lowering both parameter complexity and computation cost. Meanwhile, to make full use of different levels of features in FeCarNet, a fusion block that consists of attention-based channel recalibration and dimension reduction is developed for local and global feature fusion. Furthermore, short and long residual connections both in the feature and pixel domains are combined to build a multi-level residual structure, thereby benefiting the network training and performance. In addition, aiming at reducing computation complexity further, pixel-shuffle-based image downsampling and upsampling layers are, respectively, arranged at the head and tail of the FeCarNet, which also enlarges the receptive field of the whole network. Experimental results show the superiority of FeCarNet over state-of-the-art compression artifacts reduction approaches in terms of both restoration capacity and model complexity. The applications of FeCarNet on several computer vision tasks, including image deblurring, edge detection, image segmentation, and object detection, demonstrate the effectiveness of FeCarNet further.

Attention-guided image captioning with adaptive global and local feature fusion

In the face recognition, training data is sometimes limited, and even a single face photo per person, which causes the model less trained to recognize the multi-pose facial image with low recognition rate. In this paper, aiming at the face recognition using small sample training set, a joint Bayesian face recognition algorithm based on adaptive face data enhancement combined with global and local feature fusion is proposed. The face key points are detected and then expanded and rotated to extend the face data samples, and face model is established by combining the global and local features. The experiment result shows that the proposed algorithm makes the multi-gesture face recognition performance well.