Multiple depth-levels features fusion enhanced network for action recognition

In this paper, we propose a framework which fuses multiple features for action recognition in depth sequence. The fusion of multiple features is important for recognizing action since a single feature-based representation is inadequate to capture the variants. Hence, we use two types of features: i) a quantized vocabulary of local spatio-temporal descriptor HOG3D, and ii) a global projection based descriptor that computes the HOG from the Depth Motion Maps. To optimally combine these features, we input those features to different classifiers, where SVM is applied to estimate the probabilities of action labels. Then, we weight those probabilities respectively and sum it to find the maximum score of action labels. The proposed approach is tested on publicly available MSR Action3D dataset which demonstrates that fusion of multiple features help to achieve improved performance significantly, outperforming Li et al.[1] in most of the cases.

Multiple color representation and fusion for diabetes mellitus diagnosis based on back tongue images

Multiple features data fusion method in color texture analysis

Robust and efficient contactless human-machine interaction is critical for integrated sensor systems in clinical settings, demanding low-power solutions adaptable to edge computing platforms. This paper presents a real-time hand gesture recognition system using a low-power Frequency-Modulated Continuous Wave (FMCW) radar sensor, featuring a novel Multiple Feature Fusion (MFF) framework optimized for deployment on edge devices. The proposed system integrates velocity profiles, angular variations, and spatial-temporal features through a dual-stage processing architecture: an adaptive energy thresholding detector segments gestures, followed by an attention-enhanced neural classifier. Innovations include dynamic clutter suppression and multi-path cancellation optimized for complex clinical environments. Experimental validation demonstrates high performance, achieving 98% detection recall and 93.87% classification accuracy under LOSO cross-validation. On embedded hardware, the system processes at 28 FPS, showing higher robustness against environmental noise and lower computational overhead compared with existing methods. This low-power, edge-based solution is highly suitable for applications like sterile medical control and patient monitoring, advancing contactless interaction in healthcare by addressing efficiency and robustness challenges in radar sensing for edge computing.

This paper presents an image segmentation method which utilizes image multiple features fusion, superpixel segmentation technique and hypergraph theory. Based on the image superpixel structure, our method maps each superpixel to a vertex and the similarity between two superpixels to an edge, to form a graph structure which can be represented as an affinity matrix. Hyperedges can be generated by doing spectral clustering on this affinity graph. Similarity of superpixels can be defined by different feature, then multiple features can be used to generate the hyperedges, and finally construct a hypergraph. Our approach uses the brightness, color and edge features. Then we use normalized hypergraph cut algorithm and the K-means clustering method to obtain the final segmentation result. The method was evaluated in Berkeley Segmentation Database. Experimental results show that our method can give better segmentations for the natural images. The computational efficiency and robustness of the method was verified through a comparative analysis of some state-of-the-art segmentation algorithms.KeywordsImage segmentationHypergraphSuperpixelMultiple features fusion

Feature fusion methods have been demonstrated to be effective for many computer vision based applications. These methods generally use multiple hand-crafted features. However, in recent days, features extracted through transfer leaning procedures have been proved to be robust than the hand-crafted features in myriad applications, such as object classification and recognition. The transfer learning is a highly appreciated strategy in deep convolutional neural networks (DCNNs) due to its multifaceted benefits. It heartens us to explore the effect of fusing multiple transfer learning features of different DCNN architectures. Thus, in this work, we extract features of image statistics by exploiting three different pre-trained DCNNs through transfer learning. Then, we transform the features into a generalized subspace through a recently introduced Autoencoder network and fuse them to form intra-class invariant feature vector that is used to train a multi-class Support Vector Machine (SVM). The experimental results on various datasets, including object and action image statistics show that the fusion of multiple transfer learning features improves classification accuracy as compared to fusion of multiple hand-crafted features and usage of single component transfer learning features.

The diversity of multimedia data in the real world usually forms heterogeneous types of feature sets. How to explore the structure information and the relationships among multiple features is still an open problem. In this paper, we propose an unsupervised subspace learning method, named the shared low-rank correlation embedding (SLRCE) for multiple feature fusion. First, in the learned subspace, we implement the low-rank representation on each feature set and enforce a shared low-rank constraint to uncover the common structure information of multiple features. Second, we develop an enhanced correlation analysis in the learned subspace for simultaneously removing the redundancy of each feature set and exploring the correlation of multiple features. Finally, we incorporate the shared low-rank representation and the correlation analysis into a unified framework. The shared low-rank constraint not only depicts the data distribution consistency among multiple features, but also assists robust subspace learning. Our method is robust to noise in practice and can be extended to the kernel case to handle the nonlinear feature fusion. Experimental results on several typical datasets demonstrate the superior performance of the proposed methods.

This paper presents a robust face recognition algorithm by using transform domain-based multiple feature fusion and linear regression. Transform domain-based feature fusion can provide comprehensive face information for recognition, and decrease the effect of variations in illumination and pose. The holistic feature and local feature are extracted by discrete cosine transform and Gabor wavelet transform, respectively. Then the extracted holistic features and the local features are fused by weighted sum. The fused feature values are finally sent to linear regression classifier for recognition. The algorithm is evaluated on AR, ORL and Yale B face databases. Experiment results show that our proposed algorithm could be more robust than those single feature-based algorithms under pose and expression variations.

A palmprint image contains abundant features, such as the ridges, lines and textures, etc. Different features reflect the different characteristic of the palmprint. Fusion of multiple palmprint features may enhance the performance of a palmprint authentication system. In this paper, we investigate the fusion of two novel palmprint representations: orientationCode and diffCode. OrientationCode are defined using four directional templates and diffCode is computed using differential operation. Then these two codes are fused to measure the similarity of palmprints. The experimental results show that the proposed approach can get a very high accuracy.

Automated electrocardiogram (ECG) analysis for arrhythmia detection plays a critical role in early prevention and diagnosis of cardiovascular diseases. In this paper, we proposed a novel end-to-end deep learning method for multiclass arrhythmia detection with multiple stage features fusion. The network is composed of multiple convolution and attention module. Specifically, we use skip connection operation to fuse different levels of features extracted at different stages for target task processing. And the channel-wise attention modules are adopted for effectively extracting the features learned at the different stages. By combining the attention module and convolutional neural network, the discrimination power of the network for ECG classification is improved. We demonstrate the proposed method for ECG classification on an open ECG dataset and compare it with some state-of-the-art methods, which achieves an average F1-score of 81.3% in classification of 8 types of arrhythmias and sinus rhythm. The experimental results convince the efficiency of the proposed method.

In multi-target tracking, Multiple Hypothesis Tracking (MHT) can effectively solve the data association problem. However, traditional MHT can not make full use of motion information. In this work, we combine MHT with Interactive Multiple Model (IMM) estimator and feature fusion. New algorithm greatly improves the tracking performance due to the fact that IMM estimator provides better estimation and feature information enhances the accuracy of data association. The new algorithm is tested by tracking tropical fish in fish container. Experimental result shows that this algorithm can significantly reduce tracking lost rate and restrain the noises with higher computational effectiveness when compares with traditional MHT.

Eye blink artifact detection based on multi-dimensional EEG feature fusion and optimization

Electroencephalography (EEG) motor imagery signals have recently gained significant attention due to its ability to encode a person’s intent to perform an action. Researchers have used motor imagery signals to help disabled persons control devices, such as wheelchairs and even autonomous vehicles. Hence, the accurate decoding of these signals is important to brain–computer interface (BCI) systems. Such motor imagery-based BCI systems can become an integral part of cognitive modules that are increasingly being used in smart city frameworks. However, the classification and recognition of EEG have consistently been a challenge due to its dynamic time series data and low signal-to-noise ratio. Deep learning methods, such as the convolution neural network (CNN), have achieved remarkable success in computer vision tasks. Considering the limited applications of deep learning for motor imagery EEG classification, this work focuses on developing CNN-based deep learning methods for such purpose. We propose a multiple-CNN feature fusion architecture to extract and fuse features by using subject-specific frequency bands. CNN has been designed with variable filter sizes and split convolutions for the extraction of spatial and temporal information from raw EEG data. A feature fusion technique based on autoencoders is applied. Cross-encoding technique has been proposed and is successfully used to train autoencoders for a novel cross-subject information transfer and augmenting EEG data. This proposed method outperforms the state-of-the-art four-class motor imagery classification methods for subject-specific and cross-subject data. Autoencoder cross-encoding helps to learn subject invariant and generic features for EEG data and achieves more than 10% increase on cross-subject classification results. The fusion approaches show the potential of applying multiple CNN feature fusion techniques for the advancement of EEG-related research.

Fusion of multiple features can boost the performance of large-scale visual classification and detection tasks like TRECVID Multimedia Event Detection (MED) competition [1]. In this paper, we propose a novel feature fusion approach, namely Feature Weighting via Optimal Thresholding (FWOT) to effectively fuse various features. FWOT learns the weights, thresholding and smoothing parameters in a joint framework to combine the decision values obtained from all the individual features and the early fusion. To the best of our knowledge, this is the first work to consider the weight and threshold factors of fusion problem simultaneously. Compared to state-of-the-art fusion algorithms, our approach achieves promising improvements on HMDB [8] action recognition dataset and CCV [5] video classification dataset. In addition, experiments on two TRECVID MED 2011 collections show that our approach outperforms the state-of-the-art fusion methods for complex event detection.

This paper presents a multiple feature fusion method using topic model for social image visualization. Images in social media are represented from several aspects such as their visual information and tags. The proposed method extracts low-level features from social images and their tags and calculates their integrated high-level features. Specifically, the proposed method applies multilayer multimodal probabilistic Latent Semantic Analysis (mm-pLSA) to the low-level visual and tag features to obtain the high-level features. Then, by applying dimensionality reduction techniques to the obtained features, successful visualization becomes feasible.