Extension of Sample Dimension and Sparse Representation Based Classification of Low-Dimension Data

Neural networks are being increasingly used in many applications. While large deep neural networks are quickly advancing, for computing devices that lack powerful processors, it is desirable to use smaller neural networks. In this paper, we focus on smaller neural networks than common deep networks and examine how their size can be made even smaller without sacrificing the performance. We show that for some data types it is possible to make the networks smaller using sparse representation of the input. We have studied time delay neural networks (TDNN) for time series prediction using three different datasets. It is found that sparsifying input to the TDNN, using Discrete Cosine Transform (DCT), or Principal Component Analysis (PCA), can result in performance improvement. The improved performance can be traded off for network size reduction. Therefore, we can make the network smaller and maintain the same performance. It is found that for data that has more randomness and sudden changes in value (higher frequencies present), sparse representation methods using discrete cosine transform or through principal component analysis allow reducing network size by up to 40%.

Nowadays, sparse representation has been widely used in Magnetic Resonance Imaging (MRI). The commonly used sparse representation methods are based on symmetrical partition, which have not considered the complex structure of MRI image. In this paper, we proposed a sparse representation method for the brain MRI image, called GNAMlet transform, which is based on the gradient information and the non-symmetry and anti-packing model. The proposed sparse representation method can reduce the lost detail information, improving the reconstruction accuracy. The experiment results show the superiority of the proposed transform for the brain MRI image representation in comparison with some state-of-the-art sparse representation methods.

The sparse representation based classification (SRC) can effectively improve the face recognition rate. Smoothed l 0 algorithm has much faster calculation speed and requires fewer measured values than the other sparse representation method. In this paper, the sparse representation and smoothed l 0 algorithm for face recognition are presented to improve the face recognition under various conditions such as face disguise, illumination and pose changes, etc. The experiments on the AR, Extended Yale B and FERET face database verify the effectiveness of the presented method. The experimental results show that the face recognition algorithm increases to a certain extent in terms of recognition robustness and time than the original SRC.

Existing sparse representation methods either fail to incorporate the structure and label information of training samples, or suffer from expensive computation for $$l_1$$- or $$l_{2,1}$$-norm. In this paper, we propose three discriminative sparse representation classification methods with structure and label information based on $$l_2$$-norm regularization for robust face recognition. We propose the first classification method with structure and label information by enforcing competition among the representation results of training samples from different classes in representing a test sample. To make the classification more discriminative, we present the decorrelation classification method with structure and label information by jointly considering the competition and decorrelation regularizations. In addition, by incorporating the locality information of samples, we propose the third method called locality-constrained decorrelation classification method with structure and label information. The proposed methods not only contain the structure and label information of training samples, but also have low computational cost owing to the use of $$l_2$$-norm. All three methods have closed-form solutions, rendering them easy to solve and calculate efficiently. Importantly, the proposed methods can achieve better recognition results than most existing state-of-the-art sparse representation methods. Furthermore, based on the proposed methods, we illustrate the effect of different regularization constraints on the recognition performance. Experiments on the ORL, Extended YaleB, FERET, and LFW databases validate the effectiveness of the proposed methods.

With the rapid development of surveillance technology, there are often several cameras in one scenario. The multi-camera usage to perform gait recognition becomes a challenge problem. This paper studies multi-camera gait recognition via structure sparsity. For the multi-camera structure in the training set, we propose a structure sparsity algorithm to learn informative and discriminative sparse representations; and for the structure in the testing set, we develop a new classification criteria based on the reconstruction error of learned sparse representations. In addition, we learn a dictionary from the original gait data to further improve recognition accuracy meanwhile reduce computational cost. Experimental results show that the proposed method can efficiently deal with the multi-camera gait recognition problem and outperforms the state-of-the-art sparse representation methods.

Feature ranking is an interesting problem in data science due to the time and effort wasted on collecting, storing, and processing redundant features. This may also lead to over-fitted and under-trained machine learning (ML) and deep learning models. Although there are several feature ranking algorithms available, they lack an intuitive interpretation of the effect on the final ML model behavior. In this paper, we propose simple and intuitive feature ranking metrics based on sparse representation methods to be used in classification tasks. Sparse representation is an emerging image processing tool that can be effectively used in satellite/airborne image scene classification tasks. This paper applies two sparse representation methods, LCKSVD and Frozen Dictionary Learning, on handcrafted features taken from the Sat-4 and Sat-6 datasets as a preliminary test. Even though these methods report lower classification accuracies than state-of-art deep learning methods, they provide an intuitive understanding of the system model. Furthermore, sparse representation allows for the use of simpler linear classifiers in the classification stage to achieve relatively high performance. Finally, we present an analysis of the relationship between the learned sparse coefficients and the original feature space to explain the intuitive behavior of this model.

Based on multiple maximum scatter difference discrimination Dictionary learning, a novel face recognition algorithm is proposed. Dictionary used for sparse coding plays a key role in sparse representation classification. In this paper, a multiple maximum scatter difference discriminated criterion is used for dictionary learning. During the process of dictionary learning, the multiple maximum scatter difference computes its discriminated vectors from both the range of the between class scatter matrix and the null space of the within-class scatter matrix. The proposed algorithm is theoretically elegant and easy to calculate. Extensive experimental studies conducted on the AR database and Extended Yale Database B in comparison with existing basic sparse representation and other classification methods, it shows that the performance is a little better than the original sparse representation methods with lower complexity.

A gear’s vibration signal consists of multiple components, so it is therefore difficult to accurately extract the transient components of gear faults. Currently, sparse representation is capable of separating fault components from multicomponent noisy vibration signals. However, sparse representation methods still suffer problems with poor computational efficiency and the underestimation of amplitude. To tackle these challenges, this paper proposes a nonconvex regularized sparse representation in a tight frame. The tunable Q-factor wavelet transform (TQWT) is proposed as a sparse dictionary, which can portray the waveform characteristics of the gear’s vibration signal. TQWT satisfies the tight-frame condition, hence it can efficiently reduce the amount of calculations. The minimax concave function is used as the penalty function since it stands out from various penalty functions with the ability to maintain amplitude. The simulation and experimental analysis show that this method has a shorter operation time and a better ability to maintain the amplitude.

Sparse representation based on stacked kernel for target detection in hyperspectral imagery

Sparse coding based visual tracking: Review and experimental comparison

We consider the classification problem as a linear regression problem, and find that sparse signal representation offers the key to address this problem. Therefore, a new method, which is based on sparse representation, is proposed for classification. This new method provides insights into two critical issues in classification: sparse representation and classification. For sparse representation, we use the lasso, the elastic net and nonnegative garrote as the initial estimate of a new test sample. In the classification stage, we classify the test sample to the correct class via a simple l <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -distance measurement. Finally, we propose an efficient algorithm for computing the whole solution path of this method, and conduct extensive experiments on the number ball recognition. From the experiment results, we conclude that this method achieves high recognition rate.

Transient analysis of large-scale power delivery network (PDN) is a critical task to ensure the functional correctness and desired performance of today’s integrated circuits (ICs), especially if significant transient noises are induced by clock and/or power gating due to the utilization of extensive power management. In this paper, we propose an efficient algorithm for PDN transient analysis based on sparse approximation. The key idea is to exploit the fact that the transient response caused by clock/power gating is often localized and the voltages at many other “inactive” nodes are almost unchanged, thereby rendering a unique sparse structure. By taking advantage of the underlying sparsity of the solution structure, a modified conjugate gradient algorithm is developed and tuned to efficiently solve the PDN analysis problem with low computational cost. Our numerical experiments based on standard benchmarks demonstrate that the proposed transient analysis with sparse approximation offers up to $2.2\times $ runtime speedup over other traditional methods, while simultaneously achieving similar accuracy.

Image fusion in remote sensing is an issue to fuse the texture information of panchromatic (PAN) channel and the spectral information of multispectral (MS) channels with lower spatial resolution (LR). In this paper, a method named SPCA is proposed to deal with image fusion from the perspective of sparse representation and PCA, in which the correlations both within and between channels are effectively modeled. First, the sparse representation theory is applied to remote sensing images. Second, the dictionaries of PAN and MS images are joint-learned, and a thought of PCA is applied to construct dictionaries of MS images of high spatial resolution (HR). Then the fusion images can be calculated with constructed dictionaries and sharing coefficient. Finally, the residual produced by sparse representation is interpolated as compensation. Compared with four methods in four evaluation indexes, SPCA method gives competitive or even better results on LandSat8 and QuickBird.

In this study, we investigate various locations of sensor positions to detect stereotypical self-stimulatory behavioral patterns of children with Autism Spectrum Disorder (ASD). The study is focused on finding optimal detection performance based on sensor location and number of sensors. To perform this study, we developed a wearable sensor system that uses a 3 axis accelerometer. A microphone was used to understand the surrounding environment and video provided ground truth for analysis. The recordings were done on 2 children diagnosed with ASD who showed repeated self-stimulatory behaviors that involve part of the body such as flapping arms, body rocking and vocalization of non-word sounds. We used time-frequency methods to extract features and sparse signal representation methods to design over-complete dictionary for data analysis, detection and classification of these ASD behavioral events. We show that using single sensor on the back achieves 95.5% classification rate for rocking and 80.5% for flapping. In contrast, flapping events can be recognized with 86.5% accuracy using wrist worn sensors.

In this paper, a compact-dictionary-based sparse representation (CDSR) method is proposed for hyperspectral image (HSI) classification. The proposed dictionary in CDSR is dynamically generated according to the spatial and spectral context of each pixel. It can effectively shrink the decision range for classification, and reduce the computational burden since the compact dictionary is composed of the classes correlated with the target pixel in terms of spatial location and spectral information. In order to obtain better spatial context information, a spatial location expanding strategy is designed for spreading local explicit label information to a wider region. Experimental results demonstrate the effectiveness and superiority of the proposed method when compared with some widely used HSI classification approaches.