Group sparsity based sparse coding for region covariances

Sparse representation-based classifiers have shown outstanding accuracy and robustness in image classification tasks even with the presence of intense noise and occlusion. However, it has been discovered that the performance degrades significantly either when test image is not aligned with the dictionary atoms or the dictionary atoms themselves are not aligned with each other, in which cases the sparse linear representation assumption fails. In this paper, having both training and test images misaligned, we introduce a novel sparse coding framework that is able to efficiently adapt the dictionary atoms to the test image via large displacement optical flow. In the proposed algorithm, every dictionary atom is automatically aligned with the input image and the sparse code is then recovered using the adapted dictionary atoms. A corresponding supervised dictionary learning algorithm is also developed for the proposed framework. Experimental results on digit datasets recognition verify the efficacy and robustness of the proposed algorithm.

The paper concerns automatic recognition of emotion induced by music (MER, Music Emotion Recognition).Comparison of different sparse coding schemes in a task of MER is the main contribution of the paper.We consider a domainspecific categorization of emotions, called Geneva Emotional Music Scale (GEMS), which focuses on induced emotions rather than expressed emotions.We were able to find only one dataset, namely Emotify, in which data are annotated with GEMS categories, this set was used in our experiments.Our main goal was to compare different sparse coding approaches in a task of learning features useful for predicting musically induced emotions, taking into account categories present in the GEMS.We compared five sparse coding methods and concluded that sparse autoencoders outperform other approaches.

Learning universal multiview dictionary for human action recognition

Sea or ground clutter is one of the main bottleneck factors for target detection in skywave over-the-horizon radar (OTHR). Conventional methods can partly suppress clutter, however, they typically suffer from two shortcomings. First, weak targets beside or within the clutter tend to be submerged, which increases the probability of miss-detection. Second, a large amount of isolated clutter may be residual, which increases the probability of false alarm (PFA). In this article, a clutter suppression method based on joint low rank and group sparse representation (JLRGSR) is proposed to address these issues. The properties of OTHR echo signals are firstly analyzed, including the low rank of clutter in the range-slow time domain, the column sparsity of clutter in the range-Doppler (RD) domain, and the group sparsity of targets in the RD domain, based on which the objective function is constructed. It is observed that clutter is primarily characterized by the dual constraints of low rank and column sparsity. Furthermore, the group sparsity constraint helps maintain the target morphology's integrity and reduces isolated clutter residuals. The objective function is solved through the alternating direction method of multipliers to realize clutter suppression. Finally, the effectiveness and superiority of the JLRGSR, that is, higher probability of detection and lower probability of false alarm, are experimentally verified by simulation data and real data.

Effective representation methods and proper signal priors are crucial in most signal processing applications. In this thesis we focus on different structured models and we design appropriate schemes that allow the discovery of low dimensional latent structures that characterise and identify the signals. Motivated by the highly non-linear structure of most datasets, we firstly investigate the geometry of manifolds. Manifolds are low dimensional, non-linear structures that are naturally employed to describe sets of strongly related signals such as the images of a 3-D object captured from different viewpoints. However, the use of manifolds in applications is not straightforward due to their usually non-analytic and non-linear form. We propose here a way to `disassemble' a manifold into simpler components by approximating it with affine subspaces. Our objective is to discover a set of low dimensional affine subspaces that can represent manifold data accurately while preserving the manifold's structure. To this end, we employ a greedy technique that iteratively merges manifold samples into groups based on the difference of local tangents. We use our algorithm to approximate synthetic and real manifolds and to demonstrate that it is competitive to state-of-the-art techniques. Then, we consider structured sparse representations of signals and we propose a new sparsity model, where signals are essentially composed of a small number of structured {\it molecules }. We define the molecules to be linear combinations of a small number of atoms in a redundant dictionary. Our multi-level model takes into account the energy distribution of the significant signal components in addition to their support. It permits to define typical visual patterns and recognise them in prototypical or deformed form. We define a new structural difference measure between molecules and their deformed versions, which is based on their sparse codes and we create an algorithm for decomposing signals into molecules that can account for different deviations in the internal molecule structure. Our experiments verify the benefits of the new image model in various restoration tasks and they confirm that the development of proper models that extend the mere notion of sparsity can be very useful for various inverse problems in imaging. Finally, we investigate the problem of learning molecule representations directly in the sparse code domain. We constrain sparse codes to be linear combinations of a few, possibly deformed, molecules and we design an algorithm that can learn the structure from the codes without transforming them back into the signal domain. To this end, we take advantage of our structural difference which is based on the sparse codes and we devise a scheme for representing the codes with molecules and learn the molecules at the same time. To illustrate the effectiveness of our proposed algorithm we apply it to various synthetic and real datasets and we compare the results with traditional sparse coding and dictionary learning techniques. From the experiments, we verify the superior performance of our scheme in interpreting and recognising correctly the underlying structure. In short, in this thesis we are interested in low-dimensional, structured models. Among the various choices, we focus on manifolds and sparse representations and we propose schemes that enhance their structural properties and highlight their effectiveness in signal representations.

Various sparse regularizers have been applied to machine learning problems, among which structured sparsity has been proposed for a better adaption to structured data. In this paper, motivated by effectively classifying linked data (e.g. Web pages, tweets, articles with references, and biological network data) where a group structure exists over the whole dataset and links exist between specific samples, we propose a joint sparse representation model that combines group sparsity and graph sparsity, to select a small number of connected components from the graph of linked samples, meanwhile promoting the sparsity of edges that link samples from different groups in each connected component. Consequently, linked samples are selected from a few sparsely-connected groups. Both theoretical analysis and experimental results on four benchmark datasets show that the joint sparsity model outperforms traditional group sparsity model and graph sparsity model, as well as the latest group-graph sparsity model.

A hierarchical latent topic model based on sparse coding

Sparse coding allows the representation of complex data as a linear combination of basis sparse vectors (alternatively called atoms or codewords), a collection of which constitutes a dictionary. Dictionary learning is a learning process aimed at finding a small number of optimal basis vectors for a more accurate representation of the original data. The existing dictionary learning methods do not address the inherent uncertainty of the input data in their learning processes. To compensate for the uncertainty, and to obtain a flexible and effective learning system, we introduce a new adaptive fuzzy dictionary learning (AFDL) method for image classification purposes. The new method iteratively alternates between sparse coding based on a given dictionary and an adaptive fuzzy dictionary learning approach to learn (improve) dictionary atoms. The adjustability of the dictionary and coefficients vectors, in this method, provide us a more accurate and straight representation of input data. AFDL was applied on magnetic resonance images from the cancer image archive datasets, for medical image classification of cancer tumors. Finally, the overall experimental results clearly show that our approach outperforms its rival techniques in terms of accuracy, sensitivity, and specificity. Convergence speed in the experimental results shows that AFDL can achieve its acceptable precision in a reasonable time.

Morphed face images are maliciously used by criminals to circumvent the official process for receiving a passport where a look-alike accomplice embarks on requesting a passport. Morphed images are either synthesized by alpha-blending or generative networks such as Generative Adversarial Networks (GAN). Detecting morphed images is one of the fundamental problems associated with border control scenarios. Deep Neural Networks (DNN) have emerged as a promising solution for a myriad of applications such as face recognition, face verification, fake image detection, and so forth. The Biometrics communities have leveraged DNN to tackle fundamental problems such as morphed face detection. In this dissertation, we delve into data-driven morph detection which is of great significance in terms of national security. We propose several wavelet-based face morph detection schemes which employ some of the computer vision algorithms such as image wavelet analysis, group sparsity, feature selection, and the visual attention mechanisms. Wavelet decomposition enables us to leverage the fine-grained frequency content of an image to boost localizing manipulated areas in an image. Our methodologies are as follows: (1) entropy-based single morph detection, (2) entropy-based differential morph detection, (3) morph detection using group sparsity, and (4) Attention aware morph detection. In the first methodology, we harness mismatches between the entropy distribution of wavelet subbands corresponding to a pair of real and morph images to find a subset of most discriminative wavelet subbands which leads to an increase of morph detection accuracy. As the second methodology, we adopt entropy-based subband selection to tackle differential morph detection. In the third methodology, group sparsity is leveraged for subband selection. In other words, adding a group sparsity constraint to the loss function of our DNN leads to an implicit subband selection. Our fourth methodology consists of different types of visual attention mechanisms such as convolutional block attention modules and self-attention resulting in boosting morph detection accuracy. We demonstrate efficiency of our proposed algorithms through several morph datasets via extensive evaluations as well as visualization methodologies.

In this paper, we propose the convolutional sparse coding based model in the wavelet domain for the task of single image super-resolution (SISR). The conventional sparse coding based approaches work on overlapping image patches and use the dictionary atoms to sparse code an image patch. Further, at the final stage, an overlap-add mechanism is used to get the final high-resolution image estimate. However, these algorithms fail to take into account the consistency present in the overlapping patches which limits their performance. We propose the use of wavelet integrated convolutional sparse coding approach where instead of dictionary atoms we utilize the convolution summations between the learned filters and their mappings for sparse representation based SISR. The use of wavelets is proposed owing to their unique directional and compact features. A pair of filters are learned along with a mapping function for each wavelet sub-band to exploit the consistency among patches. The proposed wavelet integrated convolutional sparse coding model helps capture useful contextual information. The proposed model is evaluated on publicly available datasets for different scale-up parameters. To show the efficacy of the proposed model we compare it with recent state-of-the-art algorithms. The visual results along with the quantitative ones indicate that the proposed model performs well for the tasks of super-resolution.

Sparse representation based classification has led to interesting image recognition results, while the dictionary used for sparse coding plays a key role in it. This paper presents a novel dictionary learning (DL) method to improve the pattern classification performance. Based on the Fisher discrimination criterion, a structured dictionary, whose dictionary atoms have correspondence to the class labels, is learned so that the reconstruction error after sparse coding can be used for pattern classification. Meanwhile, the Fisher discrimination criterion is imposed on the coding coefficients so that they have small within-class scatter but big between-class scatter. A new classification scheme associated with the proposed Fisher discrimination DL (FDDL) method is then presented by using both the discriminative information in the reconstruction error and sparse coding coefficients. The proposed FDDL is extensively evaluated on benchmark image databases in comparison with existing sparse representation and DL based classification methods.

This paper seeks to combine dictionary learning and hierarchical image representation in a principled way. To make dictionary atoms capturing additional information from extended receptive fields and attain improved descriptive capacity, we present a two-pass multi-resolution cascade framework for dictionary learning and sparse coding. The cascade allows collaborative reconstructions at different resolutions using the same dimensional dictionary atoms. Our jointly learned dictionary comprises atoms that adapt to the information available at the coarsest layer where the support of atoms reaches their maximum range and the residual images where the supplementary details progressively refine the reconstruction objective. The residual at a layer is computed by the difference between the aggregated reconstructions of the previous layers and the downsampled original image at that layer. Our method generates more flexible and accurate representations using much less number of coefficients. Its computational efficiency stems from encoding at the coarsest resolution, which is minuscule, and encoding the residuals, which are relatively much sparse. Our extensive experiments on multiple datasets demonstrate that this new method is powerful in image coding, denoising, inpainting and artifact removal tasks outperforming the state-of-the-art techniques.

Image classification is an important problem in computer vision. The sparse coding spatial pyramid matching (ScSPM) framework is widely used in this field. However, the sparse coding cannot effectively handle very large training sets because of its high computational complexity, and ignoring the mutual dependence among local features results in highly variable sparse codes even for similar features. To overcome the shortcomings of previous sparse coding algorithm, we present an image classification method, which replaces the sparse dictionary with a stable dictionary learned via low computational complexity clustering, more specifically, a k-medoids cluster method optimized by k-means++. The proposed method can reduce the learning complexity and improve the feature’s stability. In the experiments, we compared the effectiveness of our method with the existing ScSPM method and its improved versions. We evaluated our approach on two diverse datasets: Caltech-101 and UIUC-Sports. The results show that our method can increase the accuracy of spatial pyramid matching, which suggests that our method is capable of improving performance of sparse coding features.

Recent research has shown the initial success of sparse coding (Sc) in solving many computer vision tasks. Motivated by the fact that kernel trick can capture the nonlinear similarity of features, which helps in finding a sparse representation of nonlinear features, we propose kernel sparse representation (KSR). Essentially, KSR is a sparse coding technique in a high dimensional feature space mapped by an implicit mapping function. We apply KSR to feature coding in image classification, face recognition, and kernel matrix approximation. More specifically, by incorporating KSR into spatial pyramid matching (SPM), we develop KSRSPM, which achieves a good performance for image classification. Moreover, KSR-based feature coding can be shown as a generalization of efficient match kernel and an extension of Sc-based SPM. We further show that our proposed KSR using a histogram intersection kernel (HIK) can be considered a soft assignment extension of HIK-based feature quantization in the feature coding process. Besides feature coding, comparing with sparse coding, KSR can learn more discriminative sparse codes and achieve higher accuracy for face recognition. Moreover, KSR can also be applied to kernel matrix approximation in large scale learning tasks, and it demonstrates its robustness to kernel matrix approximation, especially when a small fraction of the data is used. Extensive experimental results demonstrate promising results of KSR in image classification, face recognition, and kernel matrix approximation. All these applications prove the effectiveness of KSR in computer vision and machine learning tasks.

Semi-Nonnegative Matrix Factorization (Semi-NMF), as a variant of NMF, inherits the merit of parts-based representation of NMF and possesses the ability to process mixed sign data, which has attracted extensive attention. However, standard Semi-NMF still suffers from the following limitations. First of all, Semi-NMF fits data in a Euclidean space, which ignores the geometrical structure in the data. What’s more, Semi-NMF does not incorporate the discriminative information in the learned subspace. Last but not least, the learned basis in Semi-NMF is unnecessarily part based because there are no explicit constraints to ensure that the representation is part based. To settle these issues, in this paper, we propose a novel Semi-NMF algorithm, called Group sparsity and Graph regularized Semi-Nonnegative Matrix Factorization with Discriminability (GGSemi-NMFD) to overcome the aforementioned problems. GGSemi-NMFD adds the graph regularization term in Semi-NMF, which can well preserve the local geometrical information of the data space. To obtain the discriminative information, approximation orthogonal constraints are added in the learned subspace. In addition, ℓ 21 norm constraints are adopted for the basis matrix, which can encourage the basis matrix to be row sparse. Experimental results in six datasets demonstrate the effectiveness of the proposed algorithms.