Performance Evaluation of Latent Variable Models with Sparse Priors

Scalable and memory-efficient sparse learning for classification with approximate Bayesian regularization priors

In this article we study the sparse signal recovery problem in a Bayesian framework using a novel Bootstrapped Sparse Bayesian Learning method. Sparse Bayesian Learning (SBL) framework is an effective tool for pruning out the irrelevant features and ending up with a sparse representation. In SBL the choice of prior over the variances of the Gaussian Scale mixture has been an interesting area of research for some time now. This motivates us to use a more generalized maximum entropy density as the prior which results in a new variant of SBL. It has been shown to perform better than traditional SBL empirically and it also accelerates the pruning procedure. Because of this advantage, this variant of SBL can be claimed as more robust choice as it is less sensitive to the threshold for pruning. Theoretical justifications have also been provided to show that the proposed model actually promotes sparse point estimates.

Compressive sensing based techniques have recently been applied successfully to underwater acoustics problems such as beamforming and matched field processing. Sparse Bayesian Learning (SBL) is one of the fast compressive sensing methods and is formulated using Bayesian statistics. In the SBL framework, source amplitudes are modeled as complex Gaussian random variables with unknown variance. Evidence maximization is performed to estimate the unknown source amplitude variance and source position. A major advantage of SBL over more commonly used methods such as basis pursuit is that it is computationally faster. In this work, we develop a robust sparse Bayesian learning algorithm that can account for model mismatch leading to errors in the replica vector dictionary. Specifically, the likelihood function is modified so that it takes into account the covariance matrix of error in replica vectors. We also extend the SBL algorithm to process observations from multiple frequencies. The derived update rule combines observations from all the frequencies in a holistic manner. We demonstrate the robust SBL algorithm with simulations. A comparison is done with other compressed sensing methods including basis pursuit and orthogonal matching pursuit.

In this paper,we propose a sparse Baysesian learning (SBL) based approach for the DOA estimation in the presence of coherent sources. First, the difference technique is used to enhance the input SNR. Then, we construct a virtual array manifold to eliminate the cross-term effect between each coherent group, after eigenvalue decomposition (EVD) of the difference covariance matrix, reduce the dimension of input data and regard each column as one snapshot of the virtual signal containing all bearing information of the coherent sources. Finally, the DOAs of the coherent sources are estimated via the SBL algorithm. Numerical results validate superior performance of the proposed approach.

A new maximum a posteriori (MAP) formulation is shown to be a straightforward and intuitive way to derive optimal blind equalization cost functions. This MAP method provides a general, systematic way to derive blind adaptation methods using the given pdf of the input and a convolutional noise model. A general blind equalization/deconvolution cost function known as Gray's Variable Norm, is shown to be derivable using the MAP formulation presented here. Gray's Variable Norm (1979) is a superset of existing blind equalization cost functions, including the Godard and Sato algorithms as special cases. The MAP method is capable of deriving cost functions needed for a wide variety of problems, including those with infinite variance pdfs and even multichannel problems. >

This paper showcases novel underground stratification based on sparse machine learning (SML) methods, including sparse Bayesian learning (SBL) and least absolute shrinkage selection operator (Lasso). The SML methods proposed by the authors were applied to two- and three-dimensional underground stratification of actual sites, Odagawa Riverbank (Okayama, Japan) and New Lock (Terneuzen, the Netherlands), to demonstrate their performances. Cone penetration test (CPT) data were available in both sites, and they were converted to the soil behavior type (SBT) index for the underground stratification analysis. The trends of Ic/SBT profiles and their distribution colormaps estimated by the two methods were compared to discuss the methodological characteristics. For the Odagawa Riverbank case, the detection ratio of SBT obtained by the two methods was also compared to investigate the estimation accuracy in terms of stratification ability.

This paper describes how to construct a probability map using sparse representation and dictionary learning to indicate the probability of each optic disk pixel of belonging to the optic cup. This probability map will be used in the future as input to a method for automatically detecting glaucoma from color fundus images. The probability map was obtained constructing a model (using the Bayes classifier) which takes into account texture information, by means of sparse representation and RLS-DLA dictionary learning technique, and intensity information. Several experiments on a private database are presented in this work. The results are compared with the segmentation made by specialists, highlighting the promising performance of this technique in difficult cases where the optic cup is barely visible.

Introduces nonparametric frequency-domain methods for tuning closed-loop systems. A linear quadratic criterion is used for guiding the scheme towards an optimal-regulation state. The computation of tuning directions is performed with unbiased estimates of first and second derivatives of frequency-domain expressions of the cost function. The actual cost function is computed with time-domain measurements taken along the tuning directions. Similarly to other iterative schemes, caution has to be observed at each adjustment step. A seemingly innocuous example is used to illustrate a situation where a sudden unstable closed-loop behaviour might happen but is arrested by the cautious modification.

This paper proposed a new global optimization approach for electric machine design problems. Its algorithm uses the error function between the actual cost function at a given computational step, and its limit value which is lower than the desired global minimum. The error function provides a good indication of how far or close the cost function is approaching its limit solution. This method may be considered to be self adaptive in a sense that the higher the error is, the higher the step size of the variables will be. This feature will help the function to escape local solutions and tend to reach the targeted solution. The proposed approach has been tested on various typical multimodal functions and particularly, applied to induction motor design optimization problems. The algorithm of this method is very simple and easy to implement but yet powerful and effective as highlighted in the test results.

Sparse representation is efficient to approximately recover signals by a linear composition of a few bases from an over-complete dictionary. However, in the scenario of data compression, its efficiency and popularity are hindered due to the extra overhead for encoding the sparse coefficients. Therefore, how to establish an accurate rate model in sparse coding and dictionary learning becomes meaningful, which has been not fully exploited in the context of sparse representation. According to the Shannon entropy inequality, the variance of data source can bound its entropy, thus can reflect the actual coding bits. Therefore, a Globally Variance-Constrained Sparse Representation (GVCSR) model is proposed, where a variance-constrained rate term is introduced to the conventional sparse representation. To solve the non-convex optimization problem, we employ the Alternating Direction Method of Multipliers (ADMM) for sparse coding and dictionary learning, both of which have shown state-of-the-art rate-distortion performance in image representation.

An automatic segmentation method of hippo-campus for volume measurement in MR brain images by using sparse patch representation and discriminative dictionary learning is proposed in this paper, which can overcome the limitation of multi-atlas approaches that mostly rely on similarity between target image and atlases for more accurate segmentation. In the proposed method, atlases are registered to a target image, then patch-based features including intensity and gradient are extracted from registered atlases for a dictionary learning model, from which dictionaries and classifiers are learned simultaneously. The label for voxels in the target image is determined via solving sparse representation of target patch over learned dictionary. Performance of the proposed method is evaluated on 30 subjects from the public dataset CIND by cross validation. Qualitative and quantitative comparisons illustrate that the proposed method is competitive compared with state-of-the-art approaches based on multi-atlas.

We propose a semisupervised sparse graph learning (SSGL) method for feature extraction of hyperspectral remote sensing imagery in this paper. The proposed SSGL method aims to build a semisupervised sparse graph that can maximize the class discrimination and preserve the local neighborhood information by combining labeled and unlabeled samples. In our semisupervised sparse graph, we connect labeled samples according to their label information and sparse representation, connect unlabeled samples by sparse representation which set unlabeled training samples set as dictionary. Moreover, by setting sparse connections between labeled sample and unlabeled samples, the label information can be well propagated from labeled samples to unlabeled samples. In this way, the similarity of data points can be well modelled in the spectral feature space, and the data distribution characteristics can be better expressed. Experimental results on real hyperspectral images (HSIs) demonstrate the advantages of our method compared to some related feature extraction methods.

Texture classification is a classical yet still active topic in computer vision and pattern recognition. Recently, several new texture classification approaches by modeling texture images as distributions over a set of textons have been proposed. These textons are learned as the cluster centers in the image patch feature space using the K-means clustering algorithm. However, the Euclidian distance based the K-means clustering process may not be able to well characterize the intrinsic feature space of texture textons, which if often embedded into a low dimensional manifold. Inspired by the great success of l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm minimization based sparse representation (SR), in this paper we propose a novel texture classification method via patch-based sparse texton learning. Specifically, the dictionary of textons is learned by applying SR to image patches in the training dataset. The SR coefficients of the test images over the dictionary are used to construct the histograms for texture classification. Experimental results on benchmark database validate the effectiveness of the proposed method.

Hyperspectral unmixing is a technique that can be used to find the materials and their corresponding proportions present in observed mixed pixels. It has been widely used for hyperspectral data exploitation. Traditionally, this problem is solved using the constrained least squares optimization, which would lead to a dense solution. In recent years, researchers have observed that the number of materials in a mixed pixel is generally much smaller than the whole number of materials present in a given scene. They proposed to solve the problem through sparse unmixing approaches, which generally lead to constrained optimization. These sparse unmixing approaches have received considerable attention over the past few years. From the perspective of Bayesian inference, these methods assume fixed priors. In this article, we propose to solve the spectral unmixing problem by using sparse Bayesian learning (SBL) framework. In contrast to the approaches that use fixed priors, SBL incorporates a hierarchical parameterized prior that encourages sparsity in representation in its Bayesian inference to obtain a sparse solution. Experimental results on simulated data sets and a real hyperspectral scene demonstrate that the proposed method can achieve good performance compared with some state-of-the-art sparse unmixing algorithms. Our results also show that the proposed method can achieve good performance with a set of default parameter settings.

This paper discusses about a sparsity based algorithm used for Hyperspectral Image (HSI) classification where the test pixel vectors are sparsely represented as the linear combination of a few number of training samples from a well-organised dictionary matrix. The sparse vector is obtained using Basis Pursuit (BP) which is a constrained l4 minimization problem. This problem is solved by using a simple and powerful iterative algorithm known as Alternating Direction Method of Multipliers (ADMM) which significantly reduces the computational complexity of the problem and thereby speeds up the convergence. The classification accuracy is considerably improved by including efficient preprocessing techniques to remove the unwanted information (noise) present in Hyperspectral images. This paper uses a fast and reliable denoising technique based on Legendre Fenchel Transformation (LFT) to effectively denoise each band of HSI prior to ADMM based classification (proposed method). A comparison of proposed technique with one of the convex optimization tools namely, CVX is given to exhibit the fast convergence of the former method. The experiment is performed on standard Indian Pines dataset captured using AVIRIS sensor. The potential of the proposed method is illustrated by analyzing the classification indices obtained with and without applying any preprocessing methods. With only 10% training set, an overall accuracy of 96.76% is obtained for the proposed method at a much faster rate compared to computation time taken by CVX solver.