A new descriptive clustering algorithm based on Nonnegative Matrix Factorization

In this work, we will consider the dimension reduction of the set of time series, such as economic data, to find the meaningful basis vector for the set of data, and indicate which data use which basis vector. Usually each of the time series is analyzed independently in economics but here we will analyze the set of time series simultaneously. Since some of the economic data are measured as positive values and we want to decompose them as a mixture of the parts, we will apply non-negative matrix factorization to the economic data. Non-negative matrix factorization can compress dimensions by approximating a non-negative matrix with the product of two non-negative matrices. The two non-negative matrices are called the coefficient matrix and the basis matrix, and the basis matrix can be considered as a dimensionally compressed matrix. If the standard non-negative matrix factorization is used for economic data, the basis matrix may not be smooth. We think that the basis vectors should be smooth except a few special economical incidents. In the proposed method, a Graph-based non-negative matrix factorization is introduced to regularize the basis matrix of the time series. A path graph for representing the time series of economic data is incorporated into the non-negative matrix factorization as regularization. As a result, basis vectors that maintains the time series of economic data are decomposed. Furthermore, we propose to introduce a sparsity in the non-negative matrix factorization. Traditionally, the sparsity incorporated into non-negative matrix factorization has been used for basis vectors. However, the proposed method introduces the sparsity for coefficient vectors. Thus the proposed method, which simultaneously incorporates the sparsity for the coefficient vectors and the smoothness for the basis vectors, can extract the smooth basis vectors and the original economical data are approximated as the weighted sum of the few bases vectors. This allows us to discover economic trends and the best-fit trends for each data at the same time.

Sparse nonnegative matrix factorization with ℓ0-constraints

Study of school accidents reinforces need for coordination and basic motion training: School Sports Accidents: Analysis of Causes, Modes, and Frequencies Kelm J, Ahlhelm F, Pape D, et al. J Pediatr Orthop. 2001;21:165–168. Study conducted at Chirurgische Abteilung, Krankenhaus Neunkirchen/Saar, Akademisches Lehrkrankenhaus der Universitdt des Saarlandes, Germany

In this paper, we propose a novel scheme to supervised nonnegative matrix factorization (NMF). We formulate the supervised NMF as a sparse optimization problem assuming the availability of a set of basis vectors, some of which are irrelevant to a given matrix to be decomposed. The proposed scheme is presented in the context of music transcription and musical instrument recognition. In addition to the nonnegativity constraint, we introduce three regularization terms: (i) a block ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm to select relevant basis vectors, and (ii) a temporal-continuity term plus the popular ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norm to estimate correct activation vectors. We present a state-of-the-art convex-analytic iterative solver which ensures global convergence. The number of basis vectors to be actively used is obtained as a consequence of optimization. Simulation results show the efficacy of the proposed scheme both in the case of perfect/imperfect basis matrices.

This work presents the use of Dictionary Learning methods for a sparse representation of 4D seismic data in history matching. We consider a trade-off between the number of coefficients retained in the sparse data representation, the computational cost, and how well we can capture the main features of the 4D seismic signal. K-SVD is an iterative algorithm used in Dictionary Learning problems that alternates between the calculation of the sparse representation vector and dictionary update. For the definition of the sparse representation vector, one can constrain the problem into two distinct approaches: (1) sparsity-constrained; and (2) error-constrained. We evaluated the two methods and the influence of critical parameters of the algorithm (dictionary size, number of iterations, patch size, and training dataset size) using a synthetic reservoir model. Results showed that regardless of which of the constrained approaches we used, the dictionary learning method can capture the main features of the 4D seismic signal with a sparse representation. However, the number of nonzero coefficients is highly dependent on the approach. Besides, dictionary size, number of iterations, patch size, and training dataset size, also have a significant impact on the number of coefficients and the computational cost. Consequently, the selection of these variables is crucial as it might affect the history-matching process. In a permanent-reservoir-monitoring (big data) scenario, the use of sparse representation allows us to retain the main features of the reservoir in a reasonably sized data representation that can be directly used in a history matching method, leading to improved reservoir characterization and a better understanding of the reservoir properties. This paper gives a practical implementation of a technique already used in other areas (image denoising, processing of raw seismic data, and facies representation) applied to 4D seismic data.

Non-negative matrix factorization (NMF) has been one of the most popular methods for feature learning in the field of machine learning and computer vision. Most existing works directly apply NMF on high-dimensional image datasets for computing the effective representation of the raw images. However, in fact, the common essential information of a given class of images is hidden in their low rank parts. For obtaining an effective low-rank data representation, we in this paper propose a non-negative low-rank matrix factorization (NLMF) method for image clustering. For the purpose of improving its robustness for the data in a manifold structure, we further propose a graph regularized NLMF by incorporating the manifold structure information into our proposed objective function. Finally, we develop an efficient alternating iterative algorithm to learn the low-dimensional representation of low-rank parts of images for clustering. Alternatively, we also incorporate robust principal component analysis into our proposed scheme. Experimental results on four image datasets reveal that our proposed methods outperform four representative methods.

Traditional Nonnegative Matrix Factorization (NMF) is a linear and unsupervised algorithm. This would limit the classification power of NMF for the complicated data. To overcome the above limitations of NMF, this paper proposes a novel supervised and nonlinear NMF algorithm based on kernel theory and discriminant analysis. We incorporate the class label information into the decomposition of NMF in the Reproducing Kernel Hilbert Space (RKHS). A new iterative algorithm for NMF is derived and the objective function is non-increasing under the update rules. The proposed method is evaluated on the ORL and Yale face databases. The experimental results demonstrate the the proposed method is superior to the state of-the-art algorithms.

Semi-supervised graph regularized nonnegative matrix factorization with local coordinate for image representation

Guitar string separation is a novel and complicated task. Guitar notes are not pure steady-state signals, hence, we hypothesize that neither Non-Negative Matrix Factorization (NMF) nor Non-Negative Matrix Factor Deconvolution (NMFD) are optimal for separating them. Therefore, we separate steady-state and transient parts using Harmonic-Percussive Separation (HPS) as a preprocessing step. Then, we use NMF for factorizing the harmonic part and NMFD for deconvolving the percussive part. We make use of a hexaphonic guitar dataset which allows for objective evaluation. In addition, we compare several types of time-frequency mask and introduce an intuitive way to combine a binary mask with a ratio mask. We show that the HPS mask type has an effect on source estimation. Our proposed method achieved results comparable to NMF without HPS. Finally, we show that the optimal mask at the final separation stage depends on the estimation algorithm.

Non-negative matrix factorization for speech/music separation using source dependent decomposition rank, temporal continuity term and filtering

We propose a new method to incorporate rich statistical priors, modeling temporal gain sequences in the solutions of nonnegative matrix factorization (NMF). The proposed method can be used for single-channel source separation (SCSS) applications. In NMF based SCSS, NMF is used to decompose the spectra of the observed mixed signal as a weighted linear combination of a set of trained basis vectors. In this work, the NMF decomposition weights are enforced to consider statistical and temporal prior information on the weight combination patterns that the trained basis vectors can jointly receive for each source in the observed mixed signal. The Hidden Markov Model (HMM) is used as a log-normalized gains (weights) prior model for the NMF solution. The normalization makes the prior models energy independent. HMM is used as a rich model that characterizes the statistics of sequential data. The NMF solutions for the weights are encouraged to increase the log-likelihood with the trained gain prior HMMs while reducing the NMF reconstruction error at the same time.

Non-negative Matrix Factorization (NMF) is an intensively used technique for obtaining parts-based, lower dimensional and non-negative representation. Researchers in biology, medicine, pharmacy and other fields often prefer NMF over other dimensionality reduction approaches (such as PCA) because the non-negativity of the approach naturally fits the characteristics of the domain problem and its results are easier to analyze and understand. Despite these advantages, obtaining exact characterization and interpretation of the NMF’s latent factors can still be difficult due to their numerical nature. Rule-based approaches, such as rule mining, conceptual clustering, subgroup discovery and redescription mining, are often considered more interpretable but lack lower-dimensional representation of the data. We present a version of the NMF approach that merges rule-based descriptions with advantages of part-based representation offered by the NMF. Given the numerical input data with non-negative entries and a set of rules with high entity coverage, the approach creates the lower-dimensional non-negative representation of the input data in such a way that its factors are described by the appropriate subset of the input rules. In addition to revealing important attributes for latent factors, their interaction and value ranges, this approach allows performing focused embedding potentially using multiple overlapping target labels.

Obtaining an optimum data representation is a challenging issue that arises in many intellectual data processing techniques such as data mining, pattern recognition, and gene clustering. Many existing methods formulate this problem as a nonnegative matrix factorization (NMF) approximation problem. The standard NMF uses the least square loss function, which is not robust to outlier points and noises and fails to utilize prior label information to enhance the discriminability of representations. In this study, we develop a novel matrix factorization method called robust semisupervised nonnegative local coordinate factorization by integrating robust NMF, a robust local coordinate constraint, and local spline regression into a unified framework. We use the l2,1 norm for the loss function of the NMF and a local coordinate constraint term to make our method insensitive to outlier points and noises. In addition, we exploit the local and global consistencies of sample labels to guarantee that data representation is compact and discriminative. An efficient multiplicative updating algorithm is deduced to solve the novel loss function, followed by a strict proof of the convergence. Several experiments conducted in this study on face and gene datasets clearly indicate that the proposed method is more effective and robust compared to the state-of-the-art methods.

This paper proposes two algorithms for Voice Activity Detection (VAD) based on sparse representation in spectro-temporal domain. The first algorithm was made using two-dimensional STRF (Spectro-Temporal Response Field) space based on sparse representation. Dictionaries with different atomic sizes and two dictionary learning methods were investigated in this approach. This algorithm revealed good results at high SNRs (signal-to-noise ratio). The second algorithm, whose approach is more complicated, suggests a speech detector using the sparse representation in four-dimensional STRF space. Due to the large volume of STRF's four-dimensional space, this space was divided into cubes, with dictionaries made for each cube separately by NMF (non-negative matrix factorization) learning algorithm. Simulation results were presented to illustrate the effectiveness of our new VAD algorithms. The results revealed that the achieved performance was 90.11% and 91.75% under -5 dB SNR in white and car noise respectively, outperforming most of the state-of-the-art VAD algorithms.

Nonnegative matrix factorization (NMF) is known as a parts-based linear representation for nonnegative data. This method has been applied for blind source separation (BSS) when the sources are nonnegative. This paper presents a new NMF method for independent component analysis (ICA), which is useful for BSS without the nonnegativity constraint. Using this method, we transform the sources by their cumulative distribution functions (CDFs) and perform the nonparametric quantization to construct a nonnegative matrix where each entry represents the joint probability density of two transformed signals. The NMF procedure is accordingly realized to find the ICA demixing matrix. The independence between sources is maximized towards attaining the uniformity in the joint probability density. In the experiments on the separation of signal and music signals, we show the effectiveness of the proposed NMF-ICA compared to the infomax ICA and FastICA algorithms.