Joint factor analysis and latent clustering

Coupled tensor factorization methods are useful for sensor fusion, combining information from several related datasets by simultaneously approximating them by products of latent tensors. In these methods, the choice of a suitable optimization criteria becomes difficult as observed datasets may have different statistical characteristics and their relative importance for the task at hand can vary. In this paper, we present an algorithmic framework for coupled factorization that, while estimating a latent factorization also estimates a specific ß-divergence for each dataset as well as the relative weights in an overall additive cost function. We evaluate the proposed method on both synthetical and real datasets, where we apply our methods on a link prediction problem. The results show that our method outperforms the state-of-the-art by a significant margin.

Many spectral unmixing approaches ranging from geometry, algebra to statistics have been proposed, in which nonnegative matrix factorization (NMF)-based ones form an important family. The original NMF-based unmixing algorithm loses the spectral and spatial information between mixed pixels when stacking the spectral responses of the pixels into an observed matrix. Therefore, various constrained NMF methods are developed to impose spectral structure, spatial structure, and spectral-spatial joint structure into NMF to enforce the estimated endmembers and abundances preserve these structures. Compared with matrix format, the third-order tensor is more natural to represent a hyperspectral data cube as a whole, by which the intrinsic structure of hyperspectral imagery can be losslessly retained. Extended from NMF-based methods, a matrix-vector nonnegative tensor factorization (NTF) model is proposed in this paper for spectral unmixing. Different from widely used tensor factorization models, such as canonical polyadic decomposition CPD) and Tucker decomposition, the proposed method is derived from block term decomposition, which is a combination of CPD and Tucker decomposition. This leads to a more flexible frame to model various application-dependent problems. The matrix-vector NTF decomposes a third-order tensor into the sum of several component tensors, with each component tensor being the outer product of a vector (endmember) and a matrix (corresponding abundances). From a formal perspective, this tensor decomposition is consistent with linear spectral mixture model. From an informative perspective, the structures within spatial domain, within spectral domain, and cross spectral-spatial domain are retreated interdependently. Experiments demonstrate that the proposed method has outperformed several state-of-the-art NMF-based unmixing methods.

Matrix factorization of knowledge bases in universal schema has facilitated accurate distantlysupervised relation extraction.This factorization encodes dependencies between textual patterns and structured relations using lowdimensional vectors defined for each entity pair; although these factors are effective at combining evidence for an entity pair, they are inaccurate on rare pairs, or for relations that depend crucially on the entity types.On the other hand, tensor factorization is able to overcome these shortcomings when applied to link prediction by maintaining entity-wise factors.However these models have been unsuitable for universal schema.In this paper we first present an illustration on synthetic data that explains the unsuitability of tensor factorization to relation extraction with universal schemas.Since the benefits of tensor and matrix factorization are complementary, we then investigate two hybrid methods that combine the benefits of the two paradigms.We show that the combination can be fruitful: we handle ambiguously phrased relations, achieve gains in accuracy on real-world relations, and demonstrate that entity embeddings encode entity types.

Querying cohesive subgraphs on temporal graphs (e.g., social network, finance network, etc.) with various conditions has attracted intensive research interests recently. In this paper, we study a novel Temporal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$(k,\mathcal {X})$</tex-math></inline-formula> -Core Query (TXCQ) that extends a fundamental Temporal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -Core Query (TCQ) proposed in our conference paper by optimizing or constraining an arbitrary metric <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}$</tex-math></inline-formula> of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core, such as size, engagement, interaction frequency, time span, burstiness, periodicity, etc. Our objective is to address specific TXCQ instances with conditions on different <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}$</tex-math></inline-formula> in a unified algorithm framework that guarantees scalability. For that, this journal paper proposes a taxonomy of measurement <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}(\cdot )$</tex-math></inline-formula> and achieve our objective using a two-phase framework while <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}(\cdot )$</tex-math></inline-formula> is time-insensitive or time-monotonic. Specifically, Phase 1 still leverages the query processing algorithm of TCQ to induce all distinct <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -cores during a given time range, and meanwhile locates the “time zones” in which the cores emerge. Then, Phase 2 conducts fast local search and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}$</tex-math></inline-formula> evaluation in each time zone with respect to the time insensitivity or monotonicity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}(\cdot )$</tex-math></inline-formula> . By revealing two insightful concepts named tightest time interval and loosest time interval that bound time zones, the redundant core induction and unnecessary <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}$</tex-math></inline-formula> evaluation in a zone can be reduced dramatically. Our experimental results demonstrate that TXCQ can be addressed as efficiently as TCQ, which achieves the latest state-of-the-art performance, by using a general algorithm framework that leaves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {X}(\cdot )$</tex-math></inline-formula> as a user-defined function.

In recent years, multidimensional data is becoming increasingly popular in recommender systems. For example, social tagging systems encourage users to employ user-defined keywords to help manage content in a personalized way. Recommender systems built upon social tagging systems utilize social tagging data to improve recommendation accuracy. Context-aware recommender systems incorporate context information (e.g. time, location, weather, etc.) into recommendation models to make recommendations more personalized. User profiles play a crucial role in recommender systems because user profiles provide the information about users’ information needs based on which personalized recommendations can be generated. Existing user profiling techniques for collaborative filtering (CF) recommender systems based on multidimensional data mostly analyze data through splitting the multidimensional relations into lower-dimensional relations. However, this leads to the loss of multidimensionality in user-item interactions. A different type of methods for addressing multidimensional data is factorization techniques such as tensor factorization (TF), which discover holistic latent relations between all dimensions in multidimensional data. However, TF models are not constructed to analyze any particular dimension of multidimensional data. But for many application domains, certain dimensions are of particular interest such as the user dimension in recommender systems. However, user profiles are not explicitly reflected in TF models. In this paper, we propose a unified approach to profile any particular dimension in multidimensional data based on partially-decomposed Tucker models (TM). We prove that the proposed profiling approach is a compatible higher-order extension of the matrix factorization based approaches. Moreover, we prove that the proposed TM-based profiles can be updated incrementally with new data without reconstructing the entire profiles. We integrate the proposed TF-based user/item profiling approach into neighborhood-based collaborative filtering recommenders for making top-N item recommendations. Extensive experiments have been conducted using real-world social tagging datasets to demonstrate that the CF recommenders integrated with the proposed user/item profiles outperform state-of-the-art CF recommendation approaches in terms of recommendation accuracy.

Joint Factor Analysis (JFA) has become the state-of-the-art technique in the problem of speaker verification (1, 2). At the same time, the training of eigenvoice matrix seems to be a heavy burden to us, because it requires lots of multi-channel data, which largely determines the performance of the system. In this paper, we first try to exploit an upper bound performance of the JFA system in a non-normal way, and then proposed a new technique which we referred as Residual Factor Analysis (RFA), in which we replace the eigenvoice matrix in JFA system with the residual vector, to remove the heavy burden of training eigenvoice matrix. We tested the proposed technique on the core condition of NIST 2006 speaker recognition evaluation (SRE 06) and obtained equivalent results to JFA system (equal error rate of about 3.99%)(1), while our method requires no extra multi-channel data except some for training eigenchannel matrix.

This paper proposes a hybrid model of aspect-oriented sentiment prediction which integrates tensor factorization (TF) and sentiment utility logistic model (SULM). First, using sentiment dictionary words as seeds, the aspect or opinion words can be extended iteratively through double propagation. Accordingly, the users’ reviews could be represented as the features in user-item-aspect space, in which prediction model could be built. Various combinations of the hybrid model were proposed and evaluated on the Chinese reviews on places of interest from Trip Advisor. Experimental results show that the hybrid model can achieve better prediction performance than TF or SULM. The hybrid model also outperforms either TF or SULM while handling cold-start problem.

It is known that various types of location privacy attacks can be carried out using a personalized transition matrix that is learned for each target user, or a population transition matrix that is common to all target users. However, since many users disclose only a small amount of location information in their daily lives, the training data can be extremely sparse. The aim of this paper is to clarify the risk of location privacy attacks in this realistic situation. To achieve this aim, we propose a learning method that uses tensor factorization (or matrix factorization) to accurately estimate personalized transition matrices (or a population transition matrix) from a small amount of training data. To avoid the difficulty in directly factorizing the personalized transition matrices (or population transition matrix), our learning method first factorizes a transition count tensor (or matrix), whose elements are the number of transition counts that the user has made, and then normalizes counts to probabilities. We focus on a localization attack, which derives an actual location of a user at a given time instant from an obfuscated trace, and compare our learning method with the maximum likelihood (ML) estimation method in both the personalized matrix mode and the population matrix mode. The experimental results using four real data sets show that the ML estimation method performs only as well as a random guess in many cases, while our learning method significantly outperforms the ML estimation method in all of the four data sets.

Tensors are higher order generalizations of matrices to model multi-aspect data, e.g., a set of purchase records with the schema (user_id, product_id, timestamp, feedback). Tensor factorization is a powerful technique for generating a model from a tensor, just like matrix factorization generates a model from a matrix, but with higher accuracy and richer information as more attributes are available in a higher- order tensor than a matrix. The data model obtained by tensor factorization can be used for classification, recommendation, anomaly detection, and so on. Though having a broad range of applications, tensor factorization has not been popularly applied compared with matrix factorization that has been widely used in recommender systems, mainly due to the high computational cost and poor scalability of existing tensor factorization methods. Efficient and scalable tensor factorization is particularly challenging because real world tensor data are mostly sparse and massive. In this paper, we propose a novel distributed algorithm, called Lock-Free Tensor Factorization (LFTF), which significantly improves the efficiency and scalability of distributed tensor factorization by exploiting asynchronous execution in a re-formulated problem. Our experiments show that LFTF achieves much higher CPU and network throughput than existing methods, converges at least 17 times faster and scales to much larger datasets.

We propose a simple and effective strategy to cope with dataset shifts in text-dependent speaker recognition based on Joint Factor Analysis (JFA). We have previously shown how to compensate for lexical variation in text-dependent JFA by adapting the Universal Background Model (UBM) to individual passphrases. A similar type of adaptation can be used to port a JFA model trained on out-of-domain data to a given text-dependent task domain. On the RSR2015 test set we found that this type of adaptation gave essentially the same results as in-domain JFA training. To explore this idea more fully, we experimented with several types of JFA model on the CSLU speaker recognition dataset. Taking a suitably configured JFA model trained on NIST data and adapting it in the proposed way results in a 22% reduction in error rates compared with the GMM/UBM benchmark. Error rates are still much higher than those that can be achieved on the RSR2015 test set with the same strategy but cheating experiments suggest that if large amounts of in-domain training data are available, then JFA modelling is capable in principle of achieving very low error rates even on hard tasks such as CSLU.

Tensor factorization method based on review text semantic similarity for rating prediction

This paper proposes parallel method of non-negative large sparse matrix factorization - very popular technique in computational linguistics. Proposed blocks factorization approach using Latent Dirichlet allocation, for speedup of non-negative sparse extra-large linguistic matrix and tensor factorization. The proposed method also supports the scalability of the model.

With the development of Service-Oriented technologies, the amount of Web services grows rapidly. QoS-Aware Web service recommendation can help service users to design more efficient service-oriented systems. However, existing methods assume the QoS information for service users are all known and accurate, but in real case, there are always many missing QoS values in history records, which increase the difficulty of the missing QoS value prediction. By considering the user-service-time three dimension context information, we study a Temporal QoS-Aware Web Service Recommendation Framework which aims to recommend best candidates to service user's requirements and meanwhile improve the QoS prediction accuracy. We propose to envision such QoS value data as a tensor which is known as multi-way array, and formalize this problem as a tensor factorization model and propose a Tucker decomposition (TD)algorithm which is able to deal with the triadic relations of user-service-time model. However, one major challenge is that how to deal with the dynamic incoming service QoS value data streams. Thus, we introduce the incremental tensor factorization (ITF)method which is (a) scalable, (b) space efficient (it does not need to store the past data). Extensive experiments are conductedbasedc on our real-world QoS dataset collected on Planet-Lab, comprised of service invocation response-time values from 408 users on 5,473 Web services at 240 time periods. Comprehensive empirical studies demonstrate that our approach is faster and more accuracy than other approaches.

In order to increase the recognition accuracy under different speaking manners and different channels, the paper introduces joint factor analysis into the recognition system by training speaker and channel space. Then based on analyzing distribution of the channel factor by principal component analysis method, vector quantization algorithm is proposed by the paper to achieve the optimal channel space matrix which avoids the instability of channel factor caused from the randomness of initial channel space. The experiment result shows that the modified joint factor analysis system can increase the recognition accuracy compared to traditional joint factor analysis method under complex conditions.

Low-rank matrix factorizations such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD) and Non-negative Matrix Factorization (NMF) are a large class of methods for pursuing the low-rank approximation of a given data matrix. The conventional factorization models are based on the assumption that the data matrices are contaminated stochastically by some type of noise. Thus the point estimations of low-rank components can be obtained by Maximum Likelihood (ML) estimation or Maximum a posteriori (MAP). In the past decade, a variety of probabilistic models of low-rank matrix factorizations have emerged. The most significant difference between low-rank matrix factorizations and their corresponding probabilistic models is that the latter treat the low-rank components as random variables. This paper makes a survey of the probabilistic models of low-rank matrix factorizations. Firstly, we review some probability distributions commonly-used in probabilistic models of low-rank matrix factorizations and introduce the conjugate priors of some probability distributions to simplify the Bayesian inference. Then we provide two main inference methods for probabilistic low-rank matrix factorizations, i.e., Gibbs sampling and variational Bayesian inference. Next, we classify roughly the important probabilistic models of low-rank matrix factorizations into several categories and review them respectively. The categories are performed via different matrix factorizations formulations, which mainly include PCA, matrix factorizations, robust PCA, NMF and tensor factorizations. Finally, we discuss the research issues needed to be studied in the future.