CP Tensor Decomposition with Cannot-Link Intermode Constraints.

CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem especially for CP rank . In addition, existing approaches do not take into account uncertainty information of latent factors, as well as missing entries. To address these issues, we formulate CP factorization using a hierarchical probabilistic model and employ a fully Bayesian treatment by incorporating a sparsity-inducing prior over multiple latent factors and the appropriate hyperpriors over all hyperparameters, resulting in automatic rank determination. To learn the model, we develop an efficient deterministic Bayesian inference algorithm, which scales linearly with data size. Our method is characterized as a tuning parameter-free approach, which can effectively infer underlying multilinear factors with a low-rank constraint, while also providing predictive distributions over missing entries. Extensive simulations on synthetic data illustrate the intrinsic capability of our method to recover the ground-truth of CP rank and prevent the overfitting problem, even when a large amount of entries are missing. Moreover, the results from real-world applications, including image inpainting and facial image synthesis, demonstrate that our method outperforms state-of-the-art approaches for both tensor factorization and tensor completion in terms of predictive performance.

We propose a new low-rank tensor factorization where one mode is coded as a sparse linear combination of elements from an over-complete library. Our method, Shape Constrained Tensor Decomposition (SCTD) is based upon the CANDECOMP/PARAFAC (CP) decomposition which produces r-rank approximations of data tensors via outer products of vectors in each dimension of the data. The SCTD model can leverage prior knowledge about the shape of factors along a given mode, for example in tensor data where one mode represents time. By constraining the vector in the temporal dimension to known analytic forms which are selected from a large set of candidate functions, more readily interpretable decompositions are achieved and analytic time dependencies discovered. The SCTD method circumvents traditional flattening techniques where an N-way array is reshaped into a matrix in order to perform a singular value decomposition. A clear advantage of the SCTD algorithm is its ability to extract transient and intermittent phenomena which is often difficult for SVD-based methods. We motivate the SCTD method using several intuitively appealing results before applying it on a real-world data set to illustrate the efficiency of the algorithm in extracting interpretable spatio-temporal modes. With the rise of data-driven discovery methods, the decomposition proposed provides a viable technique for analyzing multitudes of data in a more comprehensible fashion.

With the development of Service-Oriented technologies, the amount of Web services grows rapidly. QoS-Aware Web service recommendation can help service users to design more efficient service-oriented systems. However, existing methods assume the QoS information for service users are all known and accurate, but in real case, there are always many missing QoS values in history records, which increase the difficulty of the missing QoS value prediction. By considering the user-service-time three dimension context information, we study a Temporal QoS-Aware Web Service Recommendation Framework which aims to recommend best candidates to service user's requirements and meanwhile improve the QoS prediction accuracy. We propose to envision such QoS value data as a tensor which is known as multi-way array, and formalize this problem as a tensor factorization model and propose a Tucker decomposition (TD)algorithm which is able to deal with the triadic relations of user-service-time model. However, one major challenge is that how to deal with the dynamic incoming service QoS value data streams. Thus, we introduce the incremental tensor factorization (ITF)method which is (a) scalable, (b) space efficient (it does not need to store the past data). Extensive experiments are conductedbasedc on our real-world QoS dataset collected on Planet-Lab, comprised of service invocation response-time values from 408 users on 5,473 Web services at 240 time periods. Comprehensive empirical studies demonstrate that our approach is faster and more accuracy than other approaches.

Operations with tensors, or multiway arrays, are increasingly prevalent in many applications involving multiway data analysis. This paper extends a third-order factorization strategy and tensor operations defined in a recent paper [M. E. Kilmer and C. D. Martin, Linear Algebra Appl., 435 (2011), pp. 641--658] to order-$p$ tensors. The extension to order-$p$ tensors is explained in a recursive way but for computational speed is implemented directly using the fast Fourier transform. A major motivation for considering factorization strategies for order-$p$ tensors is to devise new types of algorithms for general order-$p$ tensors which can be used in applications. We conclude with two applications in imaging. The first application is image deblurring, and the second application is video facial recognition. Both applications involve order-4 tensors.

In the past few decades, there has been rapid growth in quantity and variety of healthcare data. These large sets of data are usually high dimensional (e.g. patients, their diagnoses, and medications to treat their diagnoses) and cannot be adequately represented as matrices. Thus, many existing algorithms can not analyze them. To accommodate these high dimensional data, tensor factorization, which can be viewed as a higher-order extension of methods like PCA, has attracted much attention and emerged as a promising solution. However, tensor factorization is a computationally expensive task, and existing methods developed to factor large tensors are not flexible enough for real-world situations.To address this scaling problem more efficiently, we introduce SGranite, a distributed, scalable, and sparse tensor factorization method fit through stochastic gradient descent. SGranite offers three contributions: (1) Scalability: it employs a block partitioning and parallel processing design and thus scales to large tensors, (2) Accuracy: we show that our method can achieve results faster without sacrificing the quality of the tensor decomposition, and (3) FlexibleConstraints: we show our approach can encompass various kinds of constraints including l2 norm, l1 norm, and logistic regularization. We demonstrate SGranite’s capabilities in two real-world use cases. In the first, we use Google searches for flu-like symptoms to characterize and predict influenza patterns. In the second, we use SGranite to extract clinically interesting sets (i.e., phenotypes) of patients from electronic health records. Through these case studies, we show SGranite has the potential to be used to rapidly characterize, predict, and manage a large multimodal datasets, thereby promising a novel, data-driven solution that can benefit very large segments of the population.

A novel methodology for prediction of spatial-temporal activities using latent features

A variety of real-world applications such as complex industry process usually are lack of abundant training samples since the data acquiring process is time and labor consuming. Hence, it is important to utilize the limited training samples to build a sophisticated data-driven model, which may improve industry productivity. Recently, nonlinear learning models such as artificial neural networks and support vector machines have shown to be effective in modeling small-scale data by their strong modeling ability. However, these nonlinear learning models work as a black box and are often not human understandable and are difficult to be interpreted. In addition, in many applications, domain experts could provide us valuable expertise knowledge which may help further improve the modeling process. In this paper, we propose to integrate expertise knowledge to the nonlinear learning model to advance the data-driven modeling process in real-world applications. Experimental results on six benchmark datasets and a real-world industry application validate the effectiveness of the proposed model.

CANDECOMP/PARAFAC (CP) tensor decomposition is a popular unsupervised machine learning method with numerous applications. This process involves modeling a high-dimensional, multi-modal array (a tensor) as the sum of several low-dimensional components. In order to decompose a tensor, one must solve an optimization problem, whose objective is often given by the sum of the squares of the tensor and decomposition model entry differences. One algorithm occasionally utilized to solve such problems is CP-OPT-DGN, a damped Gauss-Newton all-at-once optimization method for CP tensor decomposition. However, there are currently no published results that consider the decomposition of large-scale (with up to billions of non-zeros), sparse tensors using this algorithm. This work considers the decomposition of large-scale tensors using an efficiently implemented CP-OPT-DGN method. It is observed that CP-OPT-DGN significantly outperforms CP-ALS (CP-Alternating Least Squares) and CP-OPT-QNR (a quasi-Newton-Raphson all-at-once optimization method for CP tensor decomposition), two other widely used tensor decomposition algorithms, in terms of accuracy and latent behavior detection.

This paper deals with tracking dynamic piecewise-constant network topologies that underpin complex systems including online social networks, neural pathways in the brain, and the world-wide web. Leveraging a structural equation model (SEM) in which only second-order statistics of exogenous inputs are known, the topology inference problem is recast using three-way tensors constructed from observed nodal data. To facilitate real-time operation, an adaptive parallel factor (PARAFAC) tensor decomposition is advocated to track the topology-revealing tensor factors. Preliminary tests on simulated data corroborate the effectiveness of the novel tensor-based approach.

Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis

The main aim of this paper is to develop a new algorithm for computing a nonnegative low multi‐rank tensor approximation for a nonnegative tensor. In the literature, there are several nonnegative tensor factorizations or decompositions, and their approaches are to enforce the nonnegativity constraints in the factors of tensor factorizations or decompositions. In this paper, we study nonnegativity constraints in tensor entries directly, and a low rank approximation for the transformed tensor by using discrete Fourier transformation matrix, discrete cosine transformation matrix, or unitary transformation matrix. This strategy is particularly useful in imaging science as nonnegative pixels appear in tensor entries and a low rank structure can be obtained in the transformation domain. We propose an alternating projections algorithm for computing such a nonnegative low multi‐rank tensor approximation. The convergence of the proposed projection method is established. Numerical examples for multidimensional images are presented to demonstrate that the performance of the proposed method is better than that of nonnegative low Tucker rank tensor approximation and the other nonnegative tensor factorizations and decompositions.

Traditional cluster analysis metrics rank clustering structures in terms of compactness and distinctness of clusters. However, in real world applications this is usually insufficient for selecting the optimal clustering structure. Domain experts and visual analysis are often relied on during evaluation, which results in a selection process that tends to be adhoc, subjective and difficult to reproduce. This work proposes the use of competency questions and a cluster scoring matrix to formalise expert knowledge and application requirements for qualitative evaluation of clustering structures. We show how a qualitative ranking of clustering structures can be integrated with traditional metrics to guide cluster evaluation and selection for generating representative energy consumption profiles that characterise residential electricity demand in South Africa. The approach is shown to be highly effective for identifying usable and expressive consumption profiles within this specific application context, and certainly has wider potential for efficient, transparent and repeatable cluster selection in real-world applications.

In recent years, large amounts of information have been transmitted through SNSs. Mainstream SNSs focus mainly on images. Given that many individuals use SNSs to express their sentiments, understanding the individual feelings of such users can be useful for helping identify potential demand in the society. Sentiment analysis of images in SNSs has thus attracted research attention. However, many existing studies have utilized a supervised learning approach with a high cost for labeling. By contrast, Wang et al., who used unsupervised learning, have acquired the sentiment of images through matrix decomposition from text information and visual information. However, text and visual information are not sufficient to acquire the desired result. In addition, sentiment polarity of the image is greatly influenced by text information. Therefore, this work will conduct a sentiment analysis of images by using tensor decomposition with an unsupervised learning approach. By considering the sentiment polarity of both text and visual features, this work aims to avoid being influenced only by text information.

Many interesting matrix decompositions/factorizations, and especially many tensor decompositions, have to be solved by non-convex optimization-based algorithms, that may converge to local optima. Hence, when interpretability of the components is a requirement, practitioners have to compute the decomposition (e.g. CPD) many times, with different initializations, to verify whether the components are reproducible over repetitions of the optimization. However, it is non-trivial to assess such reliability or stability when multiple local optima are encountered. We propose an efficient algorithm that clusters the different repetitions of the decomposition according to the local optimum that they belong to, offering a diagnostic tool to practitioners. Our algorithm employs a graph-based representation of the decomposition, in which every repetition corresponds to a node, and similarities between components are encoded as edges. Clustering is then performed by exploiting a property known as cycle consistency, leading to a low-rank approximation of the graph. We demonstrate the applicability of our method on realistic electroencephalographic (EEG) data and synthetic data.

The unprecedented outbreak of Severe Acute Respiratory Syndrome Coronavirus-2 (SARS-CoV-2), or COVID-19, continues to be a significant worldwide problem. As a result, a surge of new COVID-19 related research has followed suit. The growing number of publications requires document organization methods to identify relevant information. In this paper, we expand upon our previous work with clustering the CORD-19 dataset by applying multi-dimensional analysis methods. Tensor factorization is a powerful unsupervised learning method capable of discovering hidden patterns in a document corpus. We show that a higher-order representation of the corpus allows for the simultaneous grouping of similar articles, relevant journals, authors with similar research interests, and topic keywords. These groupings are identified within and among the latent components extracted via tensor decomposition. We further demonstrate the application of this method with a publicly available interactive visualization of the dataset.