Distributed Methods for High-Dimensional and Large-Scale Tensor Factorization

Facial images change appearance due to multiple factors such as poses, lighting variations, facial expressions, etc. Tensor approach, an extension of conventional matrix, is appropriate to analyze facial factors since tensors make it possible to construct multilinear models using a multiple factor structure. We use higher-order tensors to model multiple factors of facial variations, but this provides some difficulty in use. First, it is difficult to decompose the multiple factors of a test image, especially when the factor parameters are unknown or are not in the training set. Second, for face recognition and face synthesis tasks, it is also difficult to construct reliable multilinear models which have more than two factors. In this paper, we propose a novel tensor factorization method to decompose mixing factors for unseen test images. We set up tensor factorization problem as a least squares problems with a quadratic equality constraint, and solve it using numerical optimization techniques. The novelty in our approach compared to previous work is that our tensor factorization method does not require any knowledge or assumption of test images; we can attain parameters of facial factors by our tensor factorization method. We have conducted several experiments to show the versatility of the method for both face recognition and synthesis tasks. Thus, we demonstrate the proposed method produces reliable results for trilinear models as well as bilinear models.

As tensors provide a natural representation for the higher-order relations, tensor factorization techniques such as Tucker decomposition and CANDECOMP/PARAFAC decomposition have been applied to many fields. Tucker decomposition has strong capacity of expression, but the time complexity is unpractical for the large-scale real problems. On the other hand, CANDECOMP/PARAFAC decomposition is linear in the feature dimensionality, but the assumption is so strong that it abandons some important information. Besides, both of TD and CP decompose a tensor into several factor matrices. However, the factor matrices are not natural for the representation of the higher-order relations. To overcome these problems, we propose a near linear tensor factorization approach, which decompose a tensor into factor tensors in order to model the higher-order relations, without loss of important information. In addition, to reduce the time complexity and the number of the parameters, we decompose each slice of the factor tensors into two smaller matrices. We conduct experiments on both synthetic datasets and real datasets. The experimental results on the synthetic datasets validate that our model has strong capacity of expression. The results on the real datasets show that our approach outperforms the state-of-the-art tensor factorization methods.

Analysis of spatio-temporal data is a common research topic that requires the interpolations of unknown locations and the predictions of feature observations by utilizing information about where and when the data were observed. One of the most difficult problems is to make predictions of unknown locations. Tensor factorization methods are popular in this field because of their capability of handling multiple types of spatio-temporal data, dealing with missing values, and providing computationally efficient parameter estimation procedures. However, unlike traditional approaches such as spatial autoregressive models, the existing tensor factorization methods have not tried to learn spatial autocorrelations. These methods employ previously inferred spatial dependencies, often resulting in poor performances on the problem of making interpolations and predictions of unknown locations. In this paper, we propose a new tensor factorization method that estimates low-rank latent factors by simultaneously learning the spatial and temporal autocorrelations. We introduce new spatial autoregressive regularizers based on existing spatial autoregressive models and provide an efficient estimation procedure. With experiments on publicly available traffic transporting data, we demonstrate that our proposed method significantly improves the predictive performances in our problems in comparison to the existing state-of-the-art spatio-temporal analysis methods.

This paper studies the data sparsity problem in multi-view learning. To solve data sparsity problem in multi-view ratings, we propose a generic architecture of deep transfer tensor factorization (DTTF) by integrating deep learning and cross-domain tensor factorization, where the side information is embedded to provide effective compensation for the tensor sparsity. Then we exhibit instantiation of our architecture by combining stacked denoising autoencoder (SDAE) and CANDE-COMPIPARAFAC (CP) tensor factorization in both source and target domains, where the side information of both users and items is tightly coupled with the sparse multi-view ratings and the latent factors are learned based on the joint optimization. We tightly couple the multi-view ratings and the side information to improve cross-domain tensor factorization based recommendations. Experimental results on real-world datasets demonstrate that our DTTF schemes outperform state-of-the-art methods on multi-view rating predictions.

The distributional similarity methods have proven to be a valuable tool for the induction of semantic similarity. Until now, most algorithms use two-way co-occurrence data to compute the meaning of words. Co-occurrence frequencies, however, need not be pairwise. One can easily imagine situations where it is desirable to investigate co-occurrence frequencies of three modes and beyond. This paper will investigate tensor factorization methods to build a model of three-way co-occurrences. The approach is applied to the problem of selectional preference induction, and automatically evaluated in a pseudo-disambiguation task. The results show that tensor factorization, and non-negative tensor factorization in particular, is a promising tool for Natural Language Processing (nlp).

Tensors are higher order generalizations of matrices to model multi-aspect data, e.g., a set of purchase records with the schema (user_id, product_id, timestamp, feedback). Tensor factorization is a powerful technique for generating a model from a tensor, just like matrix factorization generates a model from a matrix, but with higher accuracy and richer information as more attributes are available in a higher- order tensor than a matrix. The data model obtained by tensor factorization can be used for classification, recommendation, anomaly detection, and so on. Though having a broad range of applications, tensor factorization has not been popularly applied compared with matrix factorization that has been widely used in recommender systems, mainly due to the high computational cost and poor scalability of existing tensor factorization methods. Efficient and scalable tensor factorization is particularly challenging because real world tensor data are mostly sparse and massive. In this paper, we propose a novel distributed algorithm, called Lock-Free Tensor Factorization (LFTF), which significantly improves the efficiency and scalability of distributed tensor factorization by exploiting asynchronous execution in a re-formulated problem. Our experiments show that LFTF achieves much higher CPU and network throughput than existing methods, converges at least 17 times faster and scales to much larger datasets.

In order to achieve best neutron flux-mapping within nuclear research reactors, we investigate in this paper, the application of blind source separation algorithms to extract different independent components that form the fission chamber output signals. More specifically, we simulated the fission chamber using python-based of Fission Chambers (pyFC) code suite. The resulting simulated signals are considered observation vectors. They are then, processed via nonnegative matrix and tensor factorization (NMF&NTF) methods to isolate the independent components that form a neutron signal. Thus, in order to select the most efficient one to analyze our set of data, various algorithms have been tested and classified according to their performance index of separability (PI). The extracted independent components have been used to characterize neutron signal through spectrogram time-frequency representation. Obtained results will be illustrated and discussed.

With the development of Service-Oriented technologies, the amount of Web services grows rapidly. QoS-Aware Web service recommendation can help service users to design more efficient service-oriented systems. However, existing methods assume the QoS information for service users are all known and accurate, but in real case, there are always many missing QoS values in history records, which increase the difficulty of the missing QoS value prediction. By considering the user-service-time three dimension context information, we study a Temporal QoS-Aware Web Service Recommendation Framework which aims to recommend best candidates to service user's requirements and meanwhile improve the QoS prediction accuracy. We propose to envision such QoS value data as a tensor which is known as multi-way array, and formalize this problem as a tensor factorization model and propose a Tucker decomposition (TD)algorithm which is able to deal with the triadic relations of user-service-time model. However, one major challenge is that how to deal with the dynamic incoming service QoS value data streams. Thus, we introduce the incremental tensor factorization (ITF)method which is (a) scalable, (b) space efficient (it does not need to store the past data). Extensive experiments are conductedbasedc on our real-world QoS dataset collected on Planet-Lab, comprised of service invocation response-time values from 408 users on 5,473 Web services at 240 time periods. Comprehensive empirical studies demonstrate that our approach is faster and more accuracy than other approaches.

CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem especially for CP rank . In addition, existing approaches do not take into account uncertainty information of latent factors, as well as missing entries. To address these issues, we formulate CP factorization using a hierarchical probabilistic model and employ a fully Bayesian treatment by incorporating a sparsity-inducing prior over multiple latent factors and the appropriate hyperpriors over all hyperparameters, resulting in automatic rank determination. To learn the model, we develop an efficient deterministic Bayesian inference algorithm, which scales linearly with data size. Our method is characterized as a tuning parameter-free approach, which can effectively infer underlying multilinear factors with a low-rank constraint, while also providing predictive distributions over missing entries. Extensive simulations on synthetic data illustrate the intrinsic capability of our method to recover the ground-truth of CP rank and prevent the overfitting problem, even when a large amount of entries are missing. Moreover, the results from real-world applications, including image inpainting and facial image synthesis, demonstrate that our method outperforms state-of-the-art approaches for both tensor factorization and tensor completion in terms of predictive performance.

For embedded impedance spectroscopy, a suitable method for analyzing AC signals needs to be carefully chosen to overcome limited processing capability and memory availability. This paper compares various methods, including the fast Fourier transform (FFT), the FFT with barycenter correction, the FFT with windowing, the Goertzel filter, the discrete-time Fourier transform (DTFT), and sine fitting using linear or nonlinear least squares, and cross-correlation, for analyzing AC signals in terms of speed, memory requirements, amplitude measurement accuracy, and phase measurement accuracy. These methods are implemented in reference systems with and without hardware acceleration for validation. The investigation results show that the Goertzel algorithm has the best overall performance when hardware acceleration is excluded or in the case of memory constraints. In implementations with hardware acceleration, the FFT with barycentre correction stands out. The linear sine fitting method provides the most accurate amplitude and phase determinations at the expense of speed and memory requirements.

Regular expression matching is the state of the art in signature based intrusion detection systems. A regular expression matching algorithm used in intrusion detection systems is expected to process data at a speed linear in size of the incoming data and also to be able to run on network devices with limited memory. Traditional DFA and NFA based algorithms fail to meet either of these two requirements. The existing techniques try to either modify a DFA or a NFA, or combine both these to find a trade off between speed and memory requirements. The idea we propose in this paper is orthogonal to existing techniques. We propose a new approach to store a finite automaton in memory which otherwise is stored as a transition table. Our approach can be used by existing algorithms to further reduce the memory requirements with a minimal increase in the processing speed.

Tensor completion is the task of completing multi-aspect data represented as a tensor by accurately predicting missing entries in the tensor. It is mainly solved by tensor factorization methods, and among them, Tucker factorization has attracted considerable interests due to its powerful ability to learn latent factors and even their interactions. Although several Tucker methods have been developed to reduce the memory and computational complexity, the state-of-the-art method still 1) generates redundant computations and 2) cannot factorize a large tensor that exceeds the size of memory. This paper proposes FTcom, a fast and scalable Tucker factorization method for tensor completion. FTcom performs element-wise updates for factor matrices based on coordinate descent, and adopts a novel caching algorithm which stores frequently-required intermediate data. It also uses a tensor file for disk-based data processing and loads only a small part of the tensor at a time into the memory. Experimental results show that FTcom is much faster and more scalable compared to all other competitors. It significantly shortens the training time of Tucker factorization, especially on real-world tensors, and it can be executed on a billion-scale tensor which is bigger than the memory capacity within a single machine.

In recent years, anomaly detection in web applications has become increasingly important for security and reliability. For instance, detecting fraudulent transactions in e-commerce platforms or monitoring user behavior in real-time web dashboards requires immediate response and high accuracy. Traditional deep learning models are difficult to deploy in browsers due to CPU/GPU bottlenecks, limited memory, and stringent real-time latency requirements. Deploying deep learning models directly in the browser is challenging due to limited computational resources and memory constraints. To address these challenges, we propose DAPQ-AD, a Dynamic-Aware Pruning-Quantization compression algorithm optimized for WebAssembly execution. Unlike conventional static compression methods, DAPQ-AD not only dynamically adjusts pruning rates and quantization levels based on the characteristics of the input data, but it also adapts to varying input noise, preserves critical features for accurate anomaly detection, and reduces inference latency compared to static approaches. Extensive experiments demonstrate that our method significantly reduces model size and inference latency while maintaining high anomaly detection performance in browser environments. This approach enables lightweight, real-time anomaly detection directly on client-side web applications without compromising accuracy.

This survey is about Tensor Factorization methods for audio modeling, which focuses on probabilistic latent tensor factorization and generalized coupled tensor factorization by expectation maximization method while using several linear and nonlinear distance measure methods

This paper deals with algorithms for positive semidefinite matrix factorization (PSDMF). PSDMF is a recently-proposed extension of nonnegative matrix factorization with applications in combinatorial optimization, among others. In this paper, we focus on improving the local convergence of an alternating block gradient (ABC) method for PSDMF in a noise-free setting by replacing the quadratic objective function with the Poisson log-likelihood. This idea is based on truncated Wirtinger flow (TWF), a phase retrieval (PR) method that trims outliers in the gradient and thus regularizes it. Our motivation is a recent result linking PR with PSDMF. Our numerical experiments validate that the numerical benefits of TWF may carry over to PSDMF despite the more challenging setting, when initialized within its region of convergence. We then extend TWF from PR to affine rank minimization (ARM), and show that although the outliers are no longer an issue in the ARM setting, PSDMF with the new objective function may still achieves a smaller error for the same number of iterations. In a broader view, our results indicate that a proper choice of objective function may enhance convergence of matrix (or tensor) factorization methods.