R-Term Representation

High-order possibilistic c-means algorithms based on tensor decompositions for big data in IoT

The Canonical Polyadic Decomposition (CPD) is a fundamental tensor decomposition which has widespread use in signal processing due to its ability to extract component information. A popular for algorithm CPD is the generalized eigenvalue decomposition (GEVD) which is based on the generalized eigenvectors of a subpencil of a tensor. GEVD plays an important role in applications as it provides strong algebraic initializations for optimization routines for CPD computation. In fact, using GEVD initializations can improve final accuracy and reduce computation time. However, despite GEVD’s success, the algorithm underperforms in some settings and exhibits pencil-based instability.We present a recursive generalized eigenspace decomposition (GESD) for CPD computation. Rather using one sub-pencil, GESD combines generalized eigenspace information from many subpencils to compute a CPD. GESD is more accurate than GEVD and thereby improves the reliability of the components extracted by CPD. We also give a Cramér-Rao based analysis for the accuracy of GESD.

The tensor rank decomposition, or canonical polyadic decomposition, is the decomposition of a tensor into a sum of rank-1 tensors. The condition number of the tensor rank decomposition measures the sensitivity of the rank-1 summands with respect to structured perturbations. Those are perturbations preserving the rank of the tensor that is decomposed. On the other hand, the angular condition number measures the perturbations of the rank-1 summands up to scaling. We show for random rank-2 tensors that the expected value of the condition number is infinite for a wide range of choices of the density. Under a mild additional assumption, we show that the same is true for most higher ranks rge 3 as well. In fact, as the dimensions of the tensor tend to infinity, asymptotically all ranks are covered by our analysis. On the contrary, we show that rank-2 tensors have finite expected angular condition number. Based on numerical experiments, we conjecture that this could also be true for higher ranks. Our results underline the high computational complexity of computing tensor rank decompositions. We discuss consequences of our results for algorithm design and for testing algorithms computing tensor rank decompositions.

Low-rank matrix/tensor factorizations play a significant role in science and engineering. An important example is the canonical polyadic decomposition (CPD). There is also a growing interest in multi-set extensions of low-rank matrix/tensor factorizations in which the associated factor matrices are partially shared. In this paper we propose a more unified framework for multi-set matrix/tensor factorizations. In particular, we propose a multi-set extension of bilinear factorizations subject to monomial equality constraints to the case of shared and unshared factors. The presented framework encompasses (generalized) canonical correlation analysis (CCA) and (coupled) CPD models as special cases. CPD, CCA and hybrid models between them feature interesting uniqueness properties. We derive uniqueness conditions for CCA and multi-set low-rank factorization with partially shared entities. Computationally, we reduce multi-set low-rank factorizations with shared and unshared components into a special CPD problem, which can be solved via a matrix eigenvalue decomposition. Finally, numerical experiments demonstrate the importance of taking the coupling between multi-set low-rank factorizations into account in the actual computation.

This paper addresses the estimation of effective connectivity in epileptic brain networks. To this end, a constrained Canonical Polyadic Decomposition (CPD) of a Partial Directed Coherence tensor issued from a modeling of the observed data is investigated. The efficiency of the proposed approach is confirmed in the context of epileptic intracerebral electroencephalographic (iEEG) signals.

We propose an approach to represent the second-quantized electronic Hamiltonian in a compact sum-of-products (SOP) form. The approach is based on the canonical polyadic decomposition of the original Hamiltonian projected onto the sub-Fock spaces formed by groups of spin-orbitals. The algorithm for obtaining the canonical polyadic form starts from an exact sum-of-products, which is then optimally compactified using an alternating least squares procedure. We discuss the relation of this specific SOP with related forms, namely the Tucker format and the matrix product operator often used in conjunction with matrix product states. We benchmark the method on the electronic dynamics of an excited water molecule, trans-polyenes, and the charge migration in glycine upon inner-valence ionization. The quantum dynamics are performed with the multilayer multiconfiguration time-dependent Hartree method in second quantization representation. Other methods based on tree-tensor Ansätze may profit from this general approach.

In this paper, we present a general tensor-based nonlinear classifier, the Rank-R Feedforward Neural Network (FNN). In the proposed model, which is an extension of the Rank-1 FNN classifier, the network weights are constrained to satisfy a rank-R Canonical Polyadic Decomposition. By allowing a rank-R, instead of a rank-1, Canonical Polyadic Decomposition of the weights, the learning capacity of the model can be increased, which contributes to avoiding underfitting problems. The effectiveness of the proposed model is scrutinized on a hyperspectral image classification experimental setting, since hyperspectral data can naturally be represented as tensor objects. Performance evaluation results indicate that the proposed model outperforms other state-of-the-art models, including deep learning ones, especially in cases where the number of available training samples is small.

Traditional convolutional neural networks CNNs architecture has limitations such as over-parameterization and a large computational demand. One effective approach to address these issues is to replace convolutional kernels with their low-rank tensor approximations. Among the various methods available, Canonical Polyadic CP tensor decomposition stands out as particularly promising. However, employing CP decomposition for CNNs compression presents two major challenges. First, numerical optimization algorithms used to fit convolutional tensors can cause rank-one tensors to cancel each other out, leading to instability and complicating the fine-tuning of the resulting model. Second, determining the appropriate rank for CP decomposition is inherently complex. To overcome these challenges, we propose RanpCode, a novel compression method based on CP decomposition. This method incorporates specially designed numerical fitting techniques to ensure complete CP decomposition and address instability issues. Furthermore, it employs a rank pruning scheme to automatically determine the optimal rank for each layer, with the rank globally optimized and adjusted according to the desired compression rate. Our evaluations on popular CNNs architectures for image classification demonstrate that RanpCode achieves higher compression rates while maintaining superior accuracy.

The relation between electrophysiology and BOLD‐fMRI requires further elucidation. One approach for studying this relation is to find time‐frequency features from electrophysiology that explain the variance of BOLD time‐series. Convolution of these features with a canonical hemodynamic response function (HRF) is often required to model neurovascular coupling mechanisms and thus account for time shifts between electrophysiological and BOLD‐fMRI data. We propose a framework for extracting the spatial distribution of these time‐frequency features while also estimating more flexible, region‐specific HRFs. The core component of this method is the decomposition of a tensor containing impulse response functions using the Canonical Polyadic Decomposition. The outputs of this decomposition provide insight into the relation between electrophysiology and BOLD‐fMRI and can be used to construct estimates of BOLD time‐series. We demonstrated the performance of this method on simulated data while also examining the effects of simulated measurement noise and physiological confounds. Afterwards, we validated our method on publicly available task‐based and resting‐state EEG‐fMRI data. We adjusted our method to accommodate the multisubject nature of these datasets, enabling the investigation of inter‐subject variability with regards to EEG‐to‐BOLD neurovascular coupling mechanisms. We thus also demonstrate how EEG features for modelling the BOLD signal differ across subjects.

A new algorithm to estimate the direction of arrival (DOA) and polarization parameters of signals impinging on an array with electromagnetic (EM) vector-sensors is presented by exploiting the canonical polyadic decomposition (CPD) of tensors. In addition to spatial and temporal diversities, further information from the polarization domain is considered and used in this paper. Estimation errors of these parameters are evaluated by the Cramér-Rao lower bound (CRB) benchmark, in the presence of additive white Gaussian noise (AWGN). The superiority of the proposed algorithm is shown by comparing with the derivative algorithms of MUSIC and ESPRIT. In the proposed algorithm, the parameters can be estimated by virtue of the diversities of the spatial and polarization belonging to the factor matrices, rather than the conventional subspace which is the foundation of MUSIC and ESPRIT. Additionally, the classical CPD algorithm based on Alternating Least Squares (ALS) is introduced to verify the efficacy of the proposed CPD algorithm. Results demonstrate that when the number of snapshots is greater than 50, the proposed algorithm requires a smaller number of snapshots to achieve a high level of performance, compared against the subspace-based algorithms and the ALS-based algorithm. Furthermore, in the matter of the array with a small number of sensors, the discovered advantage concerning the Root Mean Square Error (RMSE) in estimating the DOA and the polarization state of the signal is noteworthy.

Functional near-infrared spectroscopy (fNIRS) provides an effective tool in neuroscience studies of cognition in infants. fNIRS signals are normally processed by applying ANOVA analysis on the grand average of the hemodynamic responses to investigate the cognitive-related differences between experimental groups. However, this averaging approach does not account for any differences in the temporal patterns of the responses. Therefore, we propose a new approach based on a combination of tensor decomposition and ANOVA. First, a four-way tensor of the hemodynamic responses is arranged as time × frequency × channel× subject and decomposed using Canonical Polyadic Decomposition (CPD). Next, ANOVA is applied to identify significant patterns between subjects. Instead of averaging, the CPD can capture the distinct patterns between groups in all the dimensions. We used fNIRS dataset of 70 infants who participated in an experiment to investigate cortical activation to an agent (i.e., mechanical claws vs. human hands) with different events (i.e., function and non-function). In the comparison with the traditional ANOVA, CPD+ANOVA identified the same significance factors. However, CPD+ANOVA discovered new information on the temporal and spatial patterns indicating a longer interval hemodynamic responses, which was missed using the traditional ANOVA. This new analysis of hemodynamic responses as captured using fNIRS will improve neuroscience and cognitive studies.

In this work, we consider the problem of factoring a third-order tensor into multilinear rank -($L_{r}, L_{r}$, 1) terms. This model, referred to as the rank-($L_{r}, L_{r}$, 1) block-term decomposition (BTD), finds many applications in signal processing, especially blind separation of smooth sources and unmixing spectral-spatial data (e.g., hyper-spectral image). On the other hand, finding latent factors of rank-($L_{r}, L_{r}$, 1) BTD poses a very challenging optimization problem. Some computational tools designed for canonical polyadic decomposition (CPD) (e.g., alternating least squares (ALS) and Levenberg-Marquardt (LM) based algorithms) can be modified to handle rank-($L_{r}, L_{r}$, 1) BTD. Nonetheless, these methods essentially treat rank-($L_{r}, L_{r}$, 1) BTD as a special CPD problem. This raises a number of challenges, since rank -($L_{r}, L_{r}$, 1) BTD can be viewed as a CPD problem with rank-deficient latent factors and high CP rank–and these are known as hard cases for CPD algorithms. In this work, we reformulate the rank-($L_{r}, L_{r}$, 1) BTD problem as a matrix rank-constrained matrix factorization problem. We propose a simple algorithm that combines alternating optimization and projected gradient. This way, the per-iteration complexity is much smaller than those of the ALS and LM-based algorithms. We also show that the algorithm converges to a stationary point of the problem of interest at a sublinear rate–although nonconvex constraints are involved. Numerical experiments are conducted to showcase the effectiveness of our algorithm.

The canonical polyadic decomposition (CPD) can be used to extract meaningful components from a tensor. Most existing optimization methods for fitting the CPD use as cost function the least-squares distance between the tensor and its CPD. While the minimum of this cost function coincides with the maximum likelihood estimator for data with additive i.i.d. Gaussian distributed noise, for other noise distributions, better-suited cost functions exist. For such cost functions, first-order, gradient-based optimization methods have been proposed. However, (approximate) second-order methods, which additionally use information from the Hessian of the cost function to achieve faster convergence, are still largely unexplored. In this paper, we generalize the Gauss–Newton nonlinear least-squares algorithm to twice differentiable entry-wise cost functions. The low-rank structure of the problem is exploited to keep the computational cost low. As a special case, $\beta$ -divergence cost functions are examined. We show that quadratic convergence can be obtained close to the solution with a reasonable extra cost in memory and computation time, making the proposed method particularly useful when high accuracy of the decomposition is desired.

Learning a function from input and output data pairs is one of the most fundamental tasks in machine learning. In this work, we propose a generalization of the Canonical Polyadic Decomposition (CPD) from tensors to multivariate functions of continuous variables, and show how it can be applied to supervised learning. We approximate a compactly supported multivariate function using a tensor of truncated multidimensional Fourier series coefficients and propose a hidden tensor factorization formulation for learning a low-rank CPD model of the Fourier coefficients tensor. In contrast to prior work, our method is quite general as it can model any compactly supported multivariate function that can be well-approximated by a finite multidimensional Fourier series, and under certain conditions it guarantees that the unknown function is uniquely characterized by the given input-output data. Furthermore, our model naturally allows stochastic gradient updates allowing it to scale to larger datasets. We develop two optimization algorithms and demonstrate promising results on synthetic and real multivariate regression tasks.

Tensor decomposition of EEG signals: A brief review