Multidimensional Data Processing With Bayesian Inference via Structural Block Decomposition.

Knowledge graphs are structured representations of real world facts. However, they typically contain only a small subset of all possible facts. Link prediction is the task of inferring missing facts based on existing ones. Knowledge graph embedding, representing entities and relations in the knowledge graphs with high-dimensional vectors, has made significant progress in link prediction. The tensor decomposition models are an embedding family with good performance in link prediction. The previous tensor decomposition models do not consider the problem of attribute separation. These models mainly explore particular regularization to improve performance. No matter how sophisticated the design of tensor decomposition models is, the performance is theoretically under the basic tensor decomposition model. Moreover, the unnoticed task of attribute separation in the traditional models is just handed over to the training. However, the amount of parameters for this task is tremendous, and the model is prone to overfitting. We investigate the design approaching the theoretical performance of tensor decomposition models in this paper. The observation that measuring the rationality of specific triples means comparing the matching degree of the specific attributes associated with the relations is well-known. Therefore, the comparison of actual triples needs first to separate specific attribute dimensions, which is ignored by existing models. Inspired by this observation, we design a novel tensor ecomposition model based on Separating Attribute space for knowledge graph completion (SeAttE). The major novelty of this paper is that SeAttE is the first model among the tensor decomposition family to consider the attribute space separation task. Furthermore, SeAttE transforms the learning of too many parameters for the attribute space separation task into the structure’s design. This operation allows the model to focus on learning the semantic equivalence between relations, causing the performance to approach the theoretical limit. We also prove that RESCAL, DisMult and ComplEx are special cases of SeAttE in this paper. Furthermore, we classify existing tensor decomposition models for subsequent researchers. Experiments on the benchmark datasets show that SeAttE has achieved state-of-the-art among tensor decomposition models.

Hyperspectral image (HSI) denoising is a fundamental task in remote sensing image processing, which is helpful for HSI subsequent applications, such as unmixing and classification. Thanks to the powerful representation ability of untrained deep neural networks, deep image prior (DIP)-based methods achieve tremendous successes in image processing (e.g., denoising and inpainting). However, DIP-based methods neglect the tensor low-rank prior of the underlying HSI which will be beneficial to capturing the global structure of the underlying HSI. To address this issue, we propose a novel model for HSI denoising, which can simultaneously take respective advantages of the tensor low-rank prior and the deep spatial-spectral prior. The tensor low-rank prior leads to a better global structure and the deep spatial-spectral prior is complementary to preserve better local details. On the one hand, we adopt low-rank tensor ring (TR) decomposition to characterize the tensor low-rank prior and capture the global structure of the underlying HSI. On the other hand, we employ untrained deep neural networks to flexibly represent the deep spatial-spectral prior and capture the local details of the underlying HSI. To solve the proposed model, we develop an efficient alternating minimization algorithm. Experimental results on simulated and real data validate the advantages of the proposed model in HSI denoising. Compared with state-of-the-art HSI denoising methods, the proposed method preserves better local details and the global structure of the underlying HSI.

Recently, low-rank matrix recovery has been demonstrated to be an effective tool in hyperspectral images (HSIs) denoising. However, the previous low-rank matrix recovery method with a window of the fixed-shape cannot adaptively exploit spatial structure information and nonlocal similarity. In this paper, multiscale low-rank matrix recovery (MC-LRMR) is proposed to recover HSI corrupted by different kinds of noise. The proposed method contains three main steps. First, HSI is transformed by the principal component analysis (PCA) algorithm and multiscale superpixel segmentation is applied to the first principal component, to segment the HSI into non-overlapping homogeneous regions. Then, the mixed noises, including Gaussian noise, impulse noise, dead lines noise, stripes noise, are removed by the low-rank matrix recovery (LRMR) in a superpixel-by-superpixel manner. Finally, a fusion rule, i.e., average operator, is adopted to combine denoising results of various scales, to acquire a fused noise-free estimation. Experiments on simulated and real HSI data sets can demonstrate the effectiveness of the proposed method.

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure for the subsequent HSI applications. Unfortunately, though witnessing the development of deep learning in HSI denoising area, existing convolution-based methods face the trade-off between computational efficiency and capability to model non-local characteristics of HSI. In this paper, we propose a Spatial-Spectral Transformer (SST) to alleviate this problem. To fully explore intrinsic similarity characteristics in both spatial dimension and spectral dimension, we conduct non-local spatial self-attention and global spectral self-attention with Transformer architecture. The window-based spatial self-attention focuses on the spatial similarity beyond the neighboring region. While, the spectral self-attention exploits the long-range dependencies between highly correlative bands. Experimental results show that our proposed method outperforms the state-of-the-art HSI denoising methods in quantitative quality and visual results. The code is released at https://github.com/MyuLi/SST.

The present manuscript is intended to review few applications of tensor decomposition model in array signal processing. Tensor decomposition models like HOSVD, SVD and PARAFAC are useful in signal processing. In this paper we shall use higher order tensor decomposition in signal processing. Also, a novel orthogonal non-iterative tensor decomposition technique (SUSVD), which is scalable to arbitrary high dimensional tensor, has been applied in MIMO channel estimation. The SUSVD provides a tensor model with hierarchical tree structure between the factors in different dimensions. We shall use a new model known as PARATREE, which is related to PARAFAC tensor models. The PARAFAC and PARATREE both describe a tensor as a sum of rank-1 tensors, but PARATREE has several advantages over PARAFAC, when it is applied as a lower rank approximation technique. PARATREE is orthogonal, fast and reliable to compute, and the order of the decomposition can be adaptively adjusted. The low rank PARATREE approximation has been applied to measure noise suppression for tensor valued MIMO channel sounding measurements.

Fusing a low spatial resolution hyperspectral image (HSI) with a high spatial resolution multispectral image (MSI), aiming to produce a super-resolution hyperspectral image, has recently attracted increasing research interest. In this paper, a novel approach based on coupled non-negative tensor decomposition is proposed. The proposed method performs a tucker tensor factorization of a low resolution hyperspectral image and a high resolution multispectral image under the constraint of non-negative tensor decomposition (NTD). The conventional matrix factorization methods essentially lose spatio-spectral structure information when stacking the 3D data structure of a hyperspectral image into a matrix form. Moreover, the spectral, spatial, or their joint structural features have to be imposed from the outside as a constraint to well pose the matrix factorization problem. The proposed method has the advantage of preserving the spatio-spectral structure of hyperspectral images. In this paper, the NTD is directly imposed on the coupled tensors of the HSI and MSI. Hence, the intrinsic spatio-spectral structure of the HSI is represented without loss, and spatial and spectral information can be interdependently exploited. Furthermore, multilinear interactions of different modes of the HSIs can be exactly modeled with the core tensor of the Tucker tensor decomposition. The proposed method is straightforward and easy to implement. Unlike other state-of-the-art approaches, the complexity of the proposed approach is linear with the size of the HSI cube. Experiments on two well-known datasets give promising results when compared with some recent methods from the literature.

Recently, low-rank tensor recovery methods based on subspace representation have received increased attention in the field of hyperspectral image (HSI) denoising. Unfortunately, those methods usually analyze the prior structural information within different dimensions indiscriminately, ignoring the differences between modes, leaving substantial room for improvement. In this article, we first consider the low-rank properties in the subspace and prove that the structure correlation across the nonlocal self-similarity mode is much stronger than in the spatial sparsity and spectral correlation modes. On that basis, we introduce a new multidirectional low-rank regularization, in which each mode is assigned a different weight to characterize its contribution to estimating the tensor rank. After that, integrating the proposed regularization with the subspace-based tensor recovery framework, an optimization model for HSI mixed noise removal is developed. The proposed model can be addressed efficiently via the alternating minimization algorithm. Extensive experiments implemented with synthetic and real data demonstrate that the proposed method significantly outperforms other state-of-the-art HSI denoising methods, which clearly indicates the effectiveness of the proposed approach in HSI denoising.

Hyperspectral image denoising by total variation-regularized bilinear factorization

Recently, tensor sparsity modeling has achieved great success in the tensor completion (TC) problem. In real applications, the sparsity of a tensor can be rationally measured by low-rank tensor decomposition. However, existing methods either suffer from limited modeling power in estimating accurate rank or have difficulty in depicting hierarchical structure underlying such data ensembles. To address these issues, we propose a parametric tensor sparsity measure model, which encodes the sparsity for a general tensor by Laplacian scale mixture (LSM) modeling based on three-layer transform (TLT) for factor subspace prior with Tucker decomposition. Specifically, the sparsity of a tensor is first transformed into factor subspace, and then factor sparsity in the gradient domain is used to express the local similarity in within-mode. To further refine the sparsity, we adopt LSM by the transform learning scheme to self-adaptively depict deeper layer structured sparsity, in which the transformed sparse matrices in the sense of a statistical model can be modeled as the product of a Laplacian vector and a hidden positive scalar multiplier. We call the method as parametric tensor sparsity delivered by LSM-TLT. By a progressive transformation operator, we formulate the LSM-TLT model and use it to address the TC problem, and then the alternating direction method of multipliers-based optimization algorithm is designed to solve the problem. The experimental results on RGB images, hyperspectral images (HSIs), and videos demonstrate the proposed method outperforms state of the arts.

Sparse coding provides an excellent image prior for hyperspectral images (HSIs) denoising. However, on one hand, it is challenging to capture the structure within each sparse code for improving the reconstruction accuracy, on the other hand, the inconsistent recovery of the sparse codes corrupts the spectrum similarity in each homogeneous cluster of the HSI. To address these problems, we first propose a novel covariance matrix estimation based structured sparse coding method, where the sparse code matrix is modeled by a matrix normal distribution with a full covariance matrix. By estimating the covariance matrix with a latent variable based Bayesian framework, the data-dependent and noise-robust structure for each sparse code is learned from the noisy observation, with which the sparse codes are reconstructed accurately. Then, an intra-cluster filtering is employed to restore the spectrum similarity in each cluster. Experimental results demonstrate that the proposed method outperforms several state-of-the-art methods in HSIs denoising.

Probabilistic inference with noisy-threshold models based on a CP tensor decomposition

Bayesian tensor decomposition has been widely applied in channel parameter estimations, particularly in cases with the presence of interference. However, the types of interference are not considered in Bayesian tensor decomposition, making it difficult to accurately estimate the interference parameters. In this paper, we present a robust tensor variational method using a CANDECOMP/PARAFAC (CP)-based additive interference model for multiple input-multiple output (MIMO) with orthogonal frequency division multiplexing (OFDM) systems. A more realistic interference model compared to traditional colored noise is considered in terms of co-channel interference (CCI) and front-end interference (FEI). In contrast to conventional algorithms that filter out interference, the proposed method jointly estimates the channel and interference parameters in the time-frequency domain. Simulation results validate the correctness of the proposed method by the evidence lower bound (ELBO) and reveal the fact that the proposed method outperforms traditional information-theoretic methods, tensor decomposition models, and robust model based on CP (RCP) in terms of estimation accuracy. Further, the interference parameter estimation technique has profound implications for anti-interference applications and dynamic spectrum allocation.

In this paper, we propose a novel algorithm for underdetermined blind identification problem in blind source separation. The proposed algorithm is based on the charrelation matrix of observed signals. The charrelation matrix not only can be considered as a generalized second-order covariance matrix, but also incorporates higher-order information. It is significant for blind separation problem based on statistic characteristics to extract statistical information. The problem of underdetermined blind identification is converted as a tensor decomposition model. The mixing matrix can be estimated from the rank-1 terms of the tensor decomposition. Theoretical analysis and simulation results illustrate the proposed algorithm performs competitive estimation performance compared to the underdetermined blind identification algorithm based on second-order covariance and fourth-order cumulant.

Hyperspectral image (HSI) denoising is an effective image processing technology to improve the quality of HSI. By exploiting the spectral-spatial characteristics of HSI, this paper proposes a HSI denoising method based on the Graph-structured Low Rank model and Non-Local constraint (GLRNL). Graph-structured low rank model can reveal the spectral similarity in HSI and preserve the distribution characteristics of spectral bands. The graph structure is used to ensure that the denoised HSI has similar spectral characteristics as the real HSI, and reduce the spectral distortions. Moreover, the non-local constraint in GLRNL adopts the spatial-spectral self-similarity and provides an effective side prior information for low rank calculation. Comparing with some traditional low rank and sparse model based HSI denoising methods, the proposed method achieves the competitive results on different simulated and real HSI data with various types of noise.

Dear Editor, This letter puts forward a novel scalable temporal dimension preserved tensor completion model based on orthogonal initialization for missing traffic data (MTD) imputation. The MTD imputation acts directly on accessing the traffic state, and affects the traffic management. However, it still faces the following challenges: 1) The MTD imputation is usually formulated as matrix completion or tensor completion, which ignores the information across different dimensions; 2) Most of the existing models cannot generalize to traffic datasets of different scales or different missing rates; and 3) The MTD imputation models based on Gaussian random initialization easily leads to gradient explosion or vanishing, so that the training accuracy is not effectively improved. Inspired by these findings, the proposed scalable temporal dimension preserved tensor completion (ST-DPTC) model creatively establishes the following three-fold ideas: a) Incorporating the dimension preserved tensor completion (DPTC) to extract more distinctive traffic structure changes from the low-rank latent factor tensors; b) Adopting a scalable temporal (ST) regularization with first-order difference and second-order difference operators to adapt to different scales of traffic data; and c) Embedding ST regularization into DPTC with orthogonal initialization to perform low-rank latent factor tensor extraction and MTD imputation. Results on real-world traffic datasets with different scales show that our proposed model exceeds the state-of-the-art models in terms of the imputation accuracy.