Bayesian deep matrix factorization network for multiple images denoising

Hyperspectral image (HSI) denoising is a fundamental task in remote sensing image processing, which is helpful for HSI subsequent applications, such as unmixing and classification. Thanks to the powerful representation ability of untrained deep neural networks, deep image prior (DIP)-based methods achieve tremendous successes in image processing (e.g., denoising and inpainting). However, DIP-based methods neglect the tensor low-rank prior of the underlying HSI which will be beneficial to capturing the global structure of the underlying HSI. To address this issue, we propose a novel model for HSI denoising, which can simultaneously take respective advantages of the tensor low-rank prior and the deep spatial-spectral prior. The tensor low-rank prior leads to a better global structure and the deep spatial-spectral prior is complementary to preserve better local details. On the one hand, we adopt low-rank tensor ring (TR) decomposition to characterize the tensor low-rank prior and capture the global structure of the underlying HSI. On the other hand, we employ untrained deep neural networks to flexibly represent the deep spatial-spectral prior and capture the local details of the underlying HSI. To solve the proposed model, we develop an efficient alternating minimization algorithm. Experimental results on simulated and real data validate the advantages of the proposed model in HSI denoising. Compared with state-of-the-art HSI denoising methods, the proposed method preserves better local details and the global structure of the underlying HSI.

We study the eigenvalue distribution of random matrices pertinent to the analysis of deep neural networks. The matrices resemble the product of the sample covariance matrices, however, an important difference is that the analog of the population covariance matrix is now a function of random data matrices (synaptic weight matrices in the deep neural network terminology). The problem has been treated in recent work [J. Pennington, S. Schoenholz and S. Ganguli, The emergence of spectral universality in deep networks, Proc. Mach. Learn. Res. 84 (2018) 1924–1932, arXiv:1802.09979] by using the techniques of free probability theory. Since, however, free probability theory deals with population covariance matrices which are independent of the data matrices, its applicability in this case has to be justified. The justification has been given in [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] for Gaussian data matrices with independent entries, a standard analytical model of free probability, by using a version of the techniques of random matrix theory. In this paper, we use another version of the techniques to extend the results of [L. Pastur, On random matrices arising in deep neural networks: Gaussian case, Pure Appl. Funct. Anal. (2020), in press, arXiv:2001.06188] to the case where the entries of the data matrices are just independent identically distributed random variables with zero mean and finite fourth moment. This, in particular, justifies the mean field approximation in the infinite width limit for the deep untrained neural networks and the property of the macroscopic universality of random matrix theory in this case.

Recently, low-rank tensor recovery methods based on subspace representation have received increased attention in the field of hyperspectral image (HSI) denoising. Unfortunately, those methods usually analyze the prior structural information within different dimensions indiscriminately, ignoring the differences between modes, leaving substantial room for improvement. In this article, we first consider the low-rank properties in the subspace and prove that the structure correlation across the nonlocal self-similarity mode is much stronger than in the spatial sparsity and spectral correlation modes. On that basis, we introduce a new multidirectional low-rank regularization, in which each mode is assigned a different weight to characterize its contribution to estimating the tensor rank. After that, integrating the proposed regularization with the subspace-based tensor recovery framework, an optimization model for HSI mixed noise removal is developed. The proposed model can be addressed efficiently via the alternating minimization algorithm. Extensive experiments implemented with synthetic and real data demonstrate that the proposed method significantly outperforms other state-of-the-art HSI denoising methods, which clearly indicates the effectiveness of the proposed approach in HSI denoising.

Hyperspectral image denoising by total variation-regularized bilinear factorization

Recently, low-rank matrix recovery has been demonstrated to be an effective tool in hyperspectral images (HSIs) denoising. However, the previous low-rank matrix recovery method with a window of the fixed-shape cannot adaptively exploit spatial structure information and nonlocal similarity. In this paper, multiscale low-rank matrix recovery (MC-LRMR) is proposed to recover HSI corrupted by different kinds of noise. The proposed method contains three main steps. First, HSI is transformed by the principal component analysis (PCA) algorithm and multiscale superpixel segmentation is applied to the first principal component, to segment the HSI into non-overlapping homogeneous regions. Then, the mixed noises, including Gaussian noise, impulse noise, dead lines noise, stripes noise, are removed by the low-rank matrix recovery (LRMR) in a superpixel-by-superpixel manner. Finally, a fusion rule, i.e., average operator, is adopted to combine denoising results of various scales, to acquire a fused noise-free estimation. Experiments on simulated and real HSI data sets can demonstrate the effectiveness of the proposed method.

Recently, data-driven algorithms such as deep neural networks have attracted a lot of attention and have become a popular area of research and development. This interest is driven by several factors, such as recent advances in processing power (cheap and powerful hardware), the availability of large datasets (big data), and several small but important algorithmic advances (e.g., convolutional layers). Nowadays, deep neural networks are the state-of-the-art for several computer vision tasks, such as the ones that require high-level understanding of image semantics, e.g., image classification, object segmentation, saliency detection, and also in low level image processing tasks, such as image denoising, inpainting, and super-resolution, among others. Deep learning can handle large volumes of video data by making use of its powerful non-linear mapping and extracting high-level features with very deep networks. Incorporating deep learning into the IoVT can provide radical innovations in video sensing, coding, enhancing, understanding, and evaluation areas making it easier to handle the enormous growth in video data compared to traditional methods.

Hyperspectral image (HSI) denoising is an effective image processing technology to improve the quality of HSI. By exploiting the spectral-spatial characteristics of HSI, this paper proposes a HSI denoising method based on the Graph-structured Low Rank model and Non-Local constraint (GLRNL). Graph-structured low rank model can reveal the spectral similarity in HSI and preserve the distribution characteristics of spectral bands. The graph structure is used to ensure that the denoised HSI has similar spectral characteristics as the real HSI, and reduce the spectral distortions. Moreover, the non-local constraint in GLRNL adopts the spatial-spectral self-similarity and provides an effective side prior information for low rank calculation. Comparing with some traditional low rank and sparse model based HSI denoising methods, the proposed method achieves the competitive results on different simulated and real HSI data with various types of noise.

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure for the subsequent HSI applications. Unfortunately, though witnessing the development of deep learning in HSI denoising area, existing convolution-based methods face the trade-off between computational efficiency and capability to model non-local characteristics of HSI. In this paper, we propose a Spatial-Spectral Transformer (SST) to alleviate this problem. To fully explore intrinsic similarity characteristics in both spatial dimension and spectral dimension, we conduct non-local spatial self-attention and global spectral self-attention with Transformer architecture. The window-based spatial self-attention focuses on the spatial similarity beyond the neighboring region. While, the spectral self-attention exploits the long-range dependencies between highly correlative bands. Experimental results show that our proposed method outperforms the state-of-the-art HSI denoising methods in quantitative quality and visual results. The code is released at https://github.com/MyuLi/SST.

Many computer vision problems can be posed as learning a low-dimensional subspace from high dimensional data. The low rank matrix factorization (LRMF) represents a commonly utilized subspace learning strategy. Most of the current LRMF techniques are constructed on the optimization problem using L_1 norm and L_2 norm, which mainly deal with Laplacian and Gaussian noise, respectively. To make LRMF capable of adapting more complex noise, this paper proposes a new LRMF model by assuming noise as Mixture of Exponential Power (MoEP) distributions and proposes a penalized MoEP model by combining the penalized likelihood method with MoEP distributions. Such setting facilitates the learned LRMF model capable of automatically fitting the real noise through MoEP distributions. Each component in this mixture is adapted from a series of preliminary super-or sub-Gaussian candidates. An Expectation Maximization (EM) algorithm is also designed to infer the parameters involved in the proposed PMoEP model. The advantage of our method is demonstrated by extensive experiments on synthetic data, face modeling and hyperspectral image restoration.

Sparse coding provides an excellent image prior for hyperspectral images (HSIs) denoising. However, on one hand, it is challenging to capture the structure within each sparse code for improving the reconstruction accuracy, on the other hand, the inconsistent recovery of the sparse codes corrupts the spectrum similarity in each homogeneous cluster of the HSI. To address these problems, we first propose a novel covariance matrix estimation based structured sparse coding method, where the sparse code matrix is modeled by a matrix normal distribution with a full covariance matrix. By estimating the covariance matrix with a latent variable based Bayesian framework, the data-dependent and noise-robust structure for each sparse code is learned from the noisy observation, with which the sparse codes are reconstructed accurately. Then, an intra-cluster filtering is employed to restore the spectrum similarity in each cluster. Experimental results demonstrate that the proposed method outperforms several state-of-the-art methods in HSIs denoising.

Hyperspectral images (HSIs) frequently experience various types of noise due to atmospheric interference and sensor instability, which impairs the efficiency of subsequent operations. Consequently, HSI denoising has emerged as a crucial component of HSI preprocessing. Conventional approaches often target a single kind of noise and eliminate it repeatedly, which has disadvantages including inefficiency when handling heterogeneous noise. Lately, models based on deep neural networks have shown encouraging results in the general image denoising domain. This study, which aims to overcome shortcomings in previous techniques, provides a novel denoising methodology by leveraging the effectiveness of the SqueezeNet model. For a thorough assessment, the evaluation framework includes four main indicators: PSNR, SSIM, SAM, and ERGAS. The evaluation is based on real-world hyperspectral images from the [Harvard Hyperspectral Dataset], which cover a variety of scenarios and illumination circumstances. Fire blocks are used by the SqueezeNet-based denoising model to optimize feature extraction with fewer parameters.Benchmarks for comparison include deep learning technique QRNN3D and classical techniques like ITSReg and BM4D.In order to avoid convergence to suboptimal local minima and to speed up and stabilize the learning process, this work presents an incremental training policy. The suggested SqueezeNet-based HSI denoising model performs exceptionally well, attaining competitive results in terms of PSNR of 34.15, SSIM of 0.92, and SAM of 4.56 in addition to impressive ERGAS of 20.47. This study offers an effective denoising solution for hyperspectral images by addressing shortcomings in current techniques, showcasing improvements in efficiency and accuracy.

Extracting and inserting knowledge into stacked denoising auto-encoders

Purpose: The aim of the article is to develop an algorithm for classifying cracks in the analyzed images using modern methods of deep machine learning and transfer learning based on pretrained convolutional neural network - Inception-ResNet-v2. Design/Methodology/Approach: Transfer learning based on the pretrained convolutional neural network was used to categorize the images. The fully conected layer of the Inception-ResNet-v2 network has been modified. The last layer was trained using a two-class (binary) linear SVM (Support Vector Machine). In total, 20,000 training cases (images) were used to train the fully connected layer within transfer learning process. The research analyzed the possibility of using the deep neural networks for quick and fully automatic identification of cracks / defects on the surface of analyzed parts. Findings: The results indicate that pretrained convolutional neural network using SVM to train a fully connected layer is a very effective solution for visual crack / fault detection. In the analyzed model, a positive classification was obtained at the level of 99.89%. Practical Implications: The model presented in the article can be used in quality control carried out by process monitoring. An effective model for identifying defective parts can be used in both logistics and production processes. Originality/Value: A novelty is the use of a freely available, deep neural network trained to classify 1000 categories of various images for binary categorization of faults (cracks). The algorithm was adjusted by replacing the primary, 1000-output fully connected layer in the Inception-ResNet-v2 network with a binary layer (2 categories). The fully connected layer has been trained using the classification version of the popular SVM learner, but thanks to the combination of this layer with the sophisticated fearure extraction ability of the pre-trained Inception-ResNet-v2 deep network, the resulting predictive model enables the classification of defects with a very high level of accuracy.

Due to the favorable denoising performance of the discriminative learning model, a large number of discriminative learning models have been designed to remove noise. However, most discriminative learning models can only deal with the noise that already exists in the training data. In this paper, a multi-scale trainable deep residual convolutional neural network (DCMSNet) based on dilated convolution is proposed. DCMSNet consists of a chain of dilated convolution layers, convolution layers, normalized multi-scale convolution blocks (BNMCBlock), multi-scale convolution blocks (MCBlock) and multi-scale dropout convolution blocks (MCDBlock). The use of dilated convolution can avoid over-parameterization problems caused by too deep networks. In order to capture more feature information during feature extraction and image reconstruction, we designed novel BNMCBlock and MCBlock. In order to reduce the degree of coupling between image features and improve the generalization ability of the network, we also designed a MCDBlock. Meanwhile, residual learning, batch normalization and dropout are utilized to speed up the training process and boost the denoising performance. Unlike existing denoising models, DCMSNet is able to remove different degrees of noise. Compared with state-of-the-art image denoising methods, DCMSNet has achieved relatively competitive results.

Visual tracking is an open issue in computer vision. Designing an robust tracking algorithm is a challenging problem due to complicated appearance variations, such as partial occlusion, illumination variations, fast motion and out-of-plane rotation. Some tracking algorithms use a set of templates to represent a target candidate. Such representations are not robust to drastic appearance variations. In order to alleviate the influence of appearance variations and outliers, we propose a novel tracking algorithm based on a robust Bayesian matrix factorization, where sample data is decomposed into a basis matrix and a coefficient matrix. The sample data is consist of previous tracking results. A column vector is stacked by a tracking result in a frame. A target candidate is represented by a linear combinations of the columns in the basis matrix. The matrix factorization can capture the correlation between different features in sample data. Also the appearance variations in tracking processing can be learnt in matrix factorization. Experimental results demonstrate the appearance models is effective and the proposed tracker in a particle filter tracking framework is robust to outliers and complicated appearance variations.