Image colorization based on ADMM with fast singular value thresholding by Chebyshev polynomial approximation

We are interested in solving the following minimization problem Dτ (Y ) := arg min X∈Rm×n 1 2 ∥Y −X∥F + τ∥X∥∗, where Y ∈ Rm×n is a given matrix, and ∥ ⋅ ∥F is the Frobenius norm and ∥ ⋅ ∥∗ the nuclear norm. This problem serves as a basic subroutine in many popular numerical schemes for nuclear norm minimization problems, which arise from low rank matrix recovery such as matrix completion. As Dτ (Y ) has an explicit expression which shrinks the singular values of Y and keeps the singular vectors, Dτ is referred to singular value thresholding (SVT) operator in the literature. Conventional approaches for Dτ (Y ) first find the singular value decomposition (SVD) of Y and then shrink the singular values. However, such approaches are time consuming under some circumstances, especially when the rank of Dτ (Y ) is not low compared to the matrix dimension or is completely unpredictable. In this paper, we propose a fast algorithm for directly computing Dτ (Y ) without using SVDs. Numerical experiments show that the proposed algorithm is much more efficient than the approach using the full SVD.

In recent years, sparse representation theory and low-rank approximation model have been widely used in signal and image processing fields. In the study of natural image denoising, non-local similarity method can enhance the correlation of grouped image blocks, and a low rank prior is used to devise a bilateral sparse representation of the image matrix, consequently achieving the purpose of removing additive white Gaussian noise. However, problems such as how to select thresholds after a singular value shrinkage, and how to eliminate image artifacts in removing noise especially with high levels, have not been resolved. In this paper, a low rank adaptive singular value thresholding (ASVT) denoising algorithm based on singular value decomposition(SVD) is proposed. Our method uses random matrix and asymptotic matrix reconstruction theory to scientifically select the threshold of singular value thresholding. At the same time, a dual-domain filtering method is used to process the visual artifacts after image denoising by ASVT, which we call collaborative singular value thresholding (CSVT) algorithm. The experimental results show that the proposed algorithm has a certain improvement in subjective visual effects, and objective quantitative indicators compared with some related advanced denoising algorithms.

To develop a novel regularization method for GRAPPA by which the regularization parameters can be optimally and adaptively chosen. In the fit procedures in GRAPPA, the discrepancy principle, which chooses the regularization parameter based on a priori information about the noise level in the autocalibrating signals (ACS), is used with the truncated singular value decomposition (TSVD) regularization and the Tikhonov regularization, and its performance is compared with the singular value (SV) threshold method and the L-curve method, respectively by axial and sagittal head imaging experiments. In both axial and sagittal reconstructions, normal GRAPPA reconstruction results exhibit a relatively high level of noise. With discrepancy-based choices of parameters, regularization can improve the signal-to-noise ratio (SNR) with only a very modest increase in aliasing artifacts. The L-curve method in all of the reconstructions leads to overregularization, which causes severe residual aliasing artifacts. The 10% SV threshold method yields good overall image quality in the axial case, but in the sagittal case it also leads to an obvious increase in aliasing artifacts. Neither a fixed SV threshold nor the L-curve are robust means of choosing the appropriate parameters in GRAPPA reconstruction. However, with the discrepancy-based parameter-choice strategy, adaptively regularized GRAPPA can be used to automatically choose nearly optimal parameters for reconstruction and achieve an excellent compromise between SNR and artifacts.

This paper describes a novel application of singular value decomposition (SVD) of subsets of the phase-space trajectory for calculation of the attractor dimension of a small data set. A certain number of local centres (M) are chosen randomly on the attractor and an adequate number of nearest neighbours (q=50) are ordered around each centre. The local intrinsic dimension of a local centre is determined by the number of significant singular values and the attractor dimension (D 2) by the average of the local intrinsic dimensions of the local centres. The SVD method has been evaluated for model data and EEG. The results indicate that the SVD method is a reliable approach for estimation of attractor dimension at moderate signal to noise ratios. The paper emphasises the importance of SVD approach to EEG analysis.

Ultrasound (US) image quality often suffers from random and speckle noise. Especially in the case of difficult patients, increased presence of random noise leads to compromised signal to noise ratio (SNR). The proposed technique removes random or speckle noise by promoting the low rank component of temporal US image sequences comprising of RF-data or envelope-detected data, respectively. This approach leverages the fact that due to tissue connectedness, tissue motion tends to exhibit limited temporal profiles. An approach to approximate the low rank component of the image sequence involves truncating its singular value decomposition. In the proposed algorithm, this is performed in a block-wise manner. Additionally, an adaptive singular value threshold is proposed based on Stein's unbiased risk estimator. This paper presents applications of this methodology on sequences of RF-data and envelope-detected data.

With the growing interest in convex and nonconvex low-rank matrix learning problems, the widely used singular value thresholding (SVT) operators associated with rank relaxation functions often face higher computational complexity, particularly for large-scale data matrices. To improve the efficacy of low-rank subspace clustering and overcome the issue of high computational complexity, this work proposes an efficient and effective method that avoids the need for singular value decomposition (SVD) computations in the iteration scheme. This can be achieved through the use of a computationally efficient and compact formulation, as well as automatic removal of the optimal mean, which reduces time consumption and enhances evaluation performance. A unified clustering framework based on Schatten- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> norm regularized by ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,<i>q</i></sub> -norm can be formulated using this processing way, where inner element suppression can be achieved by choosing appropriate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p, q</i> ∈ (0,1). Additionally, calculating the optimal mean enhances the robustness of the proposed method in the presence of outliers. Unlike the general iteration scheme of the alternating direction method of multiplier (ADMM) algorithms that introduce auxiliary splitting variables, the proposed alternating re-weighted least square (ARwLS) algorithm uses matrix inverse and multiplication computations to obtain analytic solutions, resulting in faster processing speeds for each sub-problem. To further investigate, we provide the computational complexity of each iteration and the theoretical analysis of the convergence property, where the derived solution is a stationary point. Experimental results on synthetic data and several benchmark datasets demonstrate the promising efficiency and efficacy of the proposed clustering method compared to classical and competing algorithms.

The nuclear norm minimization (NNM) is commonly used to approximate the matrix rank by shrinking all singular values equally. However, the singular values have clear physical meanings in many practical problems, and NNM may not be able to faithfully approximate the matrix rank. To alleviate the above-mentioned limitation of NNM, recent studies have suggested that the weighted nuclear norm minimization (WNNM) can achieve a better rank estimation than NNM, which heuristically set the weight being inverse to the singular values. However, it still lacks a rigorous explanation why WNNM is more effective than NMM in various applications. In this paper, we analyze NNM and WNNM from the perspective of group sparse representation (GSR). Concretely, an adaptive dictionary learning method is devised to connect the rank minimization and GSR models. Based on the proposed dictionary, we prove that NNM and WNNM are equivalent to l 1 -norm minimization and the weighted l 1 -norm minimization in GSR, respectively. Inspired by enhancing sparsity of the weighted l 1 -norm minimization in comparison with l 1- norm minimization in sparse representation, we thus explain that WNNM is more effective than NMM. By integrating the image nonlocal self-similarity (NSS) prior with the WNNM model, we then apply it to solve the image denoising problem. Experimental results demonstrate that WNNM is more effective than NNM and outperforms several state-of-the-art methods in both objective and perceptual quality.

Publication in the conference proceedings of EUSIPCO, Marrakech, Morocco, 2013

Solving optimization problems with sparse or low-rank optimal solutions has been an important topic since the recent emergence of compressed sensing and its matrix extensions such as the matrix rank minimization and robust principal component analysis problems. Compressed sensing enables one to recover a signal or image with fewer observations than the “length” of the signal or image, and thus provides potential breakthroughs in applications where data acquisition is costly. However, the potential impact of compressed sensing cannot be realized without efficient optimization algorithms that can handle extremely large-scale and dense data from real applications. Although the convex relaxations of these problems can be reformulated as either linear programming, second-order cone programming or semidefinite programming problems, the standard methods for solving these relaxations are not applicable because the problems are usually of huge size and contain dense data. In this dissertation, we give efficient algorithms for solving these “sparse” optimization problems and analyze the convergence and iteration complexity properties of these algorithms. Chapter 2 presents algorithms for solving the linearly constrained matrix rank minimization problem. The tightest convex relaxation of this problem is the linearly constrained nuclear norm minimization. Although the latter can be cast and solved as a semidefinite programming problem, such an approach is computationally expensive when the matrices are large. In Chapter 2, we propose fixed-point and Bregman iterative algorithms for solving the nuclear norm minimization problem and prove convergence of the first of these algorithms. By using a homotopy approach together with an approximate singular value decomposition procedure, we get a very fast, robust and powerful algorithm, which we call FPCA (Fixed Point Continuation with Approximate SVD), that can solve very large matrix rank minimization problems. Our numerical results on randomly generated and real matrix completion problems demonstrate that this algorithm is much faster and provides much better recoverability than semidefinite programming solvers such as SDPT3. For example, our algorithm can recover 1000 × 1000 matrices of rank 50 with a relative error of 10−5 in about 3 minutes by sampling only 20 percent of the elements. We know of no other method that achieves as good recoverability. Numerical experiments on online recommendation, DNA microarray data set and image inpainting problems demonstrate the effectiveness of our algorithms. In Chapter 3, we study the convergence/recoverability properties of the fixed-point continuation algorithm and its variants for matrix rank minimization. Heuristics for determining the rank of the matrix when its true rank is not known are also proposed. Some of these algorithms are closely related to greedy algorithms in compressed sensing. Numerical results for these algorithms for solving linearly constrained matrix rank minimization problems are reported. Chapters 4 and 5 considers alternating direction type methods for solving composite convex optimization problems. We present in Chapter 4 alternating linearization algorithms that are based on an alternating direction augmented Lagrangian approach for minimizing the sum of two convex functions. Our basic methods require at most O(1/e) iterations to obtain an e-optimal solution, while our accelerated (i.e., fast) versions require at most O(1/ 3 ) iterations, with little change in the computational effort required at each iteration. For more general problem, i.e., minimizing the sum of K convex functions, we propose multiple-splitting algorithms for solving them. We propose both basic and accelerated algorithms with O(1/e) and O(1/ 3 ) iteration complexity bounds for obtaining an e-optimal solution. To the best of our knowledge, the complexity results presented in these two chapters are the first ones of this type that have been given for splitting and alternating direction type methods. Numerical results on various applications in sparse and low-rank optimization, including compressed sensing, matrix completion, image deblurring, robust principal component analysis, are reported to demonstrate the efficiency of our methods.

GMDH-type neural networks are used for modelling and prediction of explosive cutting process of plates by shaped charges. The aim of such modelling is to show how the depth of penetration varies with the variation of important parameters. It is also demonstrated that singular value decomposition (SVD) can be effectively used to find the vector of coefficients of a quadratic sub-expression embodied in such GMDH-type networks. Such application of SVD will highly improve the performance of GMDH-type networks to model the very complex process of explosive cutting of plates by shaped charges.

The two papers in this issue have to do with matrices and sparsity—but from different points of view. Sparsity, in the first paper, means many zero elements in the matrix, while in the second paper it refers to many zero singular values, i.e., low rank. The context of the first paper, “On the Block Triangular Form of Symmetric Matrices” by Iain Duff and Bora Uçar, is the solution of linear systems of equations $Ax = b$ whose coefficient matrix A is sparse, i.e., has many zero elements. A direct method, such as Gaussian elimination, becomes more efficient if one can first permute the rows and columns of A into block triangular form. The classical method for permuting a matrix to block triangular form is due to Dulmage and Mendelsohn and dates back to 1963. The idea is to represent the matrix A as a bipartite graph whose nodes are columns and rows of A, and whose edges correspond to nonzero elements of A, and then to determine a matching of maximum cardinality in this graph. That means determining a maximal number of nonzero elements no two of which belong to the same row or column. Duff and Uçar analyze the block triangular form for a particular class of square matrices A: These matrices are structurally symmetric, i.e., their zero/nonzero structure is symmetric; and they can be structurally rank deficient, i.e., any rank deficiency is evident from the zero/nonzero structure of A. This paper illustrates nicely how graph theory can contribute to improving the efficiency of sparse matrix methods. The paper should appeal to those who need to solve large sparse linear systems, as well as those interested in graph theory. The second paper, “Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization” by Benjamin Recht, Maryam Fazel, and Pablo Parrilo, has applications to model reduction and system identification, machine learning, and image compression, just to name a few. Given a set of affine constraints for a matrix, the problem is to find a matrix of minimum rank that satisfies these constraints. The authors attack this hard nonconvex optimization problem by replacing it with a convex approximation: Instead of minimizing the rank, they minimize the sum of the singular values, the so-called nuclear norm. Nuclear norm minimization thus extends the compressed sensing framework from finding sparse vectors via $\ell_1$ minimization to finding low-rank matrices via $\ell_1$ minimization of the (vector of) singular values. The purpose of this paper is to justify mathematically why nuclear norm minimization does so well in practice. In addition to extending the restricted isometry property to matrices, the authors also discuss algorithms, computational performance, and numerical issues. This is a fascinating and well-written paper that combines results from compressed sensing, matrix theory, optimization, and probability.

At present, deep learning method is widely used in the field of gray image colorization. Qinghai farmer painting has distinct national characteristics. The farmer painting has bright colors, high saturation, chaotic color distribution and low color contrast, so it is difficult to restore the image color with high fidelity by using the general deep learning image colorization method. The Pix2Pix generation adversarial network of grayscale image colorization method uses the Leaky ReLU function as the activation function. The proposal algorithm replaces the maximum pooling layer with the convolution layer to retain more image feature information and further to improve the color simulation effect. Meanwhile, in view of the lack of relevant Qinghai farmer painting data set, the data set of Qinghai farmer paintings is constructed to meet the needs of network training. The experimental results show that the improved method further improves the color effect and can generate high quality color images of Qinghai farmer paintings with more real color distribution.

The field of pulsed power has enjoyed widespread utilization, from industrial welding to high power microwave generation. In pulsed power, an energy storage device is charged to a specific voltage over a long time period. To achieve high peak power, the energy storage device is discharged over a small time duration, ranging from nanoseconds to microseconds, depending on the application. As the field matures from single-shot machines that fire only a few times a day to rep-rated operation with several shots per minute, the ability for operators to determine the operational state of the system on the fly is extremely limited. This paper discusses the application of Singular Value Decomposition (SVD) to real-time data output from Rogowski coils on a capacitor bank to determine the state of health between shots. The SVD pattern recognition system can be used to rapidly ascertain if the system should continue to operate, or if there is a potential problem developing somewhere in the system. This technique may lead to improving the operation of current and future pulsed power systems.

This paper concerns accurate computation of the singular value decomposition (SVD) of an $m\times n$ matrix $A$ . As is well known, cross-product matrix based SVD algorithms compute large singular values accurately but generally deliver poor small singular values. A new novel cross-product matrix based SVD method is proposed: (a) Use a backward stable algorithm to compute the eigenpairs of $A^{\rm T}A$ and take the square roots of the large eigenvalues of it as the large singular values of $A$ ; (b) form the Rayleigh quotient of $A^{\rm T}A$ with respect to the matrix consisting of the computed eigenvectors associated with the computed small eigenvalues of $A^{\rm T}A$ ; (c) compute the eigenvalues of the Rayleigh quotient and take the square roots of them as the small singular values of $A$ . A detailed quantitative error analysis is conducted on the method. It is proved that if small singular values are well separated from the large ones then the method can compute the small ones accurately up to the order of the unit roundoff $\epsilon$ . An algorithm is developed that is not only cheaper than the standard Golub–Reinsch and Chan SVD algorithms but also can update or downdate a new SVD by adding or deleting a row and compute certain refined Ritz vectors for large matrix eigenproblems at very low cost. Several variants of the algorithm are proposed that compute some or all parts of the SVD. Typical numerical examples confirm the high accuracy of our algorithm.

Non-Cooperative Target Identification based on High Resolution Range Profiles is a key research domain in the Defense industry. In this paper a method based on the application of Singular Value Decomposition to a matrix of range profiles is defined. The decomposition is applied to reduce dimensionality and to accomplish recognition in the transformed domain. So as to confirm the feasibility of the methodology, identification experiments of profiles coming from electromagnetic simulations are conducted, revealing promising results.