An Efficient Collaborative Filtering Method for Image Noise and Artifact Removal

Ultrasound (US) image quality often suffers from random and speckle noise. Especially in the case of difficult patients, increased presence of random noise leads to compromised signal to noise ratio (SNR). The proposed technique removes random or speckle noise by promoting the low rank component of temporal US image sequences comprising of RF-data or envelope-detected data, respectively. This approach leverages the fact that due to tissue connectedness, tissue motion tends to exhibit limited temporal profiles. An approach to approximate the low rank component of the image sequence involves truncating its singular value decomposition. In the proposed algorithm, this is performed in a block-wise manner. Additionally, an adaptive singular value threshold is proposed based on Stein's unbiased risk estimator. This paper presents applications of this methodology on sequences of RF-data and envelope-detected data.

We propose an image colorization method using fast soft-thresholding of singular values (singular value thresholding). An image colorization method with nuclear norm minimization (NNM) has been proposed and brings good results. NNM usually requires iterative application of singular value decomposition (SVD) for singular value thresholding. However, the computational cost of SVD in the colorization method becomes too expensive to handle high-resolution images. In this paper, we reduce its computational cost by using Chebyshev polynomial approximation (CPA). Singular value thresholding is expressed by a multiplication of certain matrices derived from the characteristic of CPA. As a result, our CPA-based technique makes the image colorization method much more efficient. In addition, we replace the optimization method used in the image colorization method by alternating direction method of multipliers, which further accelerates the computation. Experimental results verify the effectiveness of our method with respect to the computation time and the approximation precision.

To develop a novel regularization method for GRAPPA by which the regularization parameters can be optimally and adaptively chosen. In the fit procedures in GRAPPA, the discrepancy principle, which chooses the regularization parameter based on a priori information about the noise level in the autocalibrating signals (ACS), is used with the truncated singular value decomposition (TSVD) regularization and the Tikhonov regularization, and its performance is compared with the singular value (SV) threshold method and the L-curve method, respectively by axial and sagittal head imaging experiments. In both axial and sagittal reconstructions, normal GRAPPA reconstruction results exhibit a relatively high level of noise. With discrepancy-based choices of parameters, regularization can improve the signal-to-noise ratio (SNR) with only a very modest increase in aliasing artifacts. The L-curve method in all of the reconstructions leads to overregularization, which causes severe residual aliasing artifacts. The 10% SV threshold method yields good overall image quality in the axial case, but in the sagittal case it also leads to an obvious increase in aliasing artifacts. Neither a fixed SV threshold nor the L-curve are robust means of choosing the appropriate parameters in GRAPPA reconstruction. However, with the discrepancy-based parameter-choice strategy, adaptively regularized GRAPPA can be used to automatically choose nearly optimal parameters for reconstruction and achieve an excellent compromise between SNR and artifacts.

We are interested in solving the following minimization problem Dτ (Y ) := arg min X∈Rm×n 1 2 ∥Y −X∥F + τ∥X∥∗, where Y ∈ Rm×n is a given matrix, and ∥ ⋅ ∥F is the Frobenius norm and ∥ ⋅ ∥∗ the nuclear norm. This problem serves as a basic subroutine in many popular numerical schemes for nuclear norm minimization problems, which arise from low rank matrix recovery such as matrix completion. As Dτ (Y ) has an explicit expression which shrinks the singular values of Y and keeps the singular vectors, Dτ is referred to singular value thresholding (SVT) operator in the literature. Conventional approaches for Dτ (Y ) first find the singular value decomposition (SVD) of Y and then shrink the singular values. However, such approaches are time consuming under some circumstances, especially when the rank of Dτ (Y ) is not low compared to the matrix dimension or is completely unpredictable. In this paper, we propose a fast algorithm for directly computing Dτ (Y ) without using SVDs. Numerical experiments show that the proposed algorithm is much more efficient than the approach using the full SVD.

With the growing interest in convex and nonconvex low-rank matrix learning problems, the widely used singular value thresholding (SVT) operators associated with rank relaxation functions often face higher computational complexity, particularly for large-scale data matrices. To improve the efficacy of low-rank subspace clustering and overcome the issue of high computational complexity, this work proposes an efficient and effective method that avoids the need for singular value decomposition (SVD) computations in the iteration scheme. This can be achieved through the use of a computationally efficient and compact formulation, as well as automatic removal of the optimal mean, which reduces time consumption and enhances evaluation performance. A unified clustering framework based on Schatten- <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</i> norm regularized by ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,<i>q</i></sub> -norm can be formulated using this processing way, where inner element suppression can be achieved by choosing appropriate <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p, q</i> ∈ (0,1). Additionally, calculating the optimal mean enhances the robustness of the proposed method in the presence of outliers. Unlike the general iteration scheme of the alternating direction method of multiplier (ADMM) algorithms that introduce auxiliary splitting variables, the proposed alternating re-weighted least square (ARwLS) algorithm uses matrix inverse and multiplication computations to obtain analytic solutions, resulting in faster processing speeds for each sub-problem. To further investigate, we provide the computational complexity of each iteration and the theoretical analysis of the convergence property, where the derived solution is a stationary point. Experimental results on synthetic data and several benchmark datasets demonstrate the promising efficiency and efficacy of the proposed clustering method compared to classical and competing algorithms.

We present a method for image denoising based on singular value shrinkage that fuses soft and hard thresholds. The technique simply groups similar patches from a noisy image as low-rank matrices and shrinks the singular values by the combination of soft and hard thresholds. On one hand, a hard threshold approximation method based on nonlocal self-similarity and low-rank approximation is used for fast selection of hard threshold; on the other hand, a soft threshold selection method based on random matrix and asymptotic matrix reconstruction theory is designed. In addition, we also propose an adaptive backward projection algorithm based on image phase congruency and gradient calculation so that the input images participating in the iteration are adaptive. This method improves the traditional fixed coefficient backward projection method and makes the robustness of the algorithm better. The experimental results of denoising and enhancement for a number of natural images show that the proposed algorithms have significant improvement in both subjective visual effect and objective quantization index by comparing with some related state-of-the-art denoising algorithms.

According to the image processing theory, complex texture image regions have larger entropy values, an adaptive steganography algorithm in the sparse domain is proposed here. The algorithm first divides an image into nonoverlapping image blocks, then computes the entropy of each block and chooses the mean of all entropies as a threshold. Image blocks whose entropy is higher than the threshold are selected for sparse decomposition as complex texture image regions. The secret message is embedded into the decomposition coefficients, and then the stego image is reconstructed with modified coefficients. The experimental results prove that, given the same payload, the proposed adaptive algorithm has a better performance against the steganalyzer.

This paper concerns accurate computation of the singular value decomposition (SVD) of an $m\times n$ matrix $A$ . As is well known, cross-product matrix based SVD algorithms compute large singular values accurately but generally deliver poor small singular values. A new novel cross-product matrix based SVD method is proposed: (a) Use a backward stable algorithm to compute the eigenpairs of $A^{\rm T}A$ and take the square roots of the large eigenvalues of it as the large singular values of $A$ ; (b) form the Rayleigh quotient of $A^{\rm T}A$ with respect to the matrix consisting of the computed eigenvectors associated with the computed small eigenvalues of $A^{\rm T}A$ ; (c) compute the eigenvalues of the Rayleigh quotient and take the square roots of them as the small singular values of $A$ . A detailed quantitative error analysis is conducted on the method. It is proved that if small singular values are well separated from the large ones then the method can compute the small ones accurately up to the order of the unit roundoff $\epsilon$ . An algorithm is developed that is not only cheaper than the standard Golub–Reinsch and Chan SVD algorithms but also can update or downdate a new SVD by adding or deleting a row and compute certain refined Ritz vectors for large matrix eigenproblems at very low cost. Several variants of the algorithm are proposed that compute some or all parts of the SVD. Typical numerical examples confirm the high accuracy of our algorithm.

Summary form only given. Block-based Discrete Cosine Transform (BDCT) has been widely used in image and video compression due to its energy compacting property and relative ease of implementation. However, BDCT has a major drawback, which is usually referred to as blocking artifacts. Blocking artifacts appear as grid noise along the block boundaries because each block is transformed and quantized independently. Image deblocking techniques can reduce these distortions and alleviate the conflict between bit rate reduction and visual quality preservation. Many state-of-the-art image deblocking algorithms treated the blocking artifacts reduction of the compressed image as an inverse restoration problem. Natural image prior models are well utilized into the blocking artifacts reduction processing, such as the local sparsity prior model and non-local similarity property of natural images. These two local and non-local models characterize the image prior information in two complementary perspectives. Therefore, it is necessary to combine these two models in a unified framework. In this paper, we propose a novel method to reduce the blocking artifacts of blockcoded images via patch clustering and low-rank minimization, which simultaneously exploits the local and non-local sparse representations in a unified framework. First, the whole compressed image are divided into small patches. For each patch, we perform patch clustering to collect similar patches into a group. Then the whole group are simultaneously reconstructed by a low-rank minimization approach. Singular value thresholding (SVT) algorithm is employed to solve the low-rank minimization problem. To further improve the performance of the proposed algorithm, we adopt an iterative procedure to utilize the newly output data in each iteration and update the noise and signal variance adaptively. Experimental results show that the proposed method achieves higher PSNR and SSIM than the state-of-the-art methods. Comparing to the state-of-theart algorithms and, the proposed algorithm achieves about 0.37dB and 0.11dB improvement on average. For visual quality assessment, the deblocking images produced by the proposed algorithm reveal much more sharp edge structures and richer textures.

The detection process of partial discharge (PD) ultra-high frequency (UHF) signal is easily affected by white noise and periodic narrowband noise, which hinder the fault diagnosis of high-voltage electrical appliances. In order to extract PD UHF signal and suppress noise effectively, this paper provides a new method to detect PD UHF signal by EDSSV and low rank RBF neural network. Firstly, the singular value decomposition (SVD) is performed on the mixed noises of PD signal. Secondly, the peak index of energy difference spectrum of singular value (EDSSV) is selected as optimal singular value threshold, and then the periodic narrowband noise is eliminated by reconstructing the effective rank order. Finally, radial basis function (RBF) neural network is used to approximate the denoised PD signal, and Gaussian window filter is used to extract the PD signal. To verify the performance of the proposed method, we compared it with other three algorithms in simulation and field detection, including adaptive singular value decomposition (ASVD), singular value decomposition based on S-transform and MTFM (S-SVD) and EMD-WT algorithms. Particularly, four evaluation indices are designed for the detection data, which consider both the noise suppression and feature preservation. The results demonstrate the validity of the proposed method with higher signal-to-noise ratio and less waveform distortion.

This thesis deals with recurrent neural networks, a particular class of artificial neural networks which can learn a generative model of input sequences. The input is mapped, through a feedback loop and a non-linear activation function, into a hidden state, which is then projected into the output space, obtaining either a probability distribution or the new input for the next time-step. This work consists mainly of two parts: a theoretical study for helping the understanding of recurrent neural networks framework, which is not yet deeply investigated, and their application to non-linear prediction problems, since recurrent neural networks are really powerful models suitable for solving several practical tasks in different fields. For what concerns the theoretical part, we analyse the weaknesses of state-of-the-art models and tackle them in order to improve the performance of a recurrent neural network. Firstly, we contribute in the understanding of the dynamical properties of a recurrent neural network, highlighting the close relation between the definition of stable limit cycles and the echo state property of an echo state network. We provide sufficient conditions for the convergence of the hidden state to a trajectory, which is uniquely determined by the input signal, independently of the initial states. This may help extend the memory of the network and increase the design options for the network. Moreover, we develop a novel approach to address the main problem in training recurrent neural networks, the so-called vanishing gradient problem. Our new method allows us to train a very simple recurrent neural network, making the gradient not to vanish even after many time-steps. Exploiting the singular value decomposition of the vanishing factors in the gradient and random matrices theory, we find that the singular values have to be confined in a narrow interval and derive conditions about their root mean square value. Then, we also improve the efficiency of the training of a recurrent neural network, defining a new method for speeding up this process. Thanks to a least square regularization, we can initialize the parameters of the network, in order to set them closer to the minimum and running fewer epochs of classical training algorithms. Moreover, it is also possible to completely train the network with our initialization method, running more iterations of it without losing in performance with respect to classical training algorithms. Finally, it is also possible to use it as a real-time learning algorithm, adjusting the parameters to the new data through one iteration of our initialization. In the last part of this thesis, we apply recurrent neural networks to non-linear prediction problems. We consider prediction of numerical sequences, estimating the following input choosing it from a probability distribution. We study an automatic text generation problem, where we need to predict the following character in order to compose words and sentences, and a path prediction of walking mobile users in the central area of a city, as a sequence of crossroads. Then, we analyse the prediction of video frames, discovering a wide range of applications related to the prediction of movements. We study the collision problem of bouncing balls, taking into account only the sequence of video frames without any knowledge about the physical characteristics of the problem, and the distribution over days of mobile user in a city and in a whole region. Finally, we address the state-of-the-art problem of missing data imputation, analysing the incomplete spectrogram of audio signals. We restore audio signals with missing time-frequency data, demonstrating via numerical experiments that a performance improvement can be achieved involving recurrent neural networks.

Modern deep neural networks (DNNs) often require high memory consumption and large computational loads. In order to deploy DNN algorithms efficiently on edge or mobile devices, a series of DNN compression algorithms have been explored, including factorization methods. Factorization methods approximate the weight matrix of a DNN layer with the multiplication of two or multiple low-rank matrices. However, it is hard to measure the ranks of DNN layers during the training process. Previous works mainly induce low-rank through implicit approximations or via costly singular value decomposition (SVD) process on every training step. The former approach usually induces a high accuracy loss while the latter has a low efficiency. In this work, we propose SVD training, the first method to explicitly achieve low-rank DNNs during training without applying SVD on every step. SVD training first decomposes each layer into the form of its full-rank SVD, then performs training directly on the decomposed weights. We add orthogonality regularization to the singular vectors, which ensure the valid form of SVD and avoid gradient vanishing/exploding. Low-rank is encouraged by applying sparsity-inducing regularizers on the singular values of each layer. Singular value pruning is applied at the end to explicitly reach a low-rank model. We empirically show that SVD training can significantly reduce the rank of DNN layers and achieve higher reduction on computation load under the same accuracy, comparing to not only previous factorization methods but also state-of-the-art filter pruning methods.

This paper investigates image denoising algorithm based on sparse representation with over-complete dictionary. Here K-SVD(Singular Value Decomposition) algorithm is used to train the over-complete dictionary, followed by OMP(Orthogonal Matching Pursuit) algorithm for image sparse decomposition. While this strategy achieves good performance on image denoising, it has high computational complexity. In order to speed up computation, Batch-OMP instead of OMP algorithm is adopted to improve the denosing algorithm, which significantly shortens the running time. Finally, analysis is made on the configuration of parameters versus the performance in the denoising algorithm; we use particle swarm optimization algorithm to learn the best parameters.

We monitored the occupancy of a functionally important non-coordinated water molecule in the distal heme pocket of sperm whale myoglobin over the pH range 4.3-9.4. Water occupancy was assessed by using time-resolved spectroscopy to detect the perturbation of the heme visible band absorption spectrum caused by water entry after CO photodissociation ( Goldbeck, R. A., Bhaskaran, S., Ortega, C., Mendoza, J. L., Olson, J. S., Soman, J., Kliger, D. S., and Esquerra, R. M. (2006) Proc. Natl. Acad. Sci. U. S. A. 103, 1254-1259 ). We found that the water occupancy observed during the time interval between ligand photolysis and diffusive recombination decreased by nearly 20% as the pH was lowered below 6. This decrease accounted for most of the concomitant increase in the observed CO bimolecular recombination rate constant, as the lower water occupancy presented a smaller kinetic barrier to CO entry into the pocket at lower pH. These results were consistent with a model in which the distal histidine, which stabilizes the water molecule within the distal pocket by accepting a hydrogen bond, tends to swing out of the pocket upon protonation and destabilize the water occupancy at low pH. Extrapolation of this model to lower pH suggests that the additional increase in ligand association rate constant observed previously in stopped-flow studies at pH 3 may also be due in part to reduced distal water occupancy concomitant with further His64 protonation and coupled protein conformational change.

When seismic waves encounter abrupt points of stratum or lithology in the process of propagation, such as fault edges, pinch-out points, or the protrusion of unconformity surfaces, the Snell law will break down and many diffractions will be generated. However, the diffraction information is typically masked by strong reflections; thus, separating diffractions is one key issue for diffraction imaging. The traditional singular value decomposition (SVD) method with the ability for wavefield decomposition and reconstruction is useful in removing strong reflections of large singular values. However, the continuously changing characteristics of the singular value of steep-slope reflections make it difficult to choose a suitable parameter for separating diffractions. To solve the problem of singular value selection in diffractions reconstruction by the SVD method, we have developed a Cook-distance SVD method by mathematically least-squares measuring the contribution of every singular value. The Cook-distance SVD can amplify the difference between continuously changing singular values and easily find the singular values representing diffractions. Synthetic examples and field data applications demonstrate that our Cook-distance SVD method can avoid parameter testing and has good performance in diffraction separation and can highlight reservoir-involved small-scale geologic edges and scatterers.