A Non-local Low Rank and Total Variation Approach for Depth Image Estimation

Rapid advancement and active research in computer vision applications and 3D imaging have made a high demand for efficient depth image estimation techniques. The depth image acquisition, however, is typically challenged due to poor hardware performance and high computation cost. To tackle such limitations, this paper proposes an efficient approach for depth image reconstruction using low rank (LR) and total variation (TV) regularizations. The key idea is LR incorporates non-local depth information and TV takes into account the local spatial consistency. The proposed model reformulates the task of depth image estimate as a joint LR-TV regularized minimization problem, in which LR is used to approximate the low-dimensional structure of the depth image, and TV is employed to promote the depth sparsity in the gradient domain. Furthermore, this paper introduces an algorithm based on alternating direction method of multipliers (ADMM) for solving the minimization problem, whose solution provides an estimate of the depth map from incomplete pixels. Experimental results are conducted and the results show that the proposed approach is very effective at estimating high-quality depth images and is robust to different types of data missing models.

Hyperspectral and multispectral image fusion plays an important role in the fields such as image restoration, space robotics, remote sensing, computer vision, etc. The fused images take advantages of the high spectral resolutions of hyperspectral images and the high spatial resolutions of multispectral images. So there has been a recent outgrowth of interests of research in this field. However, the local structural image regularity and non-local image self-similarity are not yet explored in existing fusion methods. This paper develops a novel method for fusing hyperspectral and multispectral images with non-local structure tensor total variation regularization. Specifically, the weight function to measure self-similarity in non-local domain is introduced on the basis of the definition of structure tensor. Therefore, the proposed method can exploit the regularity and self-similarity of spatial-spectral image’s non-local structures simultaneously. In addition to the general formulation of non-local structure tensor variation, this paper presents the discrete non-local structure tensor variation for hyperspectral and multispectral image fusion whose measurements are discrete. This method can provide an alternative way to measure the image variation of multi-band image. The resulting problem is solved by alternating direction method of multipliers (ADMM), a simple but powerful algorithm for convex optimization problems. The optimization problem is handled by a splitting variables strategy, which is implemented by introducing four auxiliary variables regarding to the required fused image. The iteration related to the proposed non-local structure tensor regularization term is solved by the proximal mapping method. The solutions of other iterations are given through fast Fourier transforms. Extensive computational experiments are performed on various datasets. The results of index evaluation demonstrate the effectiveness of the proposed method. These results also indicate non-local structure tensor total variation regularization has obvious advantages over the state-of-the-art fusion methods.KeywordsImage fusionNon-local tensor regularizationOptimization method

The use of non-local self-similarity prior between image blocks can improve image reconstruction performance significantly. We propose a compressive sensing image reconstruction algorithm that combines bilateral total variation and nonlocal low-rank regularization to overcome over-smoothing and degradation of edge information which result from the prior reconstructed image. The proposed algorithm makes use of the preservation of image edge information by bilateral total variation operator to enhance the edge details of the reconstructed image. In addition, we use weighted nuclear norm regularization as a low-rank constraint for similar blocks of the image. To solve this convex optimization problem, the Alternating Direction Method of Multipliers (ADMM) is employed to optimize and iterate the algorithm model effectively. Experimental results show that the proposed algorithm can obtain better image reconstruction quality than conventional algorithms with using total variation regularization or considering the nonlocal structure of the image only. At 10% sampling rate, the peak signal-to-noise ratio gain is up to 2.39 dB in noiseless measurements compared with Nonlocal Low-rank Regularization (NLR-CS). Reconstructed image comparison shows that the proposed algorithm retains more high frequency components. In noisy measurements, the proposed algorithm is robust to noise and the reconstructed image retains more detail information.

Regularization plays a crucial role in reliably utilizing imaging systems for scientific and medical investigations. It helps to stabilize the process of computationally undoing any degradation caused by physical limitations of the imaging process. In the past decades, total variation regularization played a dominant role in the literature. Two forms of total variation regularizations, namely the first-order and the second-order total variation (TV-1 and TV-2) have been widely used. TV-1 has a disadvantage: it reconstructs images in the form of piece-wise constants when the noise and/or under-sampling is severe, while TV-2 reconstructs natural-looking images under such scenarios. On the other hand, TV-1 can recover sharp jumps better than TV-2. Two forms of generalizations, namely Hessian-Schatten norm (HSN) regularization, and total generalized variation (TGV) regularization, have been proposed and have become significant developments in the area of regularization for imaging inverse problems owing to their performance. While the strength of TGV is that it can combine the advantages of TV-1 and TV-2, HSN has better structure-preserving property. Here, we develop a novel regularization for image recovery that combines the strengths of TGV and HSN. We achieve this by restricting the maximization space in the dual form of HSN in the same way that TGV is obtained from TV-2. We call the new regularization the generalized Hessian-Schatten norm regularization (GHSN). We develop a novel computational method for image reconstruction using the new form of regularization based on the well-known framework called the alternating direction method of multipliers (ADMM). We demonstrate the strength of the GHSN using some reconstruction examples.

It is required that input features are represented as vectors or scalars in machine learning for classification, e.g. support vector machine (SVM). However, real world data such as 2D images is naturally represented as matrices or tensors with higher dimensions. Thus, structural information of the data whose dimensions are more than two is not successfully considered. One typical structural information which is useful for the classification task is the spatial relationship of the nearby data points. In this paper, to leverage this kind of structural information, we propose a novel classification method which combines total variational (TV) regularization with SVM, called TV-SVM. Since TV achieves a local smoothing property by penalizing the local discontinuity of data, TV-SVM preserves better local structure than the original SVM due to TV regularization. We solve the objective function of TV-SVM via the alternating direction method of multipliers (ADMM) algorithm. Experimental results on image classification show that TV-SVM is competitive to the state-of-the-art learning method in both classification accuracy and computational complexity.

Regularization method is an effective tool for synthetic aperture radar (SAR) image despeckling. Design of the effective regularization terms describing the image priors plays a vital role in this kind of method. In this article, a new combinational regularization model for speckle reduction (CRM-SR) is proposed, in which a regularization term is elaborately designed to contain both a fractional-order total variation (FrTV) regularization and a nonlocal low rank (NLR) regularization. The new regularization model inherits both the advantages of FrTV and NLR and improves the performance of SAR despeckling and, therefore, better preserves the edges and geometrical features of the images during the despeckling process. An efficient algorithm based on alternating direction optimization is derived to solve the proposed combinational regularization model. Experimental results show that the proposed model can effectively remove SAR image speckle and preserve the geometrical features of images according to both subjective visual assessment of image quality and objective evaluation.

<p>Total variation (TV) regularization is a powerful tool in image denoising, but it often exhibits limited performance in preserving edges. In contrast, non-convex TV regularization can more effectively preserve edges and contours, albeit posing challenges when solving. Recently, the convex non-convex (CNC) strategy has emerged as a potent approach that allows incorporating non-convex TV regularization terms while maintaining the overall convexity of the objective function. Its superior performance has been validated through various numerical experiments; however, theoretical analysis remains lacking. In this paper, we provided theoretical analysis of the performance of the CNC-TV denoising model. By utilizing the oracle inequality, we derived an improved upper bound on its performance compared to TV regularization. In addition, we devised an alternating direction method of multipliers (ADMM) algorithm to address the proposed model and verified its convergence properties. Our proposed model has been validated through numerical experiments in 1D and 2D denoising, demonstrating its exceptional performance.</p>

This study develops an improved Feldkamp–Davis–Kress (FDK) reconstruction algorithm using non-local total variation (NLTV) denoising and a cubic B-spline interpolation-based backprojector to enhance the image quality of low-dose cone-beam computed tomography (CBCT). The NLTV objective function is minimized on all log-transformed projections using steepest gradient descent optimization with an adaptive control of the step size to augment the difference between a real structure and noise. The proposed algorithm was evaluated using a phantom data set acquired from a low-dose protocol with lower milliampere-seconds (mAs).The combination of NLTV minimization and cubic B-spline interpolation rendered the enhanced reconstruction images with significantly reduced noise compared to conventional FDK and local total variation with anisotropic penalty. The artifacts were remarkably suppressed in the reconstructed images. Quantitative analysis of reconstruction images using low-dose projections acquired from low mAs showed a contrast-to-noise ratio with spatial resolution comparable to images reconstructed using projections acquired from high mAs. The proposed approach produced the lowest RMSE and the highest correlation. These results indicate that the proposed algorithm enables application of the conventional FDK algorithm for low mAs image reconstruction in low-dose CBCT imaging, thereby eliminating the need for more computationally demanding algorithms. The substantial reductions in radiation exposure associated with the low mAs projection acquisition may facilitate wider practical applications of daily online CBCT imaging.

Total variation (TV) regularization method is a typical method to preserve the discontinuities structure in EIT. Isotropic TV and anisotropic TV are two well-known variants of TV. The main differences between them are that the latter tends to distort the reconstructed internal inhomogeneities along the coordinate axis. In this article, we adopt the alternating direction method of multipliers (ADMM) to overcome the non-differentiability of the anisotropic TV and verify the characteristics of anisotropic TV regularization by the tank experiments.

We consider the problem of restoring images corrupted by Poisson noise. Under the framework of maximum a posteriori estimator, the problem can be converted into a minimization problem where the objective function is composed of a Kullback-Leibler (KL)-divergence term for the Poisson noise and a total variation (TV) regularization term. Due to the logarithm function in the KL-divergence term, the non-differentiability of TV term and the positivity constraint on the images, it is not easy to design stable and efficiency algorithm for the problem. Recently, many researchers proposed to solve the problem by alternating direction method of multipliers (ADMM). Since the approach introduces some auxiliary variables and requires the solution of some linear systems, the iterative procedure can be complicated. Here we formulate the problem as two new constrained minimax problems and solve them by Chambolle-Pocks first order primal-dual approach. The convergence of our approach is guaranteed by their theory. Comparing with ADMM approaches, our approach requires about half of the auxiliary variables and is matrix-inversion free. Numerical results show that our proposed algorithms are efficient and outperform the ADMM approach.

Total variation regularization is well-known for recovering sharp edges; however, it usually produces staircase artifacts. In this paper, in order to overcome the shortcoming of total variation regularization, we propose a new variational model combining high-order total variation regularization and l1 regularization. The new model has separable structure which enables us to solve the involved subproblems more efficiently. We propose a fast alternating method by employing the fast iterative shrinkage-thresholding algorithm (FISTA) and the alternating direction method of multipliers (ADMM). Compared with some current state-of-the-art methods, numerical experiments show that our proposed model can significantly improve the quality of restored images and obtain higher SNR and SSIM values.

Total variation (TV) based models are very popular in image denoising but suffer from some drawbacks. For example, local TV methods often cannot preserve edges and textures well when they face excessive smoothing. Non-local TV methods constitute an alternative, but their computational cost is huge. To overcome these issues, we propose an image denoising method named non-local patch graph total variation (NPGTV). Its main originality stands for the graph total variation method, which combines the total variation with graph signal processing. Schematically, we first construct a K-nearest graph from the original image using a non-local patch-based method. Then the model is solved with the Douglas-Rachford Splitting algorithm. By doing so, the image details can be well preserved while being denoised. Experiments conducted on several standard natural images illustrate the effectiveness of our method when compared to some other state-of-the-art denoising methods like classical total variation, non-local means filter (NLM), non-local graph based transform (NLGBT), adaptive graph-based total variation (AGTV).

Linearized proximal alternating minimization algorithm for motion deblurring by nonlocal regularization

In the last decades, unsupervised deep learning based methods have caught researchers' attention, since in many real applications, such as medical imaging, collecting a large amount of training examples is not always feasible. Moreover, the construction of a good training set is time consuming and hard because the selected data have to be enough representative for the task. In this paper, we focus on the Deep Image Prior (DIP) framework and we propose to combine it with a space-variant Total Variation regularizer with an automatic estimation of the local regularization parameters. Differently from other existing approaches, we solve the arising minimization problem via the flexible Alternating Direction Method of Multipliers (ADMM). Furthermore, we provide a specific implementation also for the standard isotropic Total Variation. The promising performances of the proposed approach, in terms of PSNR and SSIM values, are addressed through several experiments on simulated as well as real natural and medical corrupted images.

This paper presents a new algorithm for the joint restoration of depth and intensity images constructed from the time-correlated single-photon counting (TCSPC) measurement in the limit of very few photon counts [1]. Under some justified approximations, the restoration problem (regularized likelihood) reduces to a convex formulation with respect to the parameters of interest. The first advantage of this formulation is that it only processes the corrupted depth and intensity images obtained from preliminary estimation, without the need for the use of full TCSPC waveforms. The second advantage is its flexibility in being able to use different convex regularization terms such as: total variation (TV); and sparsity of the discrete cosine transform (DCT) coefficients. The estimation problems are efficiently solved using the alternating direction method of multipliers (ADMM) that presents good convergence properties and thus a reduced computational cost. Results on single photon depth data from field trials show the benefit of the proposed strategy that improves the quality of the estimated depth and intensity images.