Penalized logspline density estimation using total variation penalty

Purpose: To combine total variation (TV) and Hessian penalty in a structure adaptive way for cone-beam CT (CBCT) reconstruction. Methods: TV is a widely used first order penalty with good ability in suppressing noise and preserving edges but leads to the staircase effect in regions with smooth intensity transition. The second order Hessian penalty can effectively suppress the staircase effect with extra cost of blurring object edges. To take the best of both penalties we proposed a novel method to combine both for CBCT reconstruction in a structure adaptive way. The proposed method adaptively determined the weight of each penalty according to the geometry of local regions. An specially-designed exponent term with image gradient involved was used to characterize the local geometry such that the weights for Hessian and TV were 1 and 0 respectively at uniform local regions and 0 and 1 at edge regions. For other local regions the weights varied from 0 to 1. The objective functional was minimized using the majorzationminimization approach. We evaluated the proposed method on a modified 3D shepp-logan and a CatPhan 600 phantom. The full-width-at-halfmaximum (FWHM) and contrast-to-noise (CNR) were calculated. Results: For 3D shepp-logan the reconstructed images using TV had an obvious staircase effect while those using the proposed method and Hessian preserved the smooth transition regions well. FWHMs of the proposed method TV and Hessian penalty were 1.75 1.61 and 3.16 respectively, indicating that both TV and the proposed method is able to preserve edges. For CatPhan 600 CNR values of the proposed method were similar to those of TV and Hessian. Conclusion: The proposed method retains favorable properties of TV like preserving edges and also has the ability in better preserving gradual transition structure as Hessian does. All methods performs similarly in suppressing noise. This work was supported in part by National Natural Science Foundation of China (NNSFC) under Grant Nos.60971112 and 61375018 grants from the Cancer Prevention and Research Institute of Texas (RP130109 and RP110562-P2) National Institute of Biomedical Imaging and Bioengineering (R01 EB020366) and a grant from the American Cancer Society (RSG-13-326-01-CCE).

The ℓ 1 and total variation (TV) penalties have been used successfully in many areas, and the combination of the ℓ 1 and TV penalties can lead to further improved performance. In this work, we investigate the mathematical theory and numerical algorithms for the ℓ 1 -TV model in the context of signal recovery: we derive the sample complexity of the ℓ 1 -TV model for recovering signals with sparsity and gradient sparsity. Also we propose a novel algorithm (PGM-ISTA) for the regularized ℓ 1 -TV problem, and establish its global convergence and parameter selection criteria. Furthermore, we construct a fast learned solver (LPGM-ISTA) by unrolling PGM-ISTA. The results for the experiment on ECG signals show the superior performance of LPGM-ISTA in terms of recovery accuracy and computational efficiency.

This study examines an adaptive log-density estimation method with an $$\ell _1$$ -type penalty. The proposed estimator is guaranteed to be a valid density in the sense that it is positive and integrates to one. The smoothness of the estimator is controlled in a data-adaptive way via $$\ell _1$$ penalization. The advantages of the penalized log-density estimator are discussed with an emphasis on wavelet estimators. Theoretical properties of the estimator are studied when the quality of fit is measured by the Kullback–Leibler divergence (relative entropy). A nonasymptotic oracle inequality is obtained assuming a near orthogonality condition on the given dictionary. Based on the oracle inequality, selection consistency and minimax adaptivity are proved under some regularity conditions. The proposed method is implemented with a coordinate descent algorithm. Numerical illustrations based on the periodized Meyer wavelets are performed to demonstrate the finite sample performance of the proposed estimator.

First-photon imaging is a photon-efficient, computational imaging technique that reconstructs an image by recording only the first-photon arrival event at each spatial location and then optimizing the recorded photon information. The optimization algorithm plays a vital role in image formation. A natural scene containing spatial correlation can be reconstructed by maximum likelihood of all spatial locations constrained with a sparsity regularization penalty, and different penalties lead to different reconstructions. The l 1 -norm penalty of wavelet transform reconstructs major features but blurs edges and high-frequency details of the image. The total variational penalty preserves edges better; however, it induces a “staircase effect,” which degrades image quality. In this work, we proposed a hybrid penalty to reconstruct better edge features while suppressing the staircase effect by combining wavelet l 1 -norm and total variation into one penalty function. Results of numerical simulations indicate that the proposed hybrid penalty reconstructed better images, which have an averaged root mean square error of 12.83%, 5.68%, and 10.56% smaller than those of the images reconstructed by using only wavelet l 1 -norm penalty, total variation penalty, or recursive dyadic partitions method, respectively. Experimental results are in good agreement with the numerical ones, demonstrating the feasibility of the proposed hybrid penalty. Having been verified in a first-photon imaging system, the proposed hybrid penalty can be applied to other noise-removal optimization problems.

We propose a trust-region method that solves a sequence of linear integer programs to tackle integer optimal control problems regularized with a total variation penalty. The total variation penalty implies that the considered integer control problems admit minimizers. We introduce a local optimality concept for the problem, which arises from the infinite-dimensional perspective. In the case of a one-dimensional domain of the control function, we prove convergence of the iterates produced by our algorithm to points that satisfy first-order stationarity conditions for local optimality. We demonstrate the theoretical findings on a computational example.

We introduce novel image regularization penalties to overcome the practical problems associated with the classical total variation (TV) scheme. Motivated by novel reinterpretations of the classical TV regularizer, we derive two families of functionals involving higher degree partial image derivatives; we term these families as isotropic and anisotropic higher degree TV (HDTV) penalties, respectively. The isotropic penalty is the L(1) - L(2) mixed norm of the directional image derivatives, while the anisotropic penalty is the separable L(1) norm of directional derivatives. These functionals inherit the desirable properties of standard TV schemes such as invariance to rotations and translations, preservation of discontinuities, and convexity. The use of mixed norms in isotropic penalties encourages the joint sparsity of the directional derivatives at each pixel, thus encouraging isotropic smoothing. In contrast, the fully separable norm in the anisotropic penalty ensures the preservation of discontinuities, while continuing to smooth along the linelike features; this scheme thus enhances the linelike image characteristics analogous to standard TV. We also introduce efficient majorize-minimize algorithms to solve the resulting optimization problems. The numerical comparison of the proposed scheme with classical TV penalty, current second-degree methods, and wavelet algorithms clearly demonstrate the performance improvement. Specifically, the proposed algorithms minimize the staircase and ringing artifacts that are common with TV and wavelet schemes, while better preserving the singularities. We also observe that anisotropic HDTV penalty provides consistently improved reconstructions compared with the isotropic HDTV penalty.

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> penalty) or scattered (£1 penalty) predictive pattern with a very limited relevance. A penalty like Total Variation (TV) that exploits the natural 3D structure of the images can increase the spatial coherence of the weight map. However, TV penalization leads to non-smooth optimization problems that are hard to minimize. We propose an optimization framework that minimizes any combination of ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> , ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf> , and TV penalties while preserving the exact ℓ <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> penalty. This algorithm uses Nesterov's smoothing technique to approximate the TV penalty with a smooth function such that the loss and the penalties are minimized with an exact accelerated proximal gradient algorithm. We propose an original continuation algorithm that uses successively smaller values of the smoothing parameter to reach a prescribed precision while achieving the best possible convergence rate. This algorithm can be used with other losses or penalties. The algorithm is applied on a classification problem on the ADNI dataset. We observe that the TV penalty does not necessarily improve the prediction but provides a major breakthrough in terms of support recovery of the predictive brain regions.

Optimization-based region-of-interest reconstruction for X-ray computed tomography based on total variation and data derivative

Purpose:To develop a novel reconstruction method for X‐ray CT that can lead to accurate reconstruction at significantly reduced dose levels combining low X‐ray incident intensity and few views of projection data.Methods:The noise nature of the projection data at low X‐ray incident intensity was modeled and accounted by the weighted least‐squares (WLS) criterion. The total variation (TV) penalty was used to mitigate artifacts caused by few views of data. The first order primal‐dual (FOPD) algorithm was used to minimize TV in image domain, which avoided the difficulty of the non‐smooth objective function. The TV penalized WLS reconstruction was achieved by alternated FOPD TV minimization and projection onto convex sets (POCS) for data fidelity constraints. The proposed FOPD‐POCS method was evaluated using the FORBILD jaw phantom and the real cadaver head CT data.Results:The quantitative measures, root mean square error (RMSE) and contrast‐to‐noise ratio (CNR), demonstrate the superior denoising capability of WLS over LS‐based TV iterative reconstruction. The improvement of RMSE (WLS vs. LS) is 15%∼21% and that of CNR is 17%∼72% when the incident counts per ray are ranged from 1×105 to 1×103. In addition, the TV regularization can accurately reconstruct images from about 50 views of the jaw phantom. The FOPD‐POCS reconstruction reveals more structural details and suffers fewer artifacts in both the phantom and real head images. The FOPD‐POCS method also shows fast convergence at low X‐ray incident intensity.Conclusion:The new hybrid FOPD‐POCS method, based on TV penalized WLS, yields excellent image quality when the incident X‐ray intensity is low and the projection views are limited. The reconstruction is computationally efficient since the FOPD minimization of TV is applied only in the image domain. The characteristics of FOPD‐POCS can be exploited to significantly reduce radiation dose of X‐ray CT without compromising accuracy for diagnosis or treatment planning.

Sparse-view x-ray micro computed tomography (micro-CT) reconstruction algorithms via total variation (TV) optimize the data without introducing notable noise and artifacts, resulting in significant scanning time reduction while maintaining image quality. However, due to the piecewise constant assumption for the image, a conventional TV minimization often suffers from patchy artifacts in reconstructed images. Moreover, for lack of directional gradient in TV some directional information are lost. To obviate these drawbacks, in this study we develop a penalized weighted least-square (PWLS) strategy for micro-CT sparse-view image reconstruction by incorporating an adaptive weighted total variation in combination with an adaptive weighted diagonal total variation (AwTV+AwDTV) penalty term. The AwTV considers the vertical and horizontal gradients while the AwDTV uses the diagonal gradients. The associated weights which are defined based on the anisotropic edge properties of an image, are expressed as an exponential function and can be adaptively adjusted by the amount of the difference between voxel intensities to preserve the edge details. To evaluate the presented (AwTV+AwDTV)-PWLS algorithm, both qualitative and quantitative studies were performed by computer simulations and micro-CT data experiments. The Shepp-Logan phantom for computer simulation and the micro-CT water phantom and a rat skull for micro-CT experiments are employed to perform image reconstruction. To evaluate the performance of AwTV+AwDTV algorithm, we compared it with TV and AwTV reconstruction algorithms. The simulation results show that the presented (AwTV+AwDTV)-PWLS algorithm can achieve the lowest RMSE and highest PSNR, SSIM and MTF for different number of projections as compared to the AwTV and conventional TV algorithms. The micro-CT data results confirmed the superiority of the proposed (AwTV+AwDTV) method to the AwTV and TV methods for different number of projections.

We consider the problem of optimality, in a minimax sense, and adaptivity to the margin and to regularity in binary classification. We prove an oracle inequality, under the margin assumption (low noise condition), satisfied by an aggregation procedure which uses exponential weights. This oracle inequality has an optimal residual: (logM/n)κ/(2κ−1)where κ is the margin parameter, M the number of classifiers to aggregate and n the number of observations. We use this inequality first to construct minimax classifiers under margin and regularity assumptions and second to aggregate them to obtain a classifier which is adaptive both to the margin and regularity. Moreover, by aggregating plug-in classifiers (only logn), we provide an easily implementable classifier adaptive both to the margin and to regularity.

In this paper we propose a non-negative matrix factorization (NMF) model with piecewise-constant activation coefficients. This structure is enforced using a total variation penalty on the rows of the activation matrix. The resulting optimization problem is solved with a majorization-minimization procedure. The proposed algorithm is well suited to analyze data explained by underlying piecewise-constant sequences of states. Its properties are first illustrated using synthetic data. We then use it to solve a video structuring problem that involves both segmentation and clustering tasks. An improvement over a state-of-the-art temporally smoothed NMF algorithm of both clustering and segmentation quality measures is observed.

In this work, we apply non-convex, sparsity exploiting regularization techniques to image reconstruction in computed tomography (CT).We modify the well-known total variation (TV) penalty to use a non-convex smooth hard threshold (SHT) penalty as opposed to the typical e1 norm. The SHT penalty is different from the p <1 norms in that it is bounded above and has bounded gradient as its argument approaches the zero vector. We propose a re-weighting scheme utilizing the Chambolle-Pock (CP) algorithm in an attempt to solve a data-error constrained optimization problem utilizing the SHT penalty and call the resulting algorithm SHTCP. We then demonstrate the algorithm on sparse-view reconstruction of a simulated breast phantom with noiseless and noisy data and compare the converged images to those generated by a CP algorithm solving the analogous data-error constrained problem utilizing the TV. We demonstrate that SHTCP allows for more accurate reconstruction in the case of sparse-view noisy data and, in the case of noiseless data, allows for accurate reconstruction from fewer views than its TV counterpart.

Magnetic resonance (MR) imaging is considered as a very powerful imaging modality in clinical examination, but the process of image acquisition and transmission will be affected by noise, resulting in the degradation of imaging quality. In this paper, based on the Laplacian prior sparsity constraint and the nonconvex second order total variation (TV) penalty, we propose a MR images denoising model which consists of three terms. Specifically, in the first term, we use the L2-norm as the fidelity term to control the proximity between the observed image and the recovered MR image. Then, we introduce the Laplacian sparse prior constraint as the second term to mitigate the staircase artifacts in the recovered image. In the third term, we adopt the nonconvex second-order TV penalty to preserve important textures and edges. Finally, we use the alternating direction method of multipliers to solve the corresponding minimization problem. Comparative experiments on clinical data demonstrate the effectiveness of our approach in terms of PSNR and SSIM values.

We extend the novel higher degree total variation (HDTV) image regularization penalties to 3-D signals. These penalties generalize the popular total variation (TV) penalty to incorporate higher degree image derivatives. We adapt a fast alternating minimization algorithm designed for solving 2-D image recovery problems with HDTV regularization to the 3D setting. Numerical experiments on the compressed sensing recovery of 3-D magnetic resonance images show that HDTV and generalized HDTV improve the image quality significantly compared with TV. We also investigate the relationship between the recently introduced Hessian Schatten-norms and HDTV.