Bayesian Approach to Variable Splitting Forward Models

The fast compressed sensing (CS) reconstruction for natural image processing aims to infer the original pixel vector from randomized measurements as correctly as possible. Current first-order proximal mainstream schemes, including alternating direction methods of multipliers (ADMM) and the iterative shrinkage thresholding algorithm (ISTA), have advantages in convergence and stability at the expense of computing speed and complex tuning strategy. This paper proposes the deep iterative shrinkage network for real-time applications, where network layers replace the iterative optimization steps. The hyperparameters associated with shrinkage thresholds and step sizes are adjusted by standard deep learning updates. Numerical results suggest that the proposed method has flexibility and effectiveness in image reconstruction with different compression ratios while preserving the computational simplicity of end-to-end methods.

This thesis discusses inverse problems in interpreting satellite observations. When we solve the inverse problem in atmospheric remote sensing, we make simplifications and assumptions about the atmosphere. In addition to the observation noise, this contributes to the uncertainty of the solution. We focus on statistical inverse problems which naturally take into account uncertainties in the solution. Uncertainty analysis is important both in utilizing satellite data and in developing algorithms. In this work, we solve the inverse problem based on the Bayesian method in looking for the best match between satellite observations and the state of the atmosphere related to aerosol load and vertical ozone profiles. One important goal was to achieve a more realistic uncertainty estimate by considering forward model uncertainty, i.e., model discrepancy. Another objective is to acknowledge uncertainty in aerosol model selection. In this thesis we developed a novel method to obtain aerosol optical depth (AOD) based on the Bayesian inference and where a concept of model evidence is used for model comparison. We applied the Bayesian model averaging technique for combining inference from several aerosol optical models. We utilized the residuals of the model fits and Gaussian processes for characterizing the model discrepancy. We applied the method for the observations of the OMI and TROPOMI instruments and compared the resulting AOD with the ground-based aerosol measurements. Acknowledging of the model discrepancy as well as difficulty in the model selection are reflected in the uncertainty of the AOD estimate. The Bayesian approach can give additional information about the sources of uncertainty, assists in the uncertainty analysis of the results, and may guide improving the retrieval algorithms. This thesis also assessed the quality of the operational satellite ozone profile product depending on season, latitude and altitude. The comparison of GOME-2/Metop-A ozone profiles in a global scale concerning the years 2008-2011 was done against the reference data of three satellites. Based on the results, we assessed the reliability of the satellite ozone profile data, also considering the ageing of the instrument and identified the potential points of improvement.

The Alternating Direction Method of Multipliers(ADMM) is an important method for machine learning. A large number of stochastic versions of ADMM continue to emerge. But almost all the algorithm focused on the Steepest Descent, which cause slow convergence rate. In our work, we propose a new stochastic alternative direction method of multipliers with conjugate gradient(SCG-ADMM), and prove its convergence rate in ergodic sense. Conjugate gradient descent has a better performance in traditional cases. Experiments on LASSO and Logistics Regression show that our proposed algorithm can achieve state-of-the-art performance in real applications. And we use the linear search method to replace the manual setting of the step length.

Dual-energy computed tomography (DECT) makes it possible to get two fractions of basis materials without segmentation. One is the soft-tissue equivalent water fraction and the other is the hard-matter equivalent bone fraction. Practical DECT measurements are usually obtained with polychromatic x-ray beams. Existing reconstruction approaches based on linear forward models without counting the beam polychromaticity fail to estimate the correct decomposition fractions and result in beam-hardening artifacts (BHA). The existing BHA correction approaches either need to refer to calibration measurements or suffer from the noise amplification caused by the negative-log preprocessing and the ill-conditioned water and bone separation problem. To overcome these problems, statistical DECT reconstruction approaches based on nonlinear forward models counting the beam polychromaticity show great potential for giving accurate fraction images. This work proposes a full-spectral Bayesian reconstruction approach which allows the reconstruction of high quality fraction images from ordinary polychromatic measurements. This approach is based on a Gaussian noise model with unknown variance assigned directly to the projections without taking negative-log. Referring to Bayesian inferences, the decomposition fractions and observation variance are estimated by using the joint maximum a posteriori (MAP) estimation method. Subject to an adaptive prior model assigned to the variance, the joint estimation problem is then simplified into a single estimation problem. It transforms the joint MAP estimation problem into a minimization problem with a nonquadratic cost function. To solve it, the use of a monotone conjugate gradient algorithm with suboptimal descent steps is proposed. The performance of the proposed approach is analyzed with both simulated and experimental data. The results show that the proposed Bayesian approach is robust to noise and materials. It is also necessary to have the accurate spectrum information about the source-detector system. When dealing with experimental data, the spectrum can be predicted by a Monte Carlo simulator. For the materials between water and bone, less than 5% separation errors are observed on the estimated decomposition fractions. The proposed approach is a statistical reconstruction approach based on a nonlinear forward model counting the full beam polychromaticity and applied directly to the projections without taking negative-log. Compared to the approaches based on linear forward models and the BHA correction approaches, it has advantages in noise robustness and reconstruction accuracy.

.The plug-and-play framework makes it possible to integrate advanced image denoising priors into optimization algorithms to efficiently solve a variety of image restoration tasks generally formulated as maximum a posteriori (MAP) estimation problems. The plug-and-play alternating direction method of multipliers (ADMM) and the regularization by denoising (RED) algorithms are two examples of such methods that made a breakthrough in image restoration. However, the former plug-and-play approach only applies to proximal algorithms. And while the explicit regularization in RED can be used in various algorithms, including gradient descent, the gradient of the regularizer computed as a denoising residual leads to several approximations of the underlying image prior in the MAP interpretation of the denoiser. We show that it is possible to train a network directly modeling the gradient of a MAP regularizer while jointly training the corresponding MAP denoiser. We use this network in gradient-based optimization methods and obtain better results compared to other generic plug-and-play approaches. We also show that the regularizer can be used as a pretrained network for unrolled gradient descent. Lastly, we show that the resulting denoiser allows for a better convergence of the plug-and-play ADMM.Keywordsinverse problemsregularizationplug-and-play priorgradient descentMSC codes62H3568U1094A0868T99

In the last decades, unsupervised deep learning based methods have caught researchers' attention, since in many real applications, such as medical imaging, collecting a large amount of training examples is not always feasible. Moreover, the construction of a good training set is time consuming and hard because the selected data have to be enough representative for the task. In this paper, we focus on the Deep Image Prior (DIP) framework and we propose to combine it with a space-variant Total Variation regularizer with an automatic estimation of the local regularization parameters. Differently from other existing approaches, we solve the arising minimization problem via the flexible Alternating Direction Method of Multipliers (ADMM). Furthermore, we provide a specific implementation also for the standard isotropic Total Variation. The promising performances of the proposed approach, in terms of PSNR and SSIM values, are addressed through several experiments on simulated as well as real natural and medical corrupted images.

We consider a global variable consensus ADMM algorithm for solving large-scale PDE parameter estimation problems asynchronously and in parallel. To this end, we partition the data and distribute the resulting subproblems among the available workers. Since each subproblem can be associated with different forward models and right-hand-sides, this provides ample options for tailoring the method to different applications including multi-source and multi-physics PDE parameter estimation problems. We also consider an asynchronous variant of consensus ADMM to reduce communication and latency. Our key contribution is a novel weighting scheme that empirically increases the progress made in early iterations of the consensus ADMM scheme and is attractive when using a large number of subproblems. This makes consensus ADMM competitive for solving PDE parameter estimation, which incurs immense costs per iteration. The weights in our scheme are related to the uncertainty associated with the solutions of each subproblem. We exemplarily show that the weighting scheme combined with the asynchronous implementation improves the time-to-solution for a 3D single-physics and multiphysics PDE parameter estimation problems.

Integrated sensing and communication (ISAC) has emerged as a promising paradigm for next-generation (6G) wireless networks, unifying radar sensing and communication on a shared hardware platform. This paper proposes a dynamic array partitioning framework for monostatic ISAC systems to fully exploit available spatial degrees of freedom (DoFs) and reconfigurable antenna topologies, enhancing sensing performance in complex scenarios. We first establish a theoretical foundation for our work by deriving Bayesian Cramér-Rao bounds (BCRBs) under prior distribution constraints for heterogeneous target models, encompassing both point-like and extended targets. Building on this, we formulate a joint optimization framework for transmit beamforming and dynamic array partitioning to minimize the derived BCRBs for direction-of-arrival (DOA) estimation. The optimization problem incorporates practical constraints, including multi-user communication signal-to-interference-plus-noise ratio (SINR) requirements, transmit power budgets, and array partitioning feasibility conditions. To address the non-convexity of the problem, we develop an efficient alternating optimization algorithm combining the alternating direction method of multipliers (ADMM) with semi-definite relaxation (SDR). We also design novel maximum a posteriori (MAP) DOA estimation algorithms specifically adapted to the statistical characteristics of each target model. Extensive simulations illustrate the superiority of the proposed dynamic partitioning strategy over conventional fixed-array architectures across diverse system configurations.

Maximum a-posteriori (MAP) estimation is an important task in many applications of probabilistic graphical models. Although finding an exact solution is generally intractable, approximations based on linear programming (LP) relaxation often provide good approximate solutions. In this paper we present an algorithm for solving the LP relaxation optimization problem. In order to overcome the lack of strict convexity, we apply an augmented Lagrangian method to the dual LP. The algorithm, based on the alternating direction method of multipliers (ADMM), is guaranteed to converge to the global optimum of the LP relaxation objective. Our experimental results show that this algorithm is competitive with other state-of-the-art algorithms for approximate MAP estimation.

Reversible data hiding (RDH) is desirable in applications where both the hidden message and the cover image need to be recovered without loss. Among many RDH approaches is prediction-error expansion (PEE), containing two steps: i) prediction of a target pixel value, and ii) embedding according to the value of the prediction error. In general, higher prediction performance leads to larger embedding capacity and/or lower signal distortion. Leveraging on recent advances in graph signal processing (GSP), we pose pixel prediction as a graph-signal restoration problem, and design non-local graph-based prediction schemes where the appropriate edge weights of the underlying graph are computed using a similar patch searched in a semi-local neighborhood. Specifically, for each candidate patch, we first examine eigenvalues of its structure tensor to estimate its local smoothness. If sufficiently smooth, we pose a maximum a posteriori (MAP) problem using either a quadratic Laplacian regularizer or a graph total variation (GTV) term as signal prior. While the MAP problem using the first prior has a closed-form solution, we design an efficient algorithm for the second prior using alternating direction method of multipliers (ADMM) with nested proximal gradient descent. Finally, hidden message will be embedded into the image according to the resulting prediction errors. Experimental results show that with better quality GSP-based prediction, at low capacity the visual quality of the embedded image exceeds state-of-the-art methods noticeably.

Recently, fusing low-resolution hyperspectral images (LR-HSIs) with high-resolution multispectral images (HR-MSIs) to obtain high-resolution HSI (HR-HSI) has become an emerging study. In this letter, Bayesian nonlocal canonical polyadic (CP) factorization (BNCPF) is proposed for fusing LR-HSI with HR-MSI, which applies CP factorization on the nonlocal tensors of HR-HSI. Compared with the vanilla scheme of applying CP factorization on HR-HSI, the nonlocal tensors reveal balanced low-rank properties along different modes, and thus CP factorization can better capture their intrinsic low-rankness. To avoid the immense CP-ranks selection on the nonlocal tensors, we develop a sparse Bayesian framework for automatic rank determination. For parameters estimation, we adopt the alternating direction method of multipliers (ADMMs) for the maximum a posteriori (MAP) estimator optimization. Experimental results verify the superiority and rank-robustness of the proposed method.

Obtaining a reasonable subsurface model both qualitatively and quantitatively from recorded GPR data has always been a challenge. Traditional methods for this task are based on an approximate forward model and hence use a limited part of the data for the inversion, or they require a quite good prior knowledge of the subsurface model. The forward model that governs the GPR electromagnetic waves is the same as the one governing visco-acoustic wave propagation. Recent advances in seismic full-waveform inversion (FWI) allows estimating quantitative high-resolution subsurface images from the whole recorded data. In particular, FWI method based on the alternating-direction method of multipliers (ADMM) has shown the ability to reduce the sensitivity of the inversion to the initial model and also flexibility to remedy the ill-posedness of the problem by injecting prior information into the inversion. In this abstract, we apply this approach to GPR data to reconstruct both dielectric permittivity and electrical conductivity. Numerical tests on a simple and a more realistic models confirm good performance of the ADMM-based FWI for GPR data. Note: This paper was accepted into the Technical Program but was not presented at IMAGE 2022 in Houston, Texas.

Spectral computed tomography (CT) relies on the spectral dependence of X-ray attenuation coefficients to separate projection measurements into more than two energy bins. Such data can be used to unveil tomographic material characterization - key in national security and medical imaging. This paper explores a radical departure from conventional methods used in spectral imaging. It relies on K-edge coded apertures to create spatially and spectrally coded, lower-dose, X-ray bundles that interrogate specific voxels of the object. The new approach referred to as compressive spectral X-ray imaging (CSXI) uses low-cost standard X-ray integrating detectors and acquires compressive measurements, which enable the reconstruction of energy binned images from fewer measurements. Various spectral and spatial coding strategies for structured illumination are explored. Subsampling in CSXI is accomplished by either view angle spectral subsampling, spatial subsampling enabled by block-unblock coded apertures placed at the source or detector side, or both. The careful design of subsampling strategies, spectral filters, coded apertures, and their placement, are shown to be critical for the quality of tomographic image reconstruction. The forward imaging model of CSXI, which is a non-linear ill-posed problem, is analyzed and a multi-stage algorithm is developed to address the estimation of the energy binned sinograms from the integrating detector measurements. Then, an Alternating Direction Method of Multipliers (ADMM) is used to solve a joint sparse and low-rank optimization problem for reconstruction that exploits the structure of the spectral X-ray data cube.

We study the sparse subspace clustering problem in presence of both sparse outliers and Gaussian additive noise based on data sparse self-representation. We propose a convex optimization problem which does not only induce sparsity on the representation coefficients and the outliers, but also adopts a square-root penalty to improve the robustness against Gaussian noise. An algorithm based on alternating direction method of multipliers (ADMM) is then devised as a solver for the proposed problem. As a real application, the proposed model and algorithm are applied in motion segmentation. The performances are demonstrated and analyzed by synthetic data, and more importantly, the effectiveness is verified by some real data. Compared with the reference method, numerical results show that the new method achieves higher cluster accuracy and that the choice of the parameter can be less sensitive to the noise level. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>

Real-time distributed trajectory planning for mobile robots