Two Variational Models for Image Denoising Using Jacobian of Normals

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

We propose a new variational model in Sobolev-Orlicz spaces with non-standard growth conditions of the objective functional and discuss its applications to the simultaneous contrast enhancement and denoising of color images. The characteristic feature of the proposed model is that we deal with a constrained non-convex minimization problem that lives in variable Sobolev-Orlicz spaces where the variable exponent is unknown a priori and it depends on a particular function that belongs to the domain of the objective functional. In contrast to the standard approach, we do not apply any spatial regularization to the image gradient. We discuss the consistency of the variational model, give the scheme for its regularization, derive the corresponding optimality system, and propose an iterative algorithm for practical implementations.

Anisotropic total variation model for removing oblique stripe noise in remote sensing image

Contour and texture preservation underwater image restoration via low-rank regularizations

The decentralized approach is popular for the collaborative collision avoidance of connected vehicles in many scenarios. By modeling the task as a collaborative optimal control problem, Lagrangian methods are widely used to decouple the constraints and enable the decentralized solution. However, potential constraint-violating behavior will lead to oscillations during the Lagrangian update, resulting in more iterations and lower real-time efficiency. Existing methods generally neither address this shortcoming, nor explore the Lagrangian update mechanism. This study takes a control perspective, and solves this collaborative optimal control problem based on an extension of the Alternating Directions Method of Multipliers (ADMM) algorithm by performing the iteration update with a Proportional-Integral-(PI-) regulated controller. The link between the Lagrangian optimization and the PI controller improves the convergence performance during iterations. Simulation results in traffic intersection scenarios demonstrate the advantage of the proposed approach.

High-order TVL1-based images restoration and spatially adapted regularization parameter selection

In this paper, a novel extended form of multivariate variational mode decomposition (MVMD) method to multigroup data named as grouped MVMD (GMVMD) is proposed. GMVMD is distinct from MVMD as it extracts common frequencies with strong correlations among regional channels. Firstly, GMVMD utilizes a new clustering algorithm named as frequencies grouping algorithm to classify the nearest common frequencies among all channels to specified groups. Secondly, a generic variational optimization model which is extended from MVMD is formulated. Thirdly, alternating direction method of multipliers (ADMM) is utilized to obtain optimal solution of GMVMD model. The proposed method introduces an extra parameter to decide the number of clusterings which need to be specified by the user. The effectiveness and superiority of the algorithm are demonstrated on a series of experiments. The utility of GMVMD is verified by grouping real-world electroencephalogram (EEG) data having similar center frequencies successfully. GMVMD outperforms MVMD in mode-alignment, signal reduction error and et al. Significance: GMVMD can obtain more accurate center frequencies and less signal reduction error than MVMD.

Foreword

Two new low rank tensor completion methods based on sum nuclear norm

To solve the problem that noise seriously affects the online monitoring of parts signals of outdoor machinery, this paper proposes a signal reconstruction method integrating deep neural network and compression sensing, called ADMM-1DNet, and gives a detailed online vibration signal monitoring scheme. The basic approach of the ADMM-1DNet network is to map the update steps of the classical Alternating Direction Method of Multipliers (ADMM) into the deep network architecture with a fixed number of layers, and each phase corresponds to an iteration in the traditional ADMM. At the same time, what differs from other unfolded networks is that ADMM-1DNet learns a redundant analysis operator, which can reduce the impact of outdoor high noise on reconstruction error by improving the signal sparse level. The implementation scheme includes the field operation of mechanical equipment and the operation of the data center. The empirical network trained by the local data center conducts an online reconstruction of the received outdoor vibration signal data. Experiments are conducted on two open-source bearing datasets, which verify that the proposed method outperforms the baseline method in terms of reconstruction accuracy and feature preservation, and the proposed implementation scheme can be adapted to the needs of different types of vibration signal reconstruction tasks.

An efficient non-convex total variation approach for image deblurring and denoising

The alternating direction method of multipliers (ADMM), also well known as a special split Bregman algorithm in imaging, is being popularly used in many areas including the image processing field. One useful modification is the symmetric version of the original ADMM, which updates the Lagrange multiplier twice at each iteration and thus the variables are treated in a symmetric manner. The symmetric version of ADMM, however, is not necessarily convergent. It was recently found that the convergence of symmetric ADMM can be sufficiently ensured if both the step sizes for updating the Lagrange multiplier are shrunk conservatively. Despite the theoretical significance in ensuring convergence, however, smaller step sizes should be strongly avoided in practice. On the contrary, we actually have the desire of seeking larger step sizes whenever possible in order to accelerate the numerical performance. Another technique leading to numerical acceleration of ADMM is enlarging its step size by a constant originally p...

The performance of adaptive beamforming suffers from the imprecise knowledge of the array or signal of interest. Many efforts have been made to develop various beamformers and increase their robustness to these unknown errors. This study proposes a new robust adaptive beamformer based on the alternating direction method of multipliers (ADMM). The proposed method is introduced first as an optimisation problem to estimate the steering vector (SV). Owing to its non-convex property, it is converted to a real-valued condition problem by introducing auxiliary variables. Then, the optimisation problem is reformulated as the augmented Lagrangian function. Finally, the estimated SV can be solved by the ADMM. Numerical results prove the effectiveness of the proposed method.

Multi-task feature selection with sparse regularization to extract common and task-specific features

The least-squares is a common and important method in linear regression. However, it often leads to overfitting phenomenon as dealing with high-dimensional problems, and various regularization schemes regarding prior information for specific problems are studied to make up such a deficiency. In the sense of Kendall’s [Formula: see text] from the community of nonparametric analysis, we establish a new model wherein the ordinary least-squares is equipped with perfect positive correlation constraint, sought to maintain the concordance of the rankings of the observations and the systematic components. By sorting the observations into an ascending order, we reduce the perfect positive correlation constraint into a linear inequality system. The resulting linearly constrained least-squares problem together with its dual problem is shown to be solvable. In particular, we introduce a mild assumption on the observations and the measurement matrix which rules out the zero vector from the optimal solution set. This indicates that our proposed model is statistically meaningful. To handle large-scale instances, we propose an efficient alternating direction method of multipliers (ADMM) to solve the proposed model from the dual perspective. The effectiveness of our model compared to ordinary least-squares is evaluated in terms of rank correlation coefficient between outputs and the systematic components, and the efficiency of our dual algorithm is demonstrated with the comparison to three efficient solvers via CVX in terms of computation time, solution accuracy and rank correlation coefficient.