A Distributed, Asynchronous, and Incremental Algorithm for Nonconvex Optimization: An ADMM Approach

In this paper, we propose two low-complexity optimization methods to reduce peak-to-average power ratio (PAPR) values of orthogonal frequency division multiplexing (OFDM) signals via alternating direction method of multipliers (ADMM). First, we formulate a nonconvex signal distortion optimization model based on minimizing data carrier distortion such that the constraints are placed on PAPR and the power of free carriers. Second, to obtain the model's approximate optimal solution efficiently, we design two low-complexity ADMM algorithms, named ADMMDirect and ADMM-Relax respectively. Third, we show that, in ADMM-Direct/-Relax, all the optimization subproblems can be solved semi-analytically and the computational complexity in each iteration is roughly (9(IN log <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> IN), where I and N are oversampling factor and carrier number respectively. Moreover, we show that the resulting solution of ADMM-direct is guaranteed to be some Karush-Kuhn-Tucker (KKT) point of the nonconvex model when the iteration algorithm is convergent. For ADMMRelax, we prove that it has theoretically-guaranteed convergence and can approach arbitrarily close to some KKT point of the model if proper parameters are chosen. Simulation results demonstrate the effectiveness of the proposed approaches.

This paper develops efficient algorithms for distributed average consensus with quantized communication using the alternating direction method of multipliers (ADMM). When rounding quantization is employed, a distributed ADMM algorithm is shown to converge to a consensus within 3 + ⌈log1+δ Ω⌉ iterations where δ > 0 depends on the network topology and O is a polynomial of the quantization resolution, the agents' data and the network topology. A tight upper bound on the consensus error is also obtained, which depends only on the quantization resolution and the average degree of the graph. This bound is much preferred in large scale networks over existing algorithms whose consensus errors are increasing in the range of agents' data, the quantization resolution, and the number of agents. To minimize the consensus error, our final algorithm uses dithered quantization to obtain a good starting point and then adopts rounding quantization to reach a consensus. Simulations show that the consensus error of this algorithm is typically less than one quantization resolution for all connected networks with agents' data of arbitrary magnitudes.

We discuss an online decentralized decision making problem where the agents are coupled with affine inequality constraints. Alternating Direction Method of Multipliers (ADMM) is used as the computation engine and we discuss the convergence of the algorithm in an online setting. To be specific, when decisions have to be made sequentially with a fixed time step, there might not be enough time for the ADMM to converge before the scenario changes and the decision needs to be updated. In this case, a suboptimal solution is employed and we analyze the optimality gap given the convergence condition. Moreover, in many cases, the decision making problem changes gradually over time. We propose a warm-start scheme to accelerate the convergence of ADMM and analyze the benefit of the warm-start. The proposed method is demonstrated in a decentralized multiagent control barrier function problem with simulation.

Compressed sensing (CS) can achieve high resolution inverse synthetic aperture radar (ISAR) imaging of moving targets with limited measurements. Recently, alternating direction method of multipliers (ADMM) has been introduced to solve the optimization problem for one dimensional (1D) sparse signal recovery. The main drawback of 1D sparsity-driven algorithms are the high memory usage and the computational complexity. Thus, in this paper a novel two dimensional (2D) ADMM approach is presented which can be directly applied to the ISAR model in matrix form, and needs lower memory and computations compared to the 1D algorithm. Moreover, the performance of the 2D-ADMM method is better than the 2D smoothed L0 (2D-SL0) and 2D gradient projection sequential order one negative exponential (2D-GP-SOONE) algorithms in different signal-to-noise ratio (SNR) conditions and sampling rates. Joint simulations and measured data results based on real data of Yak-42 airplane, validate the superiority of the proposed approach.

Recent advances in neural-network architecture allow for seamless integration of convex optimization problems as differentiable layers in an end-to-end trainable neural network. Integrating medium and large scale quadratic programs into a deep neural network architecture, however, is challenging as solving quadratic programs exactly by interior-point methods has worst-case cubic complexity in the number of variables. In this paper, we present an alternative network layer architecture based on the alternating direction method of multipliers (ADMM) that is capable of scaling to moderate sized problems with 100–1000 decision variables and thousands of training examples. Backward differentiation is performed by implicit differentiation of a customized fixed-point iteration. Simulated results demonstrate the computational advantage of the ADMM layer, which for medium scale problems is approximately an order of magnitude faster than the state-of-the-art layers. Furthermore, our novel backward-pass routine is computationally efficient in comparison to the standard approach based on unrolled differentiation or implicit differentiation of the KKT optimality conditions. We conclude with examples from portfolio optimization in the integrated prediction and optimization paradigm.

We consider the problem of restoring images corrupted by Poisson noise. Under the framework of maximum a posteriori estimator, the problem can be converted into a minimization problem where the objective function is composed of a Kullback-Leibler (KL)-divergence term for the Poisson noise and a total variation (TV) regularization term. Due to the logarithm function in the KL-divergence term, the non-differentiability of TV term and the positivity constraint on the images, it is not easy to design stable and efficiency algorithm for the problem. Recently, many researchers proposed to solve the problem by alternating direction method of multipliers (ADMM). Since the approach introduces some auxiliary variables and requires the solution of some linear systems, the iterative procedure can be complicated. Here we formulate the problem as two new constrained minimax problems and solve them by Chambolle-Pocks first order primal-dual approach. The convergence of our approach is guaranteed by their theory. Comparing with ADMM approaches, our approach requires about half of the auxiliary variables and is matrix-inversion free. Numerical results show that our proposed algorithms are efficient and outperform the ADMM approach.

Purpose:To propose a new parallel‐friendly optimization algorithm, based on alternating direction method of multipliers (ADMM), for solving optimization problems emerging in treatment planning and imagingMethods:ADMM, emerging as a powerful tool for distributed optimization, is employed here for IMRT treatment planning. We modify the existing ADMM by integrating that with two optimization techniques known as Barzilai‐Borwein gradient method and line search. We apply original and integrated ADMM, with various penalty parameter values, on three IMRT treatment planning cases and we compare their performance. Each algorithm terminates once it found a solution within the relative error of 1E‐4 from the optimal solution. We also compare the performance of original and integrated ADMM against the commercial optimization software CPLEX.Results:Our experiments on three different cancer sites (prostate, head and neck, and GYN) indicate that integrated ADMM is much faster than original ADMM (5.8 times faster on average). Moreover, while the performance of original ADMM heavily depends on the selected penalty parameter, integrated ADMM performs very robust against the penalty parameter. Compared to the commercial software CPLEX, integrated ADMM turns out to be around 4–7 times faster.Conclusion:Integrated ADMM is a very efficient optimization algorithm to cope with the large‐scale optimization problems. It is a parallel‐friendly optimization algorithm and provides an idea platform for cloud‐based treatment planning and imaging.

Distributed optimization has gained popularity in many areas because it can to cope with the increasing volume of data, and the corresponding demand for computational resources. A popular distributed solver is the alternating direction method of multipliers (ADMM). Fully decentralized (D) ADMM arises when node-to-node communications have to abide by the underlying network connectivity. DADMM’s convergence however, slows down as the network diameter grows large. To deal with this challenge, the recently proposed hybrid (H) ADMM provides considerable performance boost over DADMM by exploiting local topology information. But HADMM only applies to unweighted graphs. The present contribution develops a weighted hybrid (WH) consensus-based ADMM approach that can deal with weighted graphs. The resultant scheme further improves the performance of HADMM through graph-aware weight tuning. Theoretical analysis offers convergence guarantees and establishes linear convergence rate, while numerical tests on various graphs demonstrate the WHADMM merits.

An efficient non-convex total variation approach for image deblurring and denoising

We consider a dynamic time division duplexing wireless network and propose a distributed coordinated beamforming algorithm based on Alternating Direction Method of Multipliers (ADMM) technique assuming the availability of perfect channel state information. Our design objective is to minimize the sum transmit power at the base stations subject to minimum signal-to-interference-plus-noise ratio (SINR) constraints for downlink mobile stations and a maximum interference power threshold for uplink mobile stations. First, we propose a centralized algorithm based on the relaxed Semidefinite Programming (SDP) technique. To obtain the beamforming solution in a distributed way, we further propose a distributed coordinated beamforming algorithm using the ADMM technique. Detailed simulation results are presented to examine the effectiveness of the proposed algorithms. It is shown that the proposed algorithm achieves better performance in terms of the design objective and converges faster than the reference algorithm based on primal decomposition.

Recently, several convergence rate results for Douglas-Rachford splitting and the alternating direction method of multipliers (ADMM) have been presented in the literature. In this paper, we show global linear convergence rate bounds for Douglas-Rachford splitting and ADMM under strong convexity and smoothness assumptions. We further show that the rate bounds are tight for the class of problems under consideration for all feasible algorithm parameters. For problems that satisfy the assumptions, we show how to select step-size and metric for the algorithm that optimize the derived convergence rate bounds. For problems with a similar structure that do not satisfy the assumptions, we present heuristic step-size and metric selection methods.

We focus on the problem of minimizing a finite sum f(x) = Σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">=</sub> <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> (x) of n functions f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> , where f <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sub> are convex and available only locally to an agent i. The n agents are connected in a directed network G (V, E). In this article, we present the Directed-Distributed Alternating Direction Method of Multiplier (D-DistADMM) Algorithm, which is an Alternating Direction Method of Multiplier (ADMM) based scheme and utilizes a finite-time "approximate" consensus method to solve the above optimization problem distributively. At each iteration of the proposed scheme the agents solve their local optimization problem and utilize an approximate consensus protocol to update a local estimate of the global optimization variable. We show that for convex and not-necessarily differentiable objective functions the proposed D-DistADMM method converges at a rate O(1/k), where k is the iteration counter, in terms the difference between the Lagrangian function evaluated at any iteration k of the D-DistADMM algorithm and the optimal solution. We further demonstrate the features of our algorithm by solving a distributed least squares problem.

Linear programming is the core problem of various operational research problems. The dominant approaches for linear programming are simplex and interior point methods. In this paper, we show that the alternating direction method of multipliers (ADMM), which was proposed long time ago while recently found more and more applications in a broad spectrum of areas, can also be easily used to solve the canonical linear programming model. The resulting per-iteration complexity is O(mn) where m is the constraint number and n the variable dimension. At each iteration, there are m subproblems that are eligible for parallel computation; each requiring only O(n) flops. There is no inner iteration as well. We thus introduce the new ADMM approach to linear programming, which may inspire deeper research for more complicated scenarios with more sophisticated results.

This article investigates the problem of frequency-invariant (FI) beamforming of wideband multiple-input multiple-output (MIMO) radars and presents two design criteria to synthesize wideband FI beampatterns under peak-to-average power ratio (PAPR) and nulling constraints. Precisely, the two criteria are to apply the least-squares and absolute errors, respectively, to approximate a desired transmit beampattern. Both criteria result in nonconvex constrained optimization problems. For the former, the solution is derived via reformulating the initial problem into a quadratic form, which can be solved by alternating direction method of multipliers (ADMM). For the latter, the design problem is formulated into an <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${l_{1}}$</tex-math></inline-formula> -norm real-valued form, which is solved iteratively by using a nested ADMM (N-ADMM) algorithm. Moreover, both the convergence and computational complexities of these two algorithms are discussed. Numerical simulations show that the obtained beampatterns by the proposed algorithms can well approximate the given beampattern under the required constraints.

Integrated sensing and communication (ISAC) has been envisioned to play a more important role in future wireless networks. However, the design of ISAC networks is challenging, especially when there are multiple communication and sensing (C&S) nodes and multiple sensing targets. We investigate a multi-base station (BS) ISAC network in which multiple BSs equipped with multiple antennas simultaneously provide C&S services for multiple ground communication users (CUs) and targets. To enhance the overall performance of C&S, we formulate a joint user association (UA) and multi-BS transmit beamforming optimization problem with the objective of maximizing the total sum rate of all CUs while ensuring both the minimum target detection and parameter estimation requirements in terms of the radar signal-to-noise ratio (SNR) and the Cramér-Rao bound (CRB), respectively. To efficiently solve the highly non-convex mixed integer nonlinear programming (MINLP) optimization problem, we propose an alternating optimization (AO)-based algorithm that decomposes the problem into two sub-problems, i.e., UA optimization and multi-BS transmit beamforming optimization. Inspired by the huge potential of large language models (LLMs) for prediction and inference, we propose a unified framework integrating LLMs with convex-based optimization methods to benefit from the theoretical rigor and convergence guarantees of convex-based methods, and the adaptability and flexibility of LLMs. First, we propose a comprehensive design of prompt engineering based on in-context, few-shot, chain of thought, and self-reflection techniques to guide LLMs in solving the binary integer programming UA optimization problem. Second, we utilize convex-based optimization methods to handle the non-convex beamforming optimization problem based on fractional programming (FP), majorization minimization (MM), and the alternating direction method of multipliers (ADMM) with an optimized UA from LLMs. Numerical results demonstrate that our proposed LLM-enabled AO-based algorithm achieves fast convergence and near upper-bound performance with the GPT-o1 model, outperforming various benchmark schemes, which shows the advantages of integrating LLMs into convex-based optimization for wireless networks.