Online distributed ADMM via dual averaging

Data collection plays an important role in wireless sensor networks. Recovery of spatio-temporal data from incomplete sensing data is vital to the network lifetime. Many works have utilized the spatial and temporal correlations to achieve satisfactory data recovery results. However, these methods introduce large computational overhead at the fusion center. In this paper, we develop an ADMM-Net framework for correlated spatio-temporal data recovery. Both the spatial correlation and temporal correlation of sensing data are considered in a convex optimization problem, which is solved by the alternating direction method of multipliers (ADMM) algorithm. We then unfold the ADMM algorithm into a fixed-length neural network that reduces the iterations dramatically and does not require additional location information of nodes. Experimental results on a realworld dataset demonstrate that the proposed method can achieve faster convergence speed than the baseline ADMM algorithm with slight accuracy loss.

The alternating direction method of multipliers (ADMM) has recently been recognized as a promising approach for large-scale machine learning models. However, very few results study ADMM from the aspect of communication costs, especially jointly with running time. In this letter, we investigate the communication efficiency and running time of ADMM in solving the consensus optimization problem over decentralized networks. We first review the effort of random walk ADMM (W-ADMM), which reduces communication costs at the expense of running time. To accelerate the convergence speed of W-ADMM, we propose the parallel random walk ADMM (PW-ADMM) algorithm, where multiple random walks are active at the same time. Moreover, to further reduce the running time of PW-ADMM, the intelligent parallel random walk ADMM (IPW-ADMM) algorithm is proposed through integrating the \textit{Random Walk with Choice} with PW-ADMM. By numerical results from simulations, we demonstrate that the proposed algorithms can be both communication efficient and fast in running speed compared with state-of-the-art methods.

The distributed alternating direction method of multipliers (ADMM) is an effective algorithm for solving large-scale optimization problems. However, its high communication cost limits its scalability. An asynchronous lazy ADMM algorithm based on hierarchical sparse allreduce communication mode (HSAC-ALADMM) is proposed to reduce the communication cost of the distributed ADMM: firstly, this paper proposes a lazily aggregate parameters strategy to filter the transmission parameters of the distributed ADMM, which reduces the payload of the node per iteration. Secondly, a hierarchical sparse allreduce communication mode is tailored for sparse data to aggregate the filtered transmission parameters effectively. Finally, a Calculator-Communicator-Manager framework is designed to implement the proposed algorithm, which combines the asynchronous communication protocol and the allreduce communication mode effectively. It separates the calculation and communication by multithreading, thus improving the efficiency of system calculation and communication. Experimental results for the L1-regularized logistic regression problem with public datasets show that the HSAC-ALADMM algorithm is faster than existing asynchronous ADMM algorithms. Compared with existing sparse allreduce algorithms, the hierarchical sparse allreduce algorithm proposed in this paper makes better use of the characteristics of sparse data to reduce system time in multi-core cluster.

Alternating Direction Method of Multiplier (ADMM) has been a popular algorithmic framework for separable optimization problems with linear constraints. For numerical ADMM fail to exploit the particular structure of the problem at hand nor the input data information, leveraging task-specific modules (e.g., neural networks and other data-driven architectures) to extend ADMM is a significant but challenging task. This work focuses on designing a flexible algorithmic framework to incorporate various task-specific modules (with no additional constraints) to improve the performance of ADMM in real-world applications. Specifically, we propose Guidance from Optimality (GO), a new customization strategy, to embed task-specific modules into ADMM (GO-ADMM). By introducing an optimality-based criterion to guide the propagation, GO-ADMM establishes an updating scheme agnostic to the choice of additional modules. The existing task-specific methods just plug their task-specific modules into the numerical iterations in a straightforward manner. Even with some restrictive constraints on the plug-in modules, they can only obtain some relatively weaker convergence properties for the resulted ADMM iterations. Fortunately, without any restrictions on the embedded modules, we prove the convergence of GO-ADMM regarding objective values and constraint violations, and derive the worst-case convergence rate measured by iteration complexity. Extensive experiments are conducted to verify the theoretical results and demonstrate the efficiency of GO-ADMM.

This article presents OPTIMO, a 65-nm, 16-b, fully programmable, spatial-array processor with 49 cores and a hierarchical multi-cast network for solving distributed optimizations via the alternating direction method of multipliers (ADMM). ADMM is a projection-based method for solving generic-constrained optimizations' problems. In essence, it relies upon decomposing the decision vector into subvectors, updating sequentially by minimizing an augmented Lagrangian function, and eventually updating the Lagrange multiplier. The ADMM algorithm has typically been used for solving problems in which the decision variable is decomposed into two or multiple subvectors. We demonstrate six template algorithms and their applications and measure a peak energy efficiency of 279 GOPS/W.

In reconfigurable intelligent surfaces (RISs) aided millimeter wave communication systems, accurate channel state information (CSI) is particularly crucial to achieve high passive beamforming gain. However, the complexity of channel estimation increases significantly due to the high-dimensional channel matrix and the additional RISs' assisted channels. To tackle this challenge, an alternating direction method of multipliers (ADMM) based channel estimation method is proposed to improve the CSI estimation accuracy and reduce the training overhead. By converting the cascaded channel matrix into a sparse matrix recovery problem, both low rank and sparsity properties are exploited by the ADMM algorithm to solve the sparse matrix recovery problem. The simulation results verify that the proposed ADMM algorithm can significantly increase the accuracy of channel estimation and reduce the training overhead, compared with the well-known competing channel estimation algorithms.

Quantile regression offers a powerful means of understanding the comprehensive relationship between response variables and predictors. By formulating prior domain knowledge and assumptions as constraints on parameters, the estimation efficiency can be enhanced. This paper studies some methods based on multi-block ADMM (Alternating Direction Method of Multipliers) to fit general penalized quantile regression models with linear constraints on regression coefficients. Different formulations for handling linear constraints and general penalties are explored and compared. Among these formulations, the most efficient one is identified, which provides an explicit expression for each parameter during iterations and eliminates the nested-loop in existing algorithms. Furthermore, this work addresses the challenges posed by big data by developing a parallel ADMM algorithm suitable for distributed data storage. The algorithm’s convergence and a robust stopping criterion are established. To demonstrate the excellent performance of the proposed algorithms, extensive numerical experiments and a real data example are presented. These empirical validations showcase the effectiveness of the methods in handling complex datasets. The details of theoretical proofs and different algorithm variations are provided in the Appendix.

Recently, we have proposed combining the alternating direction method of multipliers (ADMM) with a Gaussian back substitution procedure for solving the convex minimization model with linear constraints and a general separable objective function, i.e., the objective function is the sum of many functions without coupled variables. In this paper, we further study this topic and show that the decomposed subproblems in the ADMM procedure can be substantially alleviated by linearizing the involved quadratic terms arising from the augmented Lagrangian penalty. When the resolvent operators of the separable functions in the objective have closed-form representations, embedding the linearization into the ADMM subproblems becomes necessary to yield easy subproblems with closed-form solutions. We thus show theoretically that the blend of ADMM, Gaussian back substitution and linearization works effectively for the separable convex minimization model under consideration.

The reliability and safety of multi-robot systems (MRSs) are increasingly critical as such systems are deployed in real-world missions such as autonomous surveillance, infrastructure inspection, and disaster response. These systems rely on continuous communication and local sensing to achieve distributed coordination and consensus. However, this same interconnectivity exposes MRSs to vulnerabilities such as sensor faults, communication failures, and deliberate cyberattacks, which can corrupt shared information and compromise the collective objective of the system. Ensuring resilience in these settings requires not only the ability to detect and identify compromised agents, but also to guarantee stable network behavior even when adversarial or faulty information is introduced. Achieving this in a decentralized manner—without dependence on a central coordinator or trusted nodes—remains a central challenge in multi-agent robotics and distributed optimization.The first part of this thesis addresses this challenge by introducing an anchor-free, range-based integrity monitoring framework for MRSs that detects, identifies, and reconstructs the effects of cyberattacks and system faults using only information inter-robot range measurements. By formulating the problem as a distributed optimization solved via Sequential Convex Programming (SCP) and the Alternating Direction Method of Multipliers (ADMM), the proposed approach enables real-time, scalable, and fully distributed fault detection without relying on any pre-defined anchors or trusted agents. A threshold-based cold-start mechanism is introduced to ensure robustness against sensor noise and changes in network topology, maintaining algorithmic stability under realistic, time-varying conditions. The framework is validated through extensive numerical simulations and mixed-reality experiments involving heterogeneous UAV swarms subjected to GNSS spoofing attacks, demonstrating effective detection, identification, and localization of compromised agents.While this framework effectively detects and reconstructs faults, subsequent analysis revealed a critical vulnerability: falsified or inconsistent network information can bias the distributed optimization process itself, leading to divergence or erroneous consensus among nominal agents. To overcome this limitation, the second part of the thesis develops the Dynamically Weighted ADMM (DW-ADMM) algorithm, a Byzantine-resilient extension of ADMM that dynamically reweights communication edges to suppress the influence of malicious nodes. Theoretical results establish convergence to the global optimum in error-free conditions and bounded performance under Byzantine threats. Simulation studies confirm that DW-ADMM maintains consensus and optimization accuracy even when a subset of agents transmits adversarial data.Together, these contributions form a comprehensive framework for resilient distributed estimation and optimization in multi-robot systems, bridging fault detection, attack identification, and Byzantine-tolerant coordination under a unified theoretical and experimental foundation.

SummaryThis paper investigates two novel distributed algorithms based on alternating direction method of multipliers (ADMM) for network resource allocation of N agents. The main objective is to derive an optimal allocation that minimizes a global objective expressed as a sum of locally known separable convex objective functions. Based on a communication matrix, the dual resource allocation problem is changed into a consensus optimization problem, in which each agent broadcasts the outcome of its local processing to all his neighbors. In this paper, we first propose a new distributed dual consensus ADMM (DC‐ADMM) algorithm to address this consensus problem. Moreover, by applying an inexact step for each ADMM update, a distributed inexact DC‐ADMM (IDC‐ADMM) is also developed, which enables agents to perform cheap computation at each iteration. Finally, numerical simulations are delivered to illustrate and validate the proposed algorithm.

The grey model (GM) is a forecasting model known for its ability to provide accurate predictions for small sample data, but it struggles with complex nonlinear time series. This paper introduces a fractional calculus prediction model, utilizing Conformable Fractional Difference (CFD) and Conformable Fractional Accumulation (CFA) to calculate the new time series. The traditional least squares method for parameter estimation is transformed into a Least Absolute Shrinkage and Selection Operator (LASSO) problem and is solved using the dual form of the Alternating Direction Method of Multipliers (ADMM). Additionally, the Hiking Optimization Algorithm (HOA) is employed to optimize the fractional order parameter. As a result, the improved model demonstrates greater accuracy than the original model when dealing with small sample data.

Rate splitting multiple access (RSMA) has emerged as a promising technique in multi-antenna downlink transmissions, which jointly encodes the private and common messages into a common stream for removing part of interference incurred by signal multiplexing and improving the spectral efficiency. Due to the decodability constraint of the common message at each user, the precoder optimization is non-convex, which lacks efficient algorithms. For facilitating the practical implementation of RSMA in future wireless networks, e.g., massive multiple-input multiple-output (MIMO), we propose an alternating direction method of multipliers (ADMM)-induced iterative algorithm to optimize the precoder, where the precoder is optimized for maximizing the sum rate at multiple users under the decod-ability constraint of the common message. Specifically, we first employ the weighted minimum mean square error (WMMSE) algorithm to transform the non-convex problem into a sequence of convex problems, and then the ADMM algorithm is adopted to decouple the optimization variables and obtain solutions of each subproblem in closed form. Numerical results demonstrate that the proposed algorithm not only can achieve almost the same performance as existing WMMSE-based algorithms, but also has a faster convergence rate and lower computational complexity.

In this study, we focus on the feature selection problem in regression, and propose a new version of L1support vector regression (L1-SVR), known as L1-norm least squares support vector regression (L1-LSSVR). The alternating direction method of multipliers (ADMM), a method from the augmented Lagrangian family, is used to solve L1-LSSVR. The sparse solution of L1-LSSVR can realize feature selection effectively. Furthermore, L1-LSSVR is decomposed into a sequence of simpler problems by the ADMM algorithm, resulting in faster training speed. The experimental results demonstrate that L1-LSSVR is not only as effective as L1-SVR, LSSVR, and SVR in both feature selection and regression, but also much faster than L1-SVR and SVR.

In this paper, we investigate the problem of estimating a complex-valued Laplacian matrix with a focus on its application in the estimation of admittance matrices in power systems. The proposed approach is based on a constrained maximum likelihood estimator (CMLE) of the complex-valued Laplacian, which is formulated as an optimization problem with Laplacian and sparsity constraints. The complex-valued Laplacian is a symmetric, non-Hermitian matrix that exhibits a joint sparsity pattern between its real and imaginary parts. Thus, we present a group-sparse-based penalized log-likelihood approach for the Laplacian estimation. Leveraging the mixed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell_{2,1}$</tex-math></inline-formula> norm relaxation of the joint sparsity constraint, we develop a new alternating direction method of multipliers (ADMM) estimation algorithm for the implementation of the CMLE of the Laplacian matrix under a linear Gaussian model. Next, we apply the proposed ADMM algorithms for the problem of estimating the admittance matrix under three commonly-used measurement models that stem from Kirchhoff's and Ohm's laws, each with different assumptions and simplifications: 1) the nonlinear alternating current (AC) model; 2) the decoupled linear power flow (DLPF) model; and 3) the direct current (DC) model. The performance of the ADMM algorithm is evaluated using data from the IEEE <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$33$</tex-math></inline-formula>-bus power system data under different settings. The numerical experiments demonstrate that the proposed algorithm outperforms existing methods in terms of mean-squared-error (MSE) and F-score, thus providing a more accurate recovery of the admittance matrix.

In this paper, we consider waveform design for dual-function radar-communication systems based on multiple-input-multiple-out arrays. To achieve better Rician target detection performance, we use the relative entropy associated with the formulated detection problem as the design metric. We also impose a multi-user interference energy constraint on the waveforms to ensure the achievable sum-rate of the communications. Two algorithms are presented to tackle the nonlinear non-convex waveform design problem. In the first algorithm, we derive a quadratic function to minorize the objective function. To tackle the quadratically constrained quadratic programming problem at each iteration, a semidefinite relaxation approach followed by a rank-one decomposition procedure and an efficient alternating direction method of multipliers (ADMM) are proposed, respectively. In the second algorithm, we present a novel ADMM algorithm to tackle the optimization problem and employ an efficient minorization-maximization approach in the inner loop of the ADMM algorithm. Numerical results demonstrate the superiority of both algorithms. Moreover, the presented algorithms can be extended to synthesize peak-to-average-power ratio constrained waveforms, which allows the radio frequency amplifier to operate at an increased efficiency.