- Research Article
5
- 10.1109/tac.2021.3118340
On the Analysis of Inexact Augmented Lagrangian Schemes for Misspecified Conic Convex Programs
- Aug 01, 2022
- IEEE Transactions on Automatic Control
- Necdet Serhat Aybat + 2 more +2
In this article, we consider the misspecified optimization problem of minimizing a convex function <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$f(x;\theta ^*)$</tex-math></inline-formula> in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$x$</tex-math></inline-formula> over a conic constraint set represented by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$h(x;\theta ^*) \in \mathcal {K}$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta ^*$</tex-math></inline-formula> is an unknown (or misspecified) vector of parameters, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mathcal {K}$</tex-math></inline-formula> is a closed convex cone, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$h$</tex-math></inline-formula> is affine in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$x$</tex-math></inline-formula> . Suppose that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta ^*$</tex-math></inline-formula> is unavailable but may be learnt by a separate process that generates a sequence of estimators <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta _k$</tex-math></inline-formula> , each of which is an increasingly accurate approximation of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta ^*$</tex-math></inline-formula> . We develop a first-order inexact augmented Lagrangian (AL) scheme for computing an optimal solution <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$x^*$</tex-math></inline-formula> corresponding to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta ^*$</tex-math></inline-formula> while simultaneously learning <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\theta ^*$</tex-math></inline-formula> . In particular, we derive rate statements for such schemes when the penalty parameter sequence is either constant or increasing and derive bounds on the overall complexity in terms of proximal gradient steps when AL subproblems are inexactly solved via an accelerated proximal gradient scheme. Numerical results for a portfolio optimization problem with a misspecified covariance matrix suggest that these schemes perform well in practice, while naive sequential schemes may perform poorly in comparison.
Read more