A Distributed Proximal Primal-Dual Algorithm for Nonsmooth Optimization with Coupling Constraints

This paper considers a general nonlinear econometric model framework that contains a large class of estimators defined as solutions to optimization problems. For this framework we derive several asymptotically equivalent forms of a test statistic for the local (in a way made precise in the paper) multivariate nonlinear inequality constraints testH:h(β) ≥ 0 versusK: β ∈RK. We extend these results to consider local hypotheses tests of the formH:h1(β) ≥ 0 andh2(β) = 0 versusK: β ∈RK. For each test we derive the asymptotic distribution for any size test as a weighted sum of χ2-distributions. We contrast local as opposed to global inequality constraints testing and give conditions on the model and constraints when each is possible. This paper also extends the well-known duality results in testing multivariate equality constraints to the case of nonlinear multivariate inequality constraints and combinations of nonlinear inequality and equality constraints.

In this paper, a one-layer recurrent neural network is proposed for solving non-smooth convex optimization problems with linear equality constraints. Comparing with the existing neural networks, the proposed neural network has simpler architecture and the number of neurons is the same as that of decision variables in the optimization problems. The global convergence of the neural network can be guaranteed if the non-smooth objective function is convex. Simulation results are provided to show that the state trajectories of the neural network can converge to the optimal solutions of the non-smooth convex optimization problems and show the performance of the proposed neural network.KeywordsNeural NetworkIEEE TransactionGlobal ConvergenceRecurrent Neural NetworkState TrajectoryThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Distributed constrained optimization for multi-agent networks with nonsmooth objective functions

For a class of nonsmooth composite optimization problems with linear equality constraints, we utilize a Lyapunov-based approach to establish the global exponential stability of the primal-dual gradient flow dynamics based on the proximal augmented Lagrangian. The result holds when the differentiable part of the objective function is strongly convex with a Lipschitz continuous gradient; the non-differentiable part is proper, lower semi-continuous, and convex; and the matrix in the linear constraint is full row rank. Our quadratic Lyapunov function generalizes recent result from strongly convex problems with either affine equality or inequality constraints to a broader class of composite optimization problems with nonsmooth regularizers and it provides a worst-case lower bound of the exponential decay rate. Finally, we use computational experiments to demonstrate that our convergence rate estimate is less conservative than the existing alternatives.

Optical fibers aim to image in vivo biological processes. In this context, high spatial resolution and stability to fiber movements are key to enable decision-making processes (e.g. for microendoscopy). Recently, a single-pixel imaging technique based on a multicore fiber photonic lantern has been designed, named computational optical imaging using a lantern (COIL). A proximal algorithm based on a sparsity prior, dubbed SARA-COIL, has been further proposed to solve the associated inverse problem, to enable image reconstructions for high resolution COIL microendoscopy. In this work, we develop a data-driven approach for COIL. We replace the sparsity prior in the proximal algorithm by a learned denoiser, leading to a plug-and-play (PnP) algorithm. The resulting PnP method, based on a proximal primal–dual algorithm, enables to solve the Morozov formulation of the inverse problem. We use recent results in learning theory to train a network with desirable Lipschitz properties, and we show that the resulting primal–dual PnP algorithm converges to a solution to a monotone inclusion problem. Our simulations highlight that the proposed data-driven approach improves the reconstruction quality over variational SARA-COIL method on both simulated and real data.

Advances in optimization theory have been made systematically by the desire to solve more and more complicated geometric structures that are realised in contemporary applications. This is a rigorous investigation of monotone vector fields and proximal algorithms in the deep geometrical setting of generalized metric spaces (G-metric spaces). Our study fills a general deficiency in the literature by generalizing classical monotonicity principles and proximal point algorithms to support the complex three-point distance structure of G-metric spaces.In this way, by conductinga strict theoretical study, we prove the existence and uniqueness of solutions in the concept of monotone inclusion, are able to develop effective proximal algorithms with guaranteed convergence rates, and illustrate their successful application in different areas of practice. Theoretical contributions that we have made include: (1) the extension of monotonicity theory in all its forms to G-metric spaces with complete characterizations, (2) the construction of strongly convergent proximalpoint algorithms that are explicit in rate of convergence, and (3) its application to variational inequalities and multi-objective optimization problems in non-standard geometries, where the old metric structures are no longer applicable. Our findings create new opportunities to deal with optimization problems in complex networks, social systems, and the present-day machine learning paradigms. Keywords: G-metric spaces, monotone operators, proximal algorithms,

An accelerated proximal gradient algorithm for source-independent waveform inversion

This paper derives several properties unique to nonlinear model hypothesis testing problems involving linear or nonlinear inequality constraints in the null or alternative hypothesis. The paper is organized around a lemma which characterizes the set containing the least favorable parameter value for a nonlinear model inequality constraints hypothesis test. We then present two examples which illustrate several implications of this lemma. We also discuss the impact of these properties on the empirical implementation and interpretation of these test procedures.

Linear constraint equations for continuous support conditions in finite element analysis

Formal convergence analysis on deterministic [formula omitted]-regularization based mini-batch learning for RBF networks

This paper explores several approaches for articulated-pose estimation, assuming that video-rate depth information is available, from either stereo cameras or other sensors. We use these depth measurements in the traditional linear brightness constraint equation, as well as in a depth constraint equation. To capture the joint constraints, we combine the brightness and depth constraints with twist mathematics. We address several important issues in the formation of the constraint equations, including updating the body rotation matrix without using a first-order matrix approximation and removing the coupling between the rotation and translation updates. The resulting constraint equations are linear on a modified parameter set. After solving these linear constraints, there is a single closed-form non-linear transformation to return the updates to the original pose parameters. We show results for tracking body pose in oblique views of synthetic walking sequences and in moving-camera views of synthetic jumping-jack sequences. We also show results for tracking body pose in side views of a real walking sequence.

This paper studies a class of network optimization problems where the objective function is the summation of individual agents' convex functions and their decision variables are coupled by linear equality constraints. These constraints are not sparse, meaning that they do not match the pattern of the network adjacency matrix. We propose two approaches to design efficient distributed algorithms to solve the network optimization problem. Our first approach consists of transforming the non-sparse equality constraints into sparse ones by increasing the number of the agents' decision variables, yielding an exact reformulation of the original optimization problem. We discuss two reformulations, based on the addition of consensus variables and of constraint-mismatch variables, and discuss the scalability of the strategies resulting from them. Our second approach consists instead of sparsifying the non-sparse constraints by zeroing some coefficients, yielding an approximate reformulation of the original problem. We formally characterize the gap on the distance between the optimizers of the original and approximated problems as a function of the number of entries made zero in the constraints. Various simulations illustrate our results.

This paper investigates the problem of distributed control of multiple redundant mobile manipulators to collectively transport an object tracking a desired trajectory with energy and manipulability optimized.To solve this optimization problem, formation control tasks are introduced as equality constraints with the variables being the velocities. In this paper, we propose a distributed proximal gradient algorithm searching for the optimal solution, with which the stability of the closed-loop system is proved. Simulations demonstrate the effectiveness of the proposed distributed optimization scheme and proximal algorithm.

This paper presents an analog neural network model to recover sparse signals. In the original constrained optimization task for recovering sparse signals, the objective function is not differentiable. Hence, we recast the original nonlinear programming problem as a linear programming problem with linear inequality constraints and equality constraints. However, the second order gradient of the objective function is not convex at an equilibrium point. To solve this problem, we further modify the objective function such that the second order gradient is convex at the equilibrium point. This paper presents two sets of network dynamics. One is for the standard recovery of sparse signals. Another one is for the noisy situation.KeywordsOptimizationSparse signal

We derive the asymptotic approximation for the posterior distribution when the data are multinomial and the prior is Dirichlet conditioned on satisfying a finite set of linear equality and inequality constraints so the posterior is also Dirichlet conditioned on satisfying these same constraints. When only equality constraints are imposed, the asymptotic approximation is normal. Otherwise it is normal conditioned on satisfying the inequality constraints. In both cases the posterior is a root-n-consistent estimator of the parameter vector of the multinomial distribution. As an application we consider the constrained Polya posterior which is a non-informative stepwise Bayes posterior for finite population sampling which incorporates prior information involving auxiliary variables. The constrained Polya posterior is a root-n-consistent estimator of the population distribution, hence yields a root-n-consistent estimator of the population mean or any other differentiable function of the vector of population probabilities.