Large-scale quasi-Newton trust-region methods with low-dimensional linear equality constraints

This paper explores several approaches for articulated-pose estimation, assuming that video-rate depth information is available, from either stereo cameras or other sensors. We use these depth measurements in the traditional linear brightness constraint equation, as well as in a depth constraint equation. To capture the joint constraints, we combine the brightness and depth constraints with twist mathematics. We address several important issues in the formation of the constraint equations, including updating the body rotation matrix without using a first-order matrix approximation and removing the coupling between the rotation and translation updates. The resulting constraint equations are linear on a modified parameter set. After solving these linear constraints, there is a single closed-form non-linear transformation to return the updates to the original pose parameters. We show results for tracking body pose in oblique views of synthetic walking sequences and in moving-camera views of synthetic jumping-jack sequences. We also show results for tracking body pose in side views of a real walking sequence.

For a class of nonsmooth composite optimization problems with linear equality constraints, we utilize a Lyapunov-based approach to establish the global exponential stability of the primal-dual gradient flow dynamics based on the proximal augmented Lagrangian. The result holds when the differentiable part of the objective function is strongly convex with a Lipschitz continuous gradient; the non-differentiable part is proper, lower semi-continuous, and convex; and the matrix in the linear constraint is full row rank. Our quadratic Lyapunov function generalizes recent result from strongly convex problems with either affine equality or inequality constraints to a broader class of composite optimization problems with nonsmooth regularizers and it provides a worst-case lower bound of the exponential decay rate. Finally, we use computational experiments to demonstrate that our convergence rate estimate is less conservative than the existing alternatives.

Linear constraint equations for continuous support conditions in finite element analysis

Fully constrained linear spectral mixture analysis (FCLSMA) has been used for material quantification in remotely sensed imagery. In order to implement FCLSMA, two constraints are imposed on abundance fractions, referred to as Abundance Sum-to-one Constraint (ASC) and Abundance Nonnegativity Constraint (ANC). While the ASC is linear equality constraint, the ANC is a linear inequality constraint. A direct approach to imposing the ASC and ANC has been recently investigated and is called fully constrained least-squares (FCLS) method. Since there is no analytical solution resulting from the ANC, a modified fully constrained least-squares method (MFCLS) which replaces the ANC with an Absolute Abundance Sum-to-one Constraint (AASC) was proposed to convert a set of inequality constraints to a quality constraint. The results produced by these two approaches have been shown to be very close. In this paper, we take an oopposite approach to the MFCLS method, called least-squares with linear inequality constraints (LSLIC) method which also solves FCLSMA, but replaces the ASC with two linear inequalities. The proposed LSLIC transforms the FCLSMA to a linear distance programming problem which can be solved easily by a numerical algorithm. In order to demonstrate its utility in solving FCLSMA, the LSLIC method is compared to the FCLS and MFCLS methods. The experimental results show that these three methods perform very similarly with only subtle differences resulting from their problem formations.

An optimal force distribution scheme of multiple cooperating robots is proposed. General formulations of the optimal force distribution problem are built into quadratic programming with linear equality and quadratic inequality constraints. The quadratic constraints are due to the norm constraints of grasping forces and the approximation of maximum joint torque constraints which is used for simplification of the problem. This paper presents a technique for solving these general optimal force distribution problems. The original problem is transformed into a compact problem by eliminating the linear equality constraints. Then, the compact problem is transformed into a dual problem to be solved by using Lagrange multipliers and nonlinear programming dual method. This technique reduces the problem size considerably and makes the problem almost unconstrained so that no initial feasible points are needed. These features exhibit the efficiency of this technique. Simulation results for two cooperating PUMA robots indicating the ability of real time applications of the proposed technique are presented.

This paper develops a novel algorithm that tackles a general distributed convex optimization problem with a nonsmooth objective function and a series of coupling nonlinear inequality and linear equality constraints. To address such a problem, we first design a distributed proximal algorithm for problems with coupling equality constraints only, and then extend this algorithm to problems with both inequality and equality constraints by incorporating a virtual-queue-based method. The resulting algorithm, referred to as Proximal Primal-Dual Algorithm (PPDA), is shown to achieve O(1/k) rates of convergence with respect to both optimality and feasibility. Compared to the alternative methods in the literature, PPDA eliminates their assumptions on the smoothness of the objective function and achieves a stronger convergence rate result. In addition, PPDA exhibits faster convergence in a couple of numerical examples.

On a basic multivariate EIV model with linear equality constraints

We derive the asymptotic approximation for the posterior distribution when the data are multinomial and the prior is Dirichlet conditioned on satisfying a finite set of linear equality and inequality constraints so the posterior is also Dirichlet conditioned on satisfying these same constraints. When only equality constraints are imposed, the asymptotic approximation is normal. Otherwise it is normal conditioned on satisfying the inequality constraints. In both cases the posterior is a root-n-consistent estimator of the parameter vector of the multinomial distribution. As an application we consider the constrained Polya posterior which is a non-informative stepwise Bayes posterior for finite population sampling which incorporates prior information involving auxiliary variables. The constrained Polya posterior is a root-n-consistent estimator of the population distribution, hence yields a root-n-consistent estimator of the population mean or any other differentiable function of the vector of population probabilities.

A trust region and affine scaling interior point method (TRAM) is proposed for a general nonlinear minimization with linear inequality constraints in [8]. In the proposed approach, a Newton step is derived from the complementarity conditions. Based on this Newton step, a trust region subproblem is formed, and the original objective function is monotonically decreased. Explicit sufficient decrease conditions are proposed for satisfying complementarity, dual feasibility and second order optimality. The objective of this paper is to establish global and local convergence properties of the proposed trust region and affine scaling interior point method. It is shown that the proposed decrease conditions are sufficient for achieving complementarity, dual feasibility and second order optimality respectively. It is also established that a trust region solution is asymptotically in the interior of the proposed trust region subproblem and a damped trust region step can achieve quadratic convergence.

A generalization of Motzkin’s theorem of the alternative is developed, and is then used to obtain necessary optimality conditions for the Fritz John optimization problem with linear equality constraints. Under certain constraint qualifications and convexity assumptions these necessary conditions are also sufficient. An application of these results to the problem of Chebyshev approximation with interpolation is given.

The revisited total least squares problems with linear equality constraint

In this paper we develop a special-purpose iterative algorithm, of the row-action type, for solving the problem of maximizing the “$\log x$” entropy functional over linear equality constraints. The algorithm employs “projections” onto hyperplanes which we call “$\log x$” entropy projections. A complete proof of convergence is given.

The network routing problem is one of important problems to consider behaviors of multiple flows on a network with traffic. Some network routing problems can be classified into the selfish routing game in which the behaviors of the flows are competitive and the optimal routing problem in which the behaviors of the flows are cooperative. This study focuses on the optimal routing problem. The optimal routing problem is formulated as an optimization problem with a linear equality constraint and lower limit constraints. The learning problem of the support vector machine (SVM) is formulated as an optimization problem with a linear equality constraint and upper and lower limit constraints. The sequential minimal optimization (SMO) is one of the fast solvers for the learning problem. SMO can be applied to solve the optimal routing problem. This study proposes a new fast optimal routing search method using SMO. In the numerical experiments, the effectiveness of the proposed method on the computational cost is verified with the comparison to the search method using the replicator dynamics.

This article presents two novel continuous- and discrete-time neural networks (NNs) for solving quadratic minimax problems with linear equality constraints. These two NNs are established based on the conditions of the saddle point of the underlying function. For the two NNs, a proper Lyapunov function is constructed so that they are stable in the sense of Lyapunov, and will converge to some saddle point(s) for any starting point under some mild conditions. Compared with the existing NNs for solving quadratic minimax problems, the proposed NNs require weaker stability conditions. The validity and transient behavior of the proposed models are illustrated by some simulation results.

This paper generalizes the Lagrange multiplier solution for a regression subject to inhomogeneous (d ≠ 0) linear equality constraints, Vb′= d, to the case where these constraints are only partially satisfied. It extends the work of an earlier paper that generalized the Lagrange multiplier solution for a regression subject to homogeneous (d = 0) linear equality constraints, Vb′ = 0. With this method, we can insure that our chemometric models predict known standard samples more accurately by making such samples more influential. For example, we can include samples from a particular locale in a calibration set as sample ‘standards’ to create a ‘tuned’ model, b′ from a generic model, b, that is based on samples from a wider geographical area. We introduce new ways to adjustably control a sample's influence and show why they are preferable to simply including that sample multiple times. One new way is to use simultaneous counterbalanced sample‐pair distortion of both X and Y blocks. Another is to generalize the Lagrange multiplier solution that allows the constraints Vb′ = d to be only partially satisfied, which is important when the constraints interfere so that a perfect fit to one standard sample prevents a perfect fit to another standard sample. Some advantages of these new methods are that they cause no frequency filtering of the X‐block, which might remove information, and they allow the user to minimize the standard error of prediction by optimizing influence. Copyright © 2007 John Wiley & Sons, Ltd.