Updating QR factorization procedure for solution of linear least squares problem with equality constraints

The present paper is devoted to the application of the space transformation technique for solving linear and nonlinear programming problems. By using a surjective mapping the initial constrained optimization problem is transformed to a problem in a new space with only equality constraints. For the numerical solution of the latter problem we use the generalized gradient-projection method and Newton’s method. After inverse transformation to the initial space we obtain the family of numerical methods for solving optimization problems with equality and inequality constraints. In the linear programming case after some simplification and after choosing a particular exponential transformation function we obtain from the proposed methods the well-known primal-dual interior point linear programming algorithm and the so-called “variation of Kermarkar’s algorithm”.

SVD and QR Decomposition have attracted much attention recently for solving linear algebra and digital signal processing problems. The House-holder QR Decomposition requires a smaller number of operations then the Givens QR Decomposition and therefore is strongly recommended for sequential computer implementations (as found in nearly all known libraries like NAG, LINPACK, EISPACK etc.) However for parallel implementation it has not been widely used, because the Givens Rotations possess inherent parallelism and the Householder Reflections do not. This is true if one analyzes the original algorithm. Four independent loops and three bottlenecks between the loops are constraint for pipelining the computations. This leads to an inefficient solution, i.e. 4n time moments for each iteration and since there are n iterations then the total time to execute the algorithm is O(4n 2 ). Compared to 3n m 2 time steps for the Givens QR Decomposition it is uneconomic. In this paper we have used some recent results for the elimination of the computational and data broadcast, and data synchronization to derive a fully localized form of the Householder QR Decomposition algorithm. We have succeeded in reorganizing and transforming the algorithm from three bottlenecks to one and four loops to one. A linear and a double pipeline array of n+1 processors are presented to solve the problem in O ( n 2 /2) time steps. It is also shown that the bottlenecks for the bidiagonalization and SVD computation cannot be eliminated.

We propose a new dual projected pseudo quasi-Newton method for discrete-time linear quadratic optimal control problems of a large-scale multi-destination network. Our method has successfully overcome the difficulties caused by large dimensions and inequality constraints by having a constant Hessian matrix and an efficient successive projection method, respectively. Application of our method to the congestion control of multi-destination networks is presented. Above all, it takes less than one second to obtain the solution of a large-scale discrete-time linear quadratic optimal control problem with the problem size of 2800 variables, 700 equality constraints and 2400 inequality constraints.

This paper describes a decomposition approach for the solution of constrained optimal control problems (OCPs) that is implementable in a parallel manner. A transformation method is applied to the constrained OCP by means of saturation functions resulting in a new extended problem formulation with additional subdynamics and equality constraints. The structure of the reformulated OCP is then exploited to derive a decomposition method in order to split the entire OCP into smaller subproblems which can be solved independently. Moreover, an algorithm is presented to solve the decomposed problem which is based on a hierarchical optimization scheme. An example system is used to demonstrate the performance of the presented approach.

In Gaussian elimination and LU factorization we solve a linear system by transforming it to triangular form. These are not the only kind of transformations that can be used for such a task. Matrices with orthonormal columns, called unitary matrices can be used to reduce a square matrix to upper triangular form and more generally a rectangular matrix to upper triangular (also called upper trapezoidal) form. This lead to a decomposition of a rectangular matrix known as a QR decomposition and a reduced form which we refer to as a QR factorization. The QR decomposition and factorization will be used in later chapters to solve least squares- and eigenvalue problems.

In situations where processing memory is limited, the Support Vector Machine quadratic program can be decomposed into smaller sub-problems and solved sequentially. The convergence of this method has been proven previously through (he use of a counting method. In this initial investigation, we approach the convergence analysis by treating the decomposed sub-problems as subsystems of a general system. The gradients of the sub-problems and the inequality constraints are explicitly modelled as system variables. The change in these variables during optimization form a dynamic system modelled by vector differential equations. We show that the change in the objective function can be written as the energy in the system. This makes it a natural Lyapunov function, which has an asymptotically stable point at the origin. The asymptotic stability of the whole system then follows under certain assumptions.

A computational method for efficient solution of linear constrained least squares problems with Kronecker product structure is presented. The equality constraints are assumed to be linearly independent. The computational efficiency of the method is analyzed. Conditions for uniqueness of solutions are given.

Solving combinatorial optimization problems of the kind that can be codified by quadratic unconstrained binary optimization (QUBO) is a promising application of quantum computation. Some problems of this class suitable for practical applications such as the traveling salesman problem (TSP), the bin packing problem (BPP), or the knapsack problem (KP) have inequality constraints that require a particular cost function encoding. The common approach is the use of slack variables to represent the inequality constraints in the cost function. However, the use of slack variables considerably increases the number of qubits and operations required to solve these problems using quantum devices. In this work, we present an alternative method that does not require extra slack variables and consists of using an unbalanced penalization function to represent the inequality constraints in the QUBO. This function is characterized by larger penalization when the inequality constraint is not achieved than when it is. We evaluate our approach on the TSP, BPP, and KP, successfully encoding the optimal solution of the original optimization problem near the ground state cost Hamiltonian. Additionally, we employ D-Wave Advantage and D-Wave hybrid solvers to solve the BPP, surpassing the performance of the slack variables approach by achieving solutions for up to 29 items, whereas the slack variables approach only handles up to 11 items. This new approach can be used to solve combinatorial problems with inequality constraints with a reduced number of resources compared to the slack variables approach using quantum annealing or variational quantum algorithms.

As outlined in chapter 1, integrated optimum structural design represents an optimization problem with both equality constraints and inequality constraints. In this method the analysis and optimal design of the structure are carried out simultaneously. The disadvantage of this approach, however, is that the formulation results in a large size problem. Generalized geometric programming is a powerful mathematical tool that can be used to solve the multidimensional optimization problems with nonlinear equations, Burns (1985). In this chapter we will present the generalized geometric programming (GGP) and its extensions to the solution of problems with equality constraints (GGPE). We will also describe the IGGP solver, Burns (1982), and the added extensions for the optimal solution.

This paper considers the linear quadratic optimal control problem for linear implicit systems based on the dissipation inequality. We derive a necessary and sufficient condition for the dissipativeness with respect to a quadratic supply rate in terms of a linear matrix inequality (LMI) condition with an equality constraint. Based on this constrained LMI condition, the optimal control law is given by an implicit algebraic constraint among system variables. We also show that the present constrained LMI condition easily reduces to an LMI without any equality constraints.

This paper studies the discrete-time indefinite stochastic linear quadratic optimal control problem with the equality terminal state constraints,which can be transformed into a mathematical programming with equality constraints. By means of matrix Lagrange theorem, a necessary condition for the existence of optimal linear state feedback control is given. The previous results on discrete-time stochastic linear quadratic optimal control without constraints, can be viewed as corollaries of the main theorem of this paper.

Variational problems with inequality constraints

In this study, we propose a medical image encryption algorithm based on a new five-dimensional (5D) multi-band multi-wing chaotic system and QR decomposition. First, we construct a new 5D multi-band multi-wing chaotic system through feedback control, which has a relatively large Lyapunov exponent. Second, we decompose the plaintext image matrix and chaotic sequence into an orthogonal matrix and upper triangular matrix using QR decomposition. We multiply the orthogonal matrix decomposed from the original image by the orthogonal matrix decomposed from the chaotic sequence. In this process, we use the chaotic sequence to control left and right multiplication. Simultaneously, we chaotically rearrange the elements in the upper triangular matrix using the improved Joseph loop and then multiply the two resulting matrices. Finally, we subject the product matrix to bit-level scrambling. From the theoretical analysis and simulation results, we observed that the key space of this method was relatively large, the key sensitivity was relatively strong, it resisted attacks of statistical analysis and gray value analysis well, and it had a good encryption effect for medical images.

The purpose of this paper is to bring a series of rather surprising results which have appeared in the technical literature to the attention of a wider audience. We describe the results informally here; precise definitions and results will be given below. Our subject is the solution of an ill-posed problem on a digital computer. Such a problem can at best only be solved approximately. We say a problem is solvable if we can compute an "-approximation for any positive " and is unsolvable if we cannot compute an "-approximation, even for arbitrarily large ". It was shown in [17] that a linear problem is unsolvable iff it is ill-posed. Are there circumstances under which one can avoid this negative conclusion? It is common to associate an operator S with an ill-posed problem. If S is linear, then a problem is ill-posed iff S is unbounded. Note that unboundedness is a worst case concept; the supremum of S operating on elements in the unit ball of its domain is infinite. This suggests introducing the concept of a problem’s being ill-posed on the average if the expectation of S with respect to a measure is infinite and being well-posed on the average if this expectation is finite. These concepts were introduced in [17], where it was shown that if the measure is Gaussian and the linear operator S is measurable, then a linear problem is unsolvable on the average iff it is ill-posed on the average. A natural next step is to find a linear ill-posed problem which is also ill-posed on the average. Several attempts to do this for Gaussian measures were unsuccessful. This suggested the question: Is every linear problem well-posed on the average for any Gaussian measure? This question was answered in the affirmative in [5] and [15]. The conclusion in the title follows immediately from this and the preceding result. What if the measure is non-Gaussian? Now the situation is more complicated. In particular, there are problems that are even ill-posed on the average. In this paper, we round out the picture by presenting examples of such problems for non-Gaussian measures.

Simulating (elastically) deformable models that can collide with each other and with the environment remains a challenging task. The resulting contact problems can be elegantly approached using Lagrange multipliers to represent the unknown magnitude of the response forces. Typical methods construct and solve a Linear Complementarity Problem (LCP) to obtain the response forces. This requires the inverse of the generalized mass matrix, which is generally hard to obtain for deformable-body problems. In this article, we tackle such contact problems by directly solving the Mixed Linear Complementarity Problem (MLCP) and omitting the construction of an LCP matrix. Since a convex quadratic program with linear constraints is equivalent to an MLCP, we propose to use a Conjugate Residual (CR) solver as the backbone of our collision response system. By dynamically updating the set of active constraints, the MLCP with inequality constraints can be solved efficiently. We also propose a simple yet efficient preconditioner that ensures faster convergence. Finally, our approach is faster than existing methods (at the same accuracy), and it allows accurate treatment of friction.