A class of bilinear matrix constraint optimization problem and its applications

An augmented Lagrangian multiplier method based on a CHKS smoothing function for solving nonlinear bilevel programming problems

The standard traffic assignment problem (TAP) is often augmented with additional constraints to address non-standard applications. These models are called TAP with side constraints (TAPSC). Despite the rising significance of TAPSC models, the ability to efficiently solve them to satisfactory precision remains limited in real-world applications. The purpose of this paper is to fill this gap by integrating a recently developed high performance TAP solver, known as the path-based Greedy algorithm, with the augmented Lagrangian multiplier (ALM) method. This paper examines how precisely the subproblems in the ALM method should be solved to optimize the overall convergence performance. It is found that insufficiently converged subproblem solutions sometimes lead to catastrophic failures, although pursuing extremely high precision could also be counterproductive. Accordingly, it is proposed to adjust the precision required to solve the subproblems based on an approximate gap measured by the augmented Lagrangian. Results of numerical experiments show that adaptively adjusting the subproblem precision limit produces a 25% speed-up compared with the algorithm with a fixed limit.

The optimal economic operation of a thermal electric power production system without considering transmission losses is a critical problem for ships, aircrafts, island power systems and it is usually solved with Lagrange method. In this paper, an alternative solution method is proposed using artificial neural networks (ANN) based on augmented Lagrange multiplier method with equality and inequality constraints. The respective theoretical analysis is presented, while a specific case study is studied. The advantages and disadvantages of the method are discussed and compared with the classical Lagrange method and ANN method based on external penalty functions.

This paper presents an approach that allows optimizing gear ratio and vehicle dimension to achieve optimum gear transmission. Therefore, augmented Lagrangian multiplier method, defined as classical method, is utilized to find the optimum gear ratios and the corresponding number of gear teeth applied to all epicyclical gears. The new method is able to calculate and also to optimize the gear ratio based on dynamics of 4WD vehicles. Therefore, 4WD vehicles dynamic equations are employed assuming that the rear wheels or the front wheels are at the point of slip. In addition, a genetic algorithm is modified to preserve feasibility of the encountered solutions. The basic dimension of a sample commercial vehicle (2009 hummer H3 4dr AWD SUV) and its gearbox are optimized, and then the effects of changing slip angle, wheel base, and engine torque on optimum vehicle dimension are analyzed.

In this dissertation we develop numerical algorithms and software tools to facilitate parallel solutions of nonlinear programming (NLP) problems. In particular, we address large-scale, block-structured problems with an intrinsic decomposable configuration. These problems arise in a great number of engineering applications, including parameter estimation, optimal control, network optimization, and stochastic programming. The structure of these problems can be leveraged by optimization solvers to accelerate solutions and overcome memory limitations, and we propose variants to two classes of optimization algorithms: augmented Lagrangian (AL) schemes and Schur-complement interior-point methods. The convergence properties of augmented Lagrangian decomposition schemes like the alternating direction method of multipliers (ADMM) and progressive hedging (PH) are well established for convex optimization but convergence guarantees in non-convex settings are still poorly understood. In practice, however, ADMM and PH often perform satisfactorily in complex non-convex NLPs. In this work, we study connections between the method of multipliers (MM), ADMM, and PH to derive benchmarking metrics that explain why PH and ADMM work in practice. We illustrate the concepts using challenging dynamic optimization problems. Our exposition seeks to establish more formalism in benchmarking ADMM, PH, and AL schemes and to motivate algorithmic improvements. The effectiveness of nonlinear interior-point solvers for solving large-scale problems relies quite heavily on the solution of the underlying linear algebra systems. The schur-complement decomposition is very effective for parallelizing the solution of linear systems with modest coupling. However, for systems with large number of coupling variables the schur-complement method does not scale favorably. We implement an approach that uses a Krylov solver (GMRES) preconditioned with ADMM to solve block-structured linear systems that arise in the interior-point method. We show that this ADMM-GMRES approach overcomes the well-known scalability issues of Schur decomposition. One important drawback of using decomposition approaches like ADMM and PH is their convergence rate. Unlike Schur-complement interior-point algorithms that have super-linear convergence, augmented Lagrangian approaches typically exhibit linear and sublinear rates. We exploit connections between ADMM and the Schur-complement decomposition to derive an accelerated version of ADMM. Specifically, we study the effectiveness of performing a Newton-Raphson algorithm to compute multiplier estimates for augmented Lagrangian methods. We demonstrate using two-stage stochastic programming problems that our multiplier update achieves convergence in fewer iterations for MM on general nonlinear problems. In the case of ADMM, the newton update significantly reduces the number of subproblem solves for convex quadratic programs (QPs). Moreover, we show that using newton multiplier updates makes the method robust to the selection of the penalty parameter. Traditionally, state-of-the-art optimization solvers are implemented in low-level programming languages. In our experience, the development of decomposition algorithms in these frameworks is challenging. They present a steep learning curve and can slow the development and testing of new numerical algorithms. To mitigate these challenges, we developed PyNumero, a new open source framework implemented in Python and C++. The package seeks to facilitate development of optimization algorithms for large-scale optimization within a high-level programming environment while at the same time minimizing the computational burden of using Python. The efficiency of PyNumero is illustrated by implementing algorithms for problems arising in stochastic programming and optimal control. Timing results are presented for both serial and parallel implementations. Our computational studies demonstrate that with the appropriate balance between compiled code and Python, efficient implementations of optimization algorithms are achievable in these high-level languages.

Past few years have witnessed the development of the satellite technology. With the rapid growth of global mobile services, people have higher and higher requirements for the quality of satellite communications. The communication delay can be effectively reduced by offloading the computing task to satellite. This paper introduces a LEO (Low Earth Orbit, LEO) satellite computing resource allocation scheme based on on-board resource pool. Through the ISL (Inter Satellite Link, ISL), the computing equipment of each satellite is combined into a resource pool to allocate computing resources to users in a centralized way. In order to ensure the fairness between users in the process of allocation, we describe the task of computing resource allocation as a constrained optimization problem with the goal of minimizing the variance between the supply and demand. Furthermore, the augmented Lagrange multiplier method is adopted to obtain the optimal solution, which dynamically allocates computing resources for each user. The simulation results are presented to verify the effectiveness and convergence of the proposed scheme.KeywordsLEO satelliteComputation resource allocationAugmented Lagrange multiplier method

In this paper, we propose a distributed algorithm for optimization problems that involve a separable, possibly nonconvex objective function subject to convex local constraints and linear coupling constraints. The method is based on the accelerated distributed augmented Lagrangians (ADAL) algorithm that was recently developed by the authors to address convex problems. Here, we extend this line of work in two ways. First, we establish convergence of the method to a local minimum of the problem, using assumptions that are common in the analysis of nonconvex optimization methods. To the best of our knowledge, this is the first work that shows convergence to local minima specifically for a distributed augmented Lagrangian (AL) method applied to nonconvex optimization problems; distributed AL methods are known to perform very well when used to solve convex problems. Second, we propose a more general and decentralized rule to select the stepsizes of the method. This improves on the authors’ original ADAL method, where the stepsize selection used global information at initialization. Numerical results are included to verify the correctness and efficiency of the proposed distributed method.

In this paper, we denoise a given noisy image by minimizing a smoothness-promoting function over a set of local similarity measures which compare the mean of the given image and some candidate image on a large collection of subboxes. The associated convex optimization problem possesses a huge number of constraints which are induced by extended real-valued functions stemming from the Kullback–Leibler divergence. Alternatively, these nonlinear constraints can be reformulated as affine ones, which makes the model seemingly more tractable. For the numerical treatment of both formulations of the model (i.e., the original one as well as the one with affine constraints), we propose a rather general augmented Lagrangian method which is capable of handling the huge amount of constraints. A self-contained, derivative-free, global convergence theory is provided, allowing an extension to other problem classes. For the solution of the resulting subproblems in the setting of our suggested image denoising models, we make use of a suitable stochastic gradient method. Results of several numerical experiments are presented in order to compare both formulations and the associated augmented Lagrangian methods.

Many kinds of real-world data, e.g., color images, videos, etc., are represented by tensors and may often be corrupted by outliers. Tensor robust principal component analysis (TRPCA) servers as a tensorial modification of the fundamental principal component analysis (PCA) which performs well in the presence of outliers. The recently proposed TRPCA model [12] based on tubal nuclear norm (TNN) has attracted much attention due to its superiority in many applications. However, TNN is computationally expensive, limiting the application of TRPCA for large tensors. To address this issue, we first propose a new TRPCA model by adopting a factorization strategy within the framework of tensor singular value decomposition (t-SVD). An algorithm based on the non-convex augmented Lagrangian method (ALM) is developed with convergence guarantee. Effectiveness and efficiency of the proposed algorithm is demonstrated through extensive experiments on both synthetic and real datasets.

<p style='text-indent:20px;'>Total generalization variation (TGV) is a very powerful and important regularization for various inverse problems and computer vision tasks. In this paper, we propose a semismooth Newton based augmented Lagrangian method for solving this problem. The augmented Lagrangian method (also called as method of multipliers) is widely used for lots of smooth or nonsmooth variational problems. However, its efficiency heavily depends on solving the corresponding coupled and nonlinear system together and simultaneously. With efficient primal-dual semismooth Newton methods for the challenging and highly coupled nonlinear subproblems involving total generalized variation, we develop a highly efficient and competitive augmented Lagrangian method compared with some fast first-order method. With the analysis of the metric subregularities of the corresponding functions, we give both the global convergence and local linear convergence rate for the proposed augmented Lagrangian methods.</p>

Given the ultrahigh dimensionality and the complex structure, which contains matrices and vectors, the mixed matrix minimization becomes crucial for the analysis of those data. Recently, the nonconvex functions such as the smoothly clipped absolute deviation, the minimax concave penalty, the capped $ \ell_1 $-norm penalty and the $ \ell_p $ quasi-norm with $ 0<p<1 $ have been shown remarkable advantages in variable selection due to the fact that they can overcome the over-penalization. In this paper, we propose and study a novel nonconvex mixed matrix minimization, which combines the low-rank and sparse regularzations and nonconvex functions perfectly. The augmented Lagrangian method (ALM) is proposed to solve the noncovnex mixed matrix minimization problem. The resulting subproblems either have closed-form solutions or can be solved by fast solvers, which makes the ALM particularly efficient. In theory, we prove that the sequence generated by the ALM converges to a stationary point when the penalty parameter is above a computable threshold. Extensive numerical experiments illustrate that our proposed nonconvex mixed matrix minimization model outperforms the existing ones.

Formal convergence analysis on deterministic [formula omitted]-regularization based mini-batch learning for RBF networks

ABSTRACTThis paper extends the energy‐based homogenization method (EBHM) to evaluate the effective elastic properties of two‐dimensional arc unit cells, where the periodic boundary conditions are constructed based on the polar strain–displacement relationships, and the displacement field is solved by the polar finite element method. Combining the extended EBHM with the parametric level set method (PLSM) and the augmented Lagrangian multiplier method (ALM), it is able to design the topological configurations of arc unit cells for seeking superior effective elastic properties. In addition, the level set function is first filtered by compactly supported radial basis functions (CSRBFs) in order to eliminate small holes in the configuration. Numerical examples are performed to demonstrate the advantages of the proposed method. The periodic boundary condition performs well when the circumferential angle of the arc unit cell is in the range of [0.0002, 0.2]. Results indicate that the effective elastic properties of optimized arc unit cells override those of geometrically transformed optimal square unit cells. This superiority mainly benefits from the fact that unisymmetrical configurations are allowed in arc unit cells but not in square unit cells. Results also imply that the capability of nucleation of new holes is retained even though small holes are eliminated by filtering. It contributes to the trade‐off between performance and manufacturability.

To solve the energy‐efficient virtual network embedding problem, this study proposes an embedding algorithm based on Hopfield neural network. An energy‐efficient virtual network embedding model was established. Wavelet diffusion was performed to take the structural feature value into consideration and provide a candidate set for virtual network embedding. In addition, the Hopfield network was used in the candidate set to solve the virtual network energy‐efficient embedding problem. The augmented Lagrangian multiplier method was used to transform the energy‐efficient virtual network embedding constraint problem into an unconstrained problem. The resulting unconstrained problem was used as the energy function of the Hopfield network, and the network weight was iteratively trained. The energy‐efficient virtual network embedding scheme was obtained when the energy function was balanced. To prove the effectiveness of the proposed algorithm, we designed two experimental environments, namely, a medium‐sized scenario and a small‐sized scenario. Simulation results show that the proposed algorithm achieved a superior performance and effectively decreased the energy consumption relative to the other methods in both scenarios. Furthermore, the proposed algorithm reduced the number of open nodes and open links leading to a reduction in the overall power consumption of the virtual network embedding process, while ensuring the average acceptance ratio and the average ratio of the revenue and cost.

Hyperspectral video sequences (HVSs) are well suited for gas plume detection (GPD). The high spectral resolution allows the detection of chemical clouds even when they are optically thin. Processing this new type of video sequences is challenging and requires advanced image and video analysis algorithms. In this paper, we propose a novel method for GPD recorded in HVSs. Based on the assumption that the background is stationary and the gas plume is moving, the proposed method separates the background from the gas plume via a low-rank and sparse decomposition. Furthermore, taking into consideration that the gas plume is continuous in both spatial and temporal dimensions, we include total variation regularization in the constrained minimization problem, which we solve using the augmented Lagrangian multiplier method. After applying the above process to each extracted feature, a novel fusion strategy is proposed to combine the information into a final detection result. Experimental results using real data sets indicate that the proposed method achieves very promising GPD performance.