A Computational Study of the Homogeneous Algorithm for Large-scale Convex Optimization

A large-scale continuous optimization benchmark suite with versatile coupled heterogeneous modules

In this paper, we propose the application of a Cooperative Co-evolutionary LSHADE (CCLSHADE) algorithm for Large-Scale Global Optimization (LSGO). We illustrate that by tuning two simple parameters of the CC framework, one can obtain very competitive results. The results are achieved without the need of incorporating local search modules, a re-initialization step, or adaptively configuring the CC framework budget allocation. The two parameters studied in this work are the number of iterations for which to run each sub-optimizer in a single cycle and the maximum size of the component containing the separable problem variables. The performance of CCLSHADE is compared against six state-of-the-art algorithms developed for LSGO using the CEC10 benchmarks. Experimental results and statistical tests confirm the competitiveness of the proposed algorithm.

We develop an algorithmic theory of convex optimization over discrete sets. Using a combination of algebraic and geometric tools we are able to provide polynomial time algorithms for solving broad classes of convex combinatorial optimization problems and convex integer programming problems in variable dimension. We discuss some of the many applications of this theory including to quadratic programming, matroids, bin packing and cutting-stock problems, vector partitioning and clustering, multiway transportation problems, and privacy and confidential statistical data disclosure. Highlights of our work include a strongly polynomial time algorithm for convex and linear combinatorial optimization over any family presented by a membership oracle when the underlying polytope has few edge-directions; a new theory of so-termed n-fold integer programming, yielding polynomial time solution of important and natural classes of convex and linear integer programming problems in variable dimension; and a complete complexity classification of high dimensional transportation problems, with practical applications to fundamental problems in privacy and confidential statistical data disclosure.

This work develops a convex optimization framework for image segmentation models, where both the unknown regions and parameters describing each region are part of the optimization process. Convex relaxations and optimization algorithms are proposed, which produce results that are independent from the initializations and closely approximate global minima. We focus especially on problems where the data fitting term depends on the mean or median image intensity within each region. We also develop a convex relaxation for the piecewise constant Mumford-Shah model, where additionally the number of regions is unknown. The approach is based on optimizing a convex energy potential over functions defined over a space of one higher dimension than the image domain.

In this paper, we propose a track routing algorithm for timing and yield optimization. The algorithm solves the problem in two stages: wire ordering, and wire spacing and sizing. The wire ordering problem is solved by an algorithm based on wire merging. For the wire spacing and sizing problem, we show that it can be represented as a Mixed Linear Geometric Programming (MLGP) problem which can be transformed into a convex optimization problem. Since general nonlinear convex optimization may take a long running time, we propose a heuristic that solves the problem much faster. Experimental results show that, compared to the algorithm that only optimizes yield, our algorithm is able to improve the minimum timing slack by 20%.

In this paper, we propose a framework to accelerate the computational efficiency of evolutionary algorithms on large-scale multiobjective optimization. The main idea is to track the Pareto optimal set (PS) directly via problem reformulation. To begin with, the algorithm obtains a set of reference directions in the decision space and associates them with a set of weight variables for locating the PS. Afterwards, the original large-scale multiobjective optimization problem is reformulated into a low-dimensional single-objective optimization problem. In the reformulated problem, the decision space is reconstructed by the weight variables and the objective space is reduced by an indicator function. Thanks to the low dimensionality of the weight variables and reduced objective space, a set of quasi-optimal solutions can be obtained efficiently. Finally, a multiobjective evolutionary algorithm is used to spread the quasi-optimal solutions over the approximate Pareto optimal front evenly. Experiments have been conducted on a variety of large-scale multiobjective problems with up to 5000 decision variables. Four different types of representative algorithms are embedded into the proposed framework and compared with their original versions, respectively. Furthermore, the proposed framework has been compared with two state-of-the-art algorithms for large-scale multiobjective optimization. The experimental results have demonstrated the significant improvement benefited from the framework in terms of its performance and computational efficiency in large-scale multiobjective optimization.

With the development of optimization heads toward large-scale problems, a series of optimization packages were developed for large-scale NLP optimization. The RSQP (reduced sequential quadratic programming) we concerned is one of best large scale NLP algorithm, and is especially efficient in process operation optimization. In this paper, the performance of our concerned algorithm RSQP, CONOPT2 and MINOS5 were tested by large-scale benchmarking COPS examples, the calculation was under the environment of GAMS. Computing results demonstrate that the performance of CONOPT2 was better than the others on robustness and efficiency. To problems with relatively small degrees of freedom, the performance of MINOS5 worked worst, though RSQP was not as robust as CONOPT2, its memory required reduced as half as CONOPT2. The results show that the RSQP we concerned has great advances for large scale optimization and further research for the algorithmpsilas stability is very necessary.

Meta-heuristic algorithms are widely viewed as feasible techniques to solve continuous large-scale numerical optimization problems. Grey wolf optimizer (GWO) is a relatively new stochastic algorithm with only a few parameters to adjust that can be easily used for global optimization. This paper presents an efficient and robust GWO (ERGWO) variant to solve large-scale numerical optimization problems. Inspired by particle swarm optimization, a nonlinearly adjustment strategy for parameter control is designed to balance exploration and exploitation. Additionally, a modified position-updating equation is presented to improve convergence speed. The performance of ERGWO is verified on 18 benchmark large-scale numerical optimization problems with dimensions ranging from 30 to 10,000, 30 benchmarks from CEC 2014, 30 functions in CEC 2017, respectively. Numerical experiments are performed to compare ERGWO to the basic GWO algorithm, other GWO variants, and other well-known meta-heuristic search techniques. Simulations demonstrate that the proposed ERGWO algorithm can find high quality solutions with low computational cost and very fast convergence.

This paper presents a review of history matching and oil field development optimization techniques with a focus on optimization algorithms. History matching algorithms are reviewed as a precursor to production optimization algorithms. Techniques for history matching and production optimization are reviewed including global and local methods. Well placement, well control, and combined well placement-control optimization using both secondary and tertiary oil production techniques are considered. Secondary and tertiary recovery techniques are commonly referred to as waterflooding and enhanced oil recovery (EOR), respectively. Benchmark models for comparison of methods are summarized while other applications of methods are discussed throughout. No single optimization method is found to be universally superior. Key areas of future work are combining optimization methods and integrating multiple optimization processes. Current challenges and future research opportunities for improved model validation and large scale optimization algorithms are also discussed.

Precise reconstruction of unknown quantum states from measurement data, a process commonly called quantum state tomography, is a crucial component in the development of quantum information processing technologies. Many different tomography methods have been proposed over the years. Maximum likelihood estimation is a prominent example, being the most popular method for a long period of time. Recently, more advanced neural network methods have started to emerge. Here, we go back to basics and present a method for continuous variable state reconstruction that is both conceptually and practically simple, based on convex optimization. Convex optimization has been used for process tomography and qubit state tomography, but seems to have been overlooked for continuous variable quantum state tomography. We demonstrate high-fidelity reconstruction of an underlying state from data corrupted by thermal noise and imperfect detection, for both homodyne and heterodyne measurements. A major advantage over other methods is that convex optimization algorithms are guaranteed to converge to the optimal solution.

Decaying with the increasing of signal propagation distance, Received Signal Strength (RSS) is used in the wireless localization due to its low cost and easily implementation.When the transmit power is unavailable, two convex optimization algorithms including semi definite programming (SDP), second order cone and semi definite programming (SOC/SDP) are designed to estimate the source locations by relaxing the non-convex problem as convex optimization.The corresponding Cramér-Rao lower bound (CRLB) of the problem is derived.The simulations demonstrate that the SOC/SDP algorithm provides the similar accuracy performance compared with the SDP algorithm.However, the computational complexity of SOC/SDP is lower than that of the SDP due to the less variables and equality constraints.When perfect knowledge of the path loss exponent is available, the simulations also show that the accuracy performance of the proposed convex optimization algorithms degrades as the path loss exponent increases.

Andersen and Ye [ Math. Programming, 84 (1999), pp. 375--399] suggested a homogeneous formulation and an interior-point algorithm for solution of the monotone complementarity problem (MCP). The advantage of the homogeneous formulation is that it always has a solution. Moreover, in the case in which the MCP is solvable or is (strongly) infeasible, the solution provides a certificate of optimality or infeasibility. In this note we demonstrate that if the suggested formulation is applied to the Karush--Kuhn--Tucker optimality conditions corresponding to a convex optimization problem, then an infeasibility certificate provides information about whether the primal or dual problem is infeasible given certain assumptions.

Ant-colony optimization (ACO) is a popular swarm intelligence metaheuristic scheme that can be applied to almost any optimization problem. In this paper, we address a performance evaluation of an ACO-based algorithm for solving large-scale global optimization problems with continuous variables, labeled Differential Ant-Stigmergy Algorithm (DASA). The DASA transforms a real-parameter optimization problem into a graph-search problem. The parameters' differences assigned to the graph vertices are used to navigate through the search space. The performance of the DASA is evaluated on the set of benchmark problems provided for CEC'2010 Special Session and Competition on Large-Scale Global Optimization.

Recent decades have witnessed the rise of data deluge generated by heterogeneous sources, e.g., social networks, streaming, marketing services etc., which has naturally created a surge of interests in theory and applications of large-scale convex and non-convex optimization. For example, real-world instances of statistical learning problems such as deep learning, recommendation systems, etc. can generate sheer volumes of spatially/temporally diverse data (up to Petabytes of data in commercial applications) with millions of decision variables to be optimized. Such problems are often referred to as Big-data problems. Solving these problems by standard optimization methods demands intractable amount of centralized storage and computational resources which is infeasible and is the foremost purpose of parallel and decentralized algorithms developed in this thesis.This thesis consists of two parts: (I) Distributed Nonconvex Optimization and (II) Distributed Convex Optimization.In Part (I), we start by studying a winning paradigm in big-data optimization, Block Coordinate Descent (BCD) algorithm, which cease to be effective when problem dimensions grow overwhelmingly. In particular, we considered a general family of constrained non-convex composite large-scale problems defined on multicore computing machines equipped with shared memory. We design a hybrid deterministic/random parallel algorithm to efficiently solve such problems combining synergically Successive Convex Approximation (SCA) with greedy/random dimensionality reduction techniques. We provide theoretical and empirical results showing efficacy of the proposed scheme in face of huge-scale problems. The next step is to broaden the network setting to general mesh networks modeled as directed graphs, and propose a class of gradient-tracking based algorithms with global convergence guarantees to critical points of the problem. We further explore the geometry of the landscape of the non-convex problems to establish second-order guarantees and strengthen our convergence to local optimal solutions results to global optimal solutions for a wide range of Machine Learning problems.In Part (II), we focus on a family of distributed convex optimization problems defined over meshed networks. Relevant state-of-the-art algorithms often consider limited problem settings with pessimistic communication complexities with respect to the complexity of their centralized variants, which raises an important question: can one achieve the rate of centralized first-order methods over networks, and moreover, can one improve upon their communication costs by using higher-order local solvers? To answer these questions, we proposed an algorithm that utilizes surrogate objective functions in local solvers (hence going beyond first-order realms, such as proximal-gradient) coupled with a perturbed (push-sum) consensus mechanism that aims to track locally the gradient of the central objective function. The algorithm is proved to match the convergence rate of its centralized counterparts, up to multiplying network factors. When considering in particular, Empirical Risk Minimization (ERM) problems with statistically homogeneous data across the agents, our algorithm employing high-order surrogates provably achieves faster rates than what is achievable by first-order methods. Such improvements are made without exchanging any Hessian matrices over the network. Finally, we focus on the ill-conditioning issue impacting the efficiency of decentralized first-order methods over networks which rendered them impractical both in terms of computation and communication cost. A natural solution is to develop distributed second-order methods, but their requisite for Hessian information incurs substantial communication overheads on the network. To work around such exorbitant communication costs, we propose a “statistically informed” preconditioned cubic regularized Newton method which provably improves upon the rates of first-order methods. The proposed scheme does not require communication of Hessian information in the network, and yet, achieves the iteration complexity of centralized second-order methods up to the statistical precision. In addition, (second-order) approximate nature of the utilized surrogate functions, improves upon the per-iteration computational cost of our earlier proposed scheme in this setting.

A topology-based single-pool decomposition framework for large-scale global optimization