Continuous Relaxation for Discrete DC Programming

Sparse optimization is about finding minimizers of functions characterized by a number of nonzero components as small as possible, such paradigm being of great practical relevance in Machine Learning, particularly in classification approaches based on support vector machines. By exploiting some properties of the k-norm of a vector, namely, of the sum of its k largest absolute-value components, we formulate a sparse optimization problem as a mixed-integer nonlinear program, whose continuous relaxation is equivalent to the unconstrained minimization of a difference-of-convex function. The approach is applied to Feature Selection in the support vector machine framework, and tested on a set of benchmark instances. Numerical comparisons against both the standard ell _1-based support vector machine and a simple version of the Slope method are presented, that demonstrate the effectiveness of our approach in achieving high sparsity level of the solutions without impairing test-correctness.

A stochastic energy aware routing framework is proposed for electric vehicles with the goal of enhancing the sustainability of future electrified transportation systems. A stochastic decision making framework is introduced to handle the random effects of environmental factors on transportation energy cost. The problem of optimal routing is formulated as a stochastic programming problem and risk control of the overall energy is applied to find the minimum energy route. The original discrete optimization problem is tackled using convex relaxation and transformation. The optimal path is constructed using a highly efficient primal-dual interior point algorithm for the relaxed problem with a subsequent reconstruction of the solution for the original discrete problem. Two electric drive limitations, i.e., battery capacity and drive line efficiency, are made an integral part of the decision making framework in order to obtain realistic results by incorporating actual vehicle data. Detailed simulations demonstrate the utility of the introduced approach and its superiority to existing, traditional routing engines.

We consider the virtualization technique in the downlink transmission of limited fronthaul capacity cloud-radio access networks. A novel virtual computing resource allocation (VCRA) method which can dynamically split the users workload into smaller fragments to be served by virtual machines is presented. Under the proposed scheme, we aim at maximizing the network energy efficiency by a joint design of virtual computing resources, transmit beamforming, remote radio head (RRH) selection, and RRH-user association. Moreover, we construct a more realistic fronthaul power consumption model, which is directly proportional to users’ rate transmitted by the corresponding RRHs. The formulated problem is combinatorial and difficult to solve in general. Our first contribution is to customize a branch-and-reduce-and-bound method to attain a globally optimal solution. To compute a high-quality approximate solution, a standard routine is used to deal with the continuous relaxation of the original problem. However, the proposed continuous relaxation is non-convex which implies another challenge. For a practically appealing solution approach, we resort to a local optimization method, namely the difference of convex algorithm. Our second contribution is on the use of Lipschitz continuity to arrive at a sequence of convex quadratic programs, which can be solved efficiently by modern convex solvers. Finally, a post-processing procedure is proposed to obtain a high-performance feasible solution from the continuous relaxation. Extensive numerical results demonstrate that the proposed algorithms converge rapidly and achieve near-optimal performance as well as outperform other known methods. Moreover, we numerically show that the VCRA scheme significantly improves the system energy efficiency compared to the existing schemes.

Randomized decision making refers to the process of making decisions randomly according to the outcome of an independent randomization device, such as a dice roll or a coin flip. The concept is unconventional, and somehow counterintuitive, in the domain of mathematical programming, in which deterministic decisions are usually sought even when the problem parameters are uncertain. However, it has recently been shown that using a randomized, rather than a deterministic, strategy in nonconvex distributionally robust optimization (DRO) problems can lead to improvements in their objective values. It is still unknown, though, what is the magnitude of improvement that can be attained through randomization or how to numerically find the optimal randomized strategy. In this paper, we study the value of randomization in mixed-integer DRO problems and show that it is bounded by the improvement achievable through its continuous relaxation. Furthermore, we identify conditions under which the bound is tight. We then develop algorithmic procedures, based on column generation, for solving both single- and two-stage linear DRO problems with randomization that can be used with both moment-based and Wasserstein ambiguity sets. Finally, we apply the proposed algorithm to solve three classical discrete DRO problems: the assignment problem, the uncapacitated facility location problem, and the capacitated facility location problem and report numerical results that show the quality of our bounds, the computational efficiency of the proposed solution method, and the magnitude of performance improvement achieved by randomized decisions. Summary of Contribution: In this paper, we present both theoretical results and algorithmic tools to identify optimal randomized strategies for discrete distributionally robust optimization (DRO) problems and evaluate the performance improvements that can be achieved when using them rather than classical deterministic strategies. On the theory side, we provide improvement bounds based on continuous relaxation and identify the conditions under which these bound are tight. On the algorithmic side, we propose a finitely convergent, two-layer, column-generation algorithm that iterates between identifying feasible solutions and finding extreme realizations of the uncertain parameter. The proposed algorithm was implemented to solve distributionally robust stochastic versions of three classical optimization problems and extensive numerical results are reported. The paper extends a previous, purely theoretical work of the first author on the idea of randomized strategies in nonconvex DRO problems by providing useful bounds and algorithms to solve this kind of problems.

Collective Flow Diffusion Model (CFDM) is a general framework to find the hidden movements underlying aggregated population data. The key procedure in CFDM analysis is MAP inference of hidden variables. Unfortunately, existing approaches fail to offer exact MAP inferences, only approximate versions, and take a lot of computation time when applied to large scale problems. In this paper, we propose an exact and efficient method for MAP inference in CFDM. Our key idea is formulating the MAP inference problem as a combinatorial optimization problem called Minimum Convex Cost Flow Problem (C-MCFP) with no approximation or continuous relaxation. On the basis of this formulation, we propose an efficient inference method that employs the C-MCFP algorithm as a subroutine. Our experiments on synthetic and real datasets show that the proposed method is effective both in single MAP inference and people flow estimation with EM algorithm.

Hashing or binary code learning has been recognized to accomplish efficient near neighbor search, and has thus attracted broad interests in recent retrieval, vision, and learning studies. One main challenge of learning to hash arises from the involvement of discrete variables in binary code optimization. While the widely used continuous relaxation may achieve high learning efficiency, the pursued codes are typically less effective due to accumulated quantization error. In this paper, we propose a novel binary code optimization method, dubbed discrete proximal linearized minimization (DPLM), which directly handles the discrete constraints during the learning process. Specifically, the discrete (thus nonsmooth nonconvex) problem is reformulated as minimizing the sum of a smooth loss term with a nonsmooth indicator function. The obtained problem is then efficiently solved by an iterative procedure with each iteration admitting an analytical discrete solution, which is thus shown to converge very fast. In addition, the proposed method supports a large family of empirical loss functions, which is particularly instantiated in this paper by both a supervised and an unsupervised hashing losses, together with the bits uncorrelation and balance constraints. In particular, the proposed DPLM with a supervised ℓ2 loss encodes the whole NUS-WIDE database into 64-b binary codes within 10 s on a standard desktop computer. The proposed approach is extensively evaluated on several large-scale data sets and the generated binary codes are shown to achieve very promising results on both retrieval and classification tasks.

Characterizing linearizable QAPs by the level-1 reformulation-linearization technique

Systems reliability plays an important role in systems design, operation and management. Systems reliability can be improved by adding redundant components or increasing the reliability levels of subsystems. Determination of the optimal amount of redundancy and reliability levels among various subsystems under limited resource constraints leads to a mixed-integer nonlinear programming problem. The continuous relaxation of this problem in a complex system is a nonconvex nonseparable optimization problem with certain monotone properties. In this paper, we propose a convexification method to solve this class of continuous relaxation problems. Combined with a branch-and-bound method, our solution scheme provides an efficient way to find an exact optimal solution to integer reliability optimization in complex systems.

A convex optimization approach for solving the single-vehicle cyclic inventory routing problem

Convex sparse regularization techniques such as l <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</inf> -regularization enjoy robustness to random initialization and computational efficiency, but often underestimate the true solution. To mitigate the estimation bias, the generalized minimax concave (GMC) penalty (Selesnick 2017) and its linearly involved extension LiGME (Abe et al. 2020) have been proposed recently to yield improved convex sparse regularization. However, empirical results indicate that existing proximal splitting algorithms for GMC/LiGME do not necessarily achieve the convergence speed desired in some real-world applications (e.g., stream data processing).To resolve this situation, we propose a fast and well scalable algorithm (termed DC-LiGME) based on DC programming, which allows us to solve the difficult original problem via solution to a sequence of simpler subproblems. We establish convergence guarantee of the DC-LiGME algorithm and present an intuitive explanation of its fast convergence. Numerical experiments demonstrate superior convergence speed of the DC-LiGME algorithm, and suggest that mere two iterations of the algorithm suffice to stably produce satisfactory solutions (at the cost of solving three standard convex regularization models), which is a favourable feature in practice.

Optimal correction of an infeasible equations system as Ax + B|x|= b leads into a non-convex fractional problem. In this paper, a regularization method(ℓp-norm, 0 &lt; p &lt; 1), is presented to solve mentioned fractional problem. In this method, the obtained problem can be formulated as a non-convex and nonsmooth optimization problem which is not Lipschitz. The objective function of this problem can be decomposed as a difference of convex functions (DC). For this reason, we use a special smoothing technique based on DC programming. The numerical results obtained for generated problem show high performance and the effectiveness of the proposed method.

An exact method for scheduling a yard crane

A hybrid framework that uses Mathematical and Constraint Programming for the scheduling of batch chemical processes is proposed. Mathematical programming is used for the high-level optimization decisions (number and type of tasks, and assignment of equipment units to tasks), and Constraint Programming is used for the low-level sequencing decisions. The original problem is decomposed into an MILP master problem and a CP subproblem. The master MILP is a relaxation of the original problem, and given a relaxed solution, the CP subproblem checks whether there is a feasible solution and generates integer cuts. The proposed framework is based on the hybrid algorithm of Maravelias and Grossmann ([1],[2]), and can be used for different objective functions and different plant configurations. In this paper we present the simplifications and enhancements that allow us to use the proposed framework in a variety of problems, and report computational results.

This paper investigates the problem of joint base station selection and resource allocation in an orthogonal frequency division multiple access (OFDMA) heterogeneous cellular network. The original throughput maximization problem is NP-hard and we propose solving it by using game theoretic stochastic learning approaches. To this end, we first transform the original problem into a tractable form, which has a weighted utility function. Then we prove that an exact potential game applies and it exists the best Nash equilibria which is a near optimal solution of the original problem when an efficient solution method of the weights is employed. To obtain the optimal solution, we redesign the utility function by leveraging a state space to formulate the original problem into an ordinal state based potential game, which is proved that it exists a recurrent state equilibrium point that maximizes system throughput. Furthermore, we propose two different variants of Max-logit learning algorithm based on these two games respectively: one is a simultaneous learning algorithm with less information exchange, which achieves the best Nash equilibria point of the exact potential game and the other is an efficient learning algorithm for the ordinal state based potential game, which can converge to the global optimization solution. Finally, numerical results are given to validate that theoretical findings.

Optimal scheduling of air traffic over the entire National Airspace System is a computationally difficult task. To speed computation, Dantzig-Wolfe decomposition is applied to a known linear integer programming approach for assigning delays to flights. The optimization model is proven to have the block-angular structure necessary for Dantzig-Wolfe decomposition. The subproblems for this decomposition are solved in parallel via independent computation threads. Experimental evidence suggests that as the number of subproblems/threads increases (and their respective sizes decrease), the solution quality, convergence, and runtime improve. A demonstration of this is provided by using one flight per subproblem, which is the finest possible decomposition. This results in thousands of subproblems and associated computation threads. This massively parallel approach is compared to one with few threads and to standard (non-decomposed) approaches in terms of solution quality and runtime. Since this method generally provides a non-integral (relaxed) solution to the original optimization problem, two heuristics are developed to generate an integral solution. Dantzig-Wolfe followed by these heuristics can provide a near-optimal (sometimes optimal) solution to the original problem hundreds of times faster than standard (non-decomposed) approaches. In addition, when massive decomposition is employed, the solution is shown to be more likely integral, which obviates the need for an integerization step. These results indicate that nationwide, real-time, high fidelity, optimal traffic flow scheduling is achievable for (at least) 3 hour planning horizons.