Approximation Algorithms for Reliable Stochastic Combinatorial Optimization

In this book, we have concentrated on those optimization problems which allow efficient (that is, polynomial time) algorithms. In contrast, the final chapter deals with an archetypical NP-complete problem: the travelling salesman problem already introduced in Chap. 1. It is one of the most famous and important problems in all of combinatorial optimization—with manyfold applications in such diverse areas as logistics, genetics, telecommunications, and neuroscience—and has been the subject of extensive study for about 60 years. We saw in Chap. 2 that no efficient algorithms are known for NP-complete problems, and that it is actually quite likely that no such algorithms can exist. Now we address the question of how such hard problems—which regularly occur in practical applications—might be approached: one uses, for instance, approximation techniques, heuristics, relaxations, post-optimization, local search, and complete enumeration. We shall explain these methods only for the TSP, but they are typical for dealing with hard problems in general. We will also brie y explain the idea of a further extremely important approach—via polyhedra—to solving hard problems and present a list of notable large scale TSPs which were solved to optimality.KeywordsMinimal Span TreeTravel Salesman ProblemLarge InstanceLocal Search AlgorithmHamiltonian PathThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

A penalty adapting ant algorithm is presented in an attempt to eliminate the dependency of ant algorithms on the penalty parameter used for the solution of constrained optimization problems. The method uses an adapting mechanism for determination of the penalty parameter leading to elimination of the costly process of penalty parameter tuning. The method is devised on the basis of observation that for large penalty parameters, infeasible solutions will have a higher total cost than feasible solutions and vice versa. The method therefore uses the best feasible and infeasible solution costs of the iteration to adaptively adjust the penalty parameter to be used in the next iteration. The pheromone updating procedure of the max–min ant system is also modified to keep ants on and around the boundary of the feasible search space where quality solutions can be found. The sensitivity of the proposed method to the initial value of the penalty parameter is investigated and indicates that the method converges to optimal or near-optimal solutions irrespective of the initial starting value of the penalty parameter. This is significant as it eliminates the need for sensitivity analysis of the method with respect to the penalty factor, thus adding to the computational efficiency of ant algorithms. Furthermore, it is shown that the success rate of the search algorithm in locating an optimal solution is increased when a self-adapting mechanism is used. The presented method is applied to a benchmark pipe network optimization problem in the literature and the results are presented and compared with those of existing algorithms.

The security of public-key cryptography relies on well-studied hard problems, problems for which we do not have efficient algorithms. Factorization and discrete logarithm are the two most known and used hard problems. Unfortunately, they can be easily solved on a quantum computer by Shor's algorithm. Also, the research area of cryptography demands for crypto-diversity which says that we should offer a range of hard problems for public-key cryptography. If one hard problem proves to be easy, we should be able to provide alternative solutions. Some of the candidates for post-quantum hard problems, i.e. problems which are believed to be hard even on a quantum computer, are the Learning Parity with Noise (LPN), the Learning with Errors (LWE) and the Shortest Vector Problem (SVP). A thorough study of these problems is needed in order to assess their hardness. In this thesis we focus on the algorithmic study of LPN. LPN is a hard problem that is attractive, as it is believed to be post-quantum resistant and suitable for lightweight devices. In practice, it has been employed in several encryption schemes and authentication protocols. At the beginning of this thesis, we take a look at the existing LPN solving algorithms. We provide the theoretical analysis that assesses their complexity. We compare the theoretical results with practice by implementing these algorithms. We study the efficiency of all LPN solving algorithms which allow us to provide secure parameters that can be used in practice. We push further the state of the art by improving the existing algorithms with the help of two new frameworks. In the first framework, we split an LPN solving algorithm into atomic steps. We study their complexity, how they impact the other steps and we construct an algorithm that optimises their use. Given an LPN instance that is characterized by the noise level and the secret size, our algorithm provides the steps to follow in order to solve the instance with optimal complexity. In this way, we can assess if an LPN instance provides the security we require and we show what are the secure instances for the applications that rely on LPN. The second framework handles problems that can be decomposed into steps of equal complexity. Here, we assume that we have an adversary that has access to a finite or infinite number of instances of the same problem. The goal of the adversary is to succeed in just one instance as soon as possible. Our framework provides the strategy that achieves this. We characterize an LPN solving algorithm in this framework and show that we can improve its complexity in the scenario where the adversary is restricted. We show that other problems, like password guessing, can be modeled in the same framework.

An inverse optimization problem is defined as follows: Let S denote the set of feasible solutions of an optimization problem P, let c be a specified cost (capacity) vector , and x 0 ∈ S. We want to perturb the cost (capacity) vector c to d such that x 0 becomes an optimal solution of P with respect to the cost (capacity) vector d, and to minimize some objective functions. In this paper, we consider the weighted inverse minimum cut problem under the sum-type Hamming distance. First, we show the general case is NP-hard. Second we present a combinatorial algorithm that run in strongly polynomial time to solve a special case.

The purpose of this work is to investigate some of the algorithms used for representing switching functions by space efficient decision diagrams. This representation is an optimization problem. Adopting the terminology from combinatorial optimization problems, we have:Problem Instance: A multivalued function f(x) where x is a vector, and a storing cost, s(xi), associated with each variable xi in x.Feasible Solution: A decision diagram D (acyclic, directed graph with indegrees of the internal nodes possibly exeeding one) representing f(x).Objective Function: The total storage cost,S(D) = Σ s(xi) for each feasible solution D.xie DOptimal Solution: A decision diagram D representing the function with minimum total storage cost S(D), i.e. a feasible solution which minimizes the objective function.To date, only two algorithms for the construction of optimal decision diagrams with respect to the total storage cost have been developed; Reinwald & Soland's branch & bound algorithm and Thayse's, based on the extension of Shannon's expansion formula. The second algorithm is analyzed and proven to be highly exponential in the number of variables. This comes as no surprise since, in most cases, the construction of optimal decision trees is an NP-hard problem. The construction of decision diagrams is generally more complicated than that of trees. One cannot seriously doubt that the constructions of optimal diagrams is also NP-hard for most costs including the total storage cost. Therefore, a heuristic approach to this problem is needed. Two heuristics, based on the greedy algorithmic technique, are presented. These heuristics with the analysis of Thayse's algorithm constitute the major achievements of this work.

In this paper, we consider an interesting variant of the inverse minimum traveling salesman problem. Given an in-stance ( G , w ) of the minimum traveling salesman problem defined on a metric space, we fix a specified Hamiltonian cycle HC 0 . The task is then to adjust the edge cost vector w to w ' so that the new cost vector w ' satisfies the triangle inequality condition and HC 0 can be returned by the minimum spanning tree algorithm in the TSP-instance defined with w '. The objective is to minimize the total deviation between the original and the new cost vectors with respect to the L 1 -norm. We call this problem the inverse metric traveling salesman problem against the minimum spanning tree algorithm and show that it is closely related to the inverse metric spanning tree problem. Keywords: Combinatorial Optimization, Inverse Optimization, TSP, Minimum Spanning Tree Algorithm * Corresponding Author, E-mail: yerimchung@yonsei.ac.kr 1. INTRODUCTION In (Chung, 2010; Chung and Demange, 2012), some variant of the inverse optimization problem is defined with respect to a specific algorithm. For a combinatorial optimization problem, we are given an instance

Stochastic optimization algorithms — an application to pattern matching

Linear optimization is many times algorithmically simpler than nonlinear convex optimization. Linear optimization over matroid polytopes, matching polytopes, and path polytopes are examples of problems for which we have simple and efficient combinatorial algorithms but whose nonlinear convex counterpart is harder and admits significantly less efficient algorithms. This motivates the computational model of convex optimization, including the offline, online, and stochastic settings, using a linear optimization oracle. In this computational model we give several new results that improve on the previous state of the art. Our main result is a novel conditional gradient algorithm for smooth and strongly convex optimization over polyhedral sets that performs only a single linear optimization step over the domain on each iteration and enjoys a linear convergence rate. This gives an exponential improvement in convergence rate over previous results. Based on this new conditional gradient algorithm we give the first algorithms for online convex optimization over polyhedral sets that perform only a single linear optimization step over the domain while having optimal regret guarantees, answering an open question of Kalai and Vempala and of Hazan and Kale. Our online algorithms also imply conditional gradient algorithms for nonsmooth and stochastic convex optimization with the same convergence rates as projected (sub)gradient methods.

Given an undirected graph G = (V, E) and a cost vector c ∈ ℝE, the weighted girth problem is to find a circuit in G having minimum total cost. This problem is in general 𝒩𝒫-hard. A promising approach to the solution of hard combinatorial optimization problems is given by the so-called cutting plane methods. These involve linear programming techniques based on a partial description of the convex hull of the incidence vectors of possible solutions. We consider the weighted girth problem in the case where G is the complete graph Kn and study the facial structure of the circuit polytope PCn and some related polyhedra. In the appendix we give complete characterizations of PCn for n up to 6. We were motivated to study the weighted girth problem by the fact that this problem and variations of it appear as a subproblem, namely the pricing problem, in a column generation approach to the vehicle routing problem.

Combinatorial optimization problems have very important applications in information technology, transportation, economics, management, network communication, and other fields. Since the problem size in real-scenario application is in large-scale, the demand for real-time and efficient solving approaches increases rapidly. The traditional exact methods guarantee the optimality of the final solution, but these methods can hardly solve the problem in acceptable time due to extremely high computational costs. Heuristic approaches can find feasible solutions in a limited time, while these approaches cannot meet the demand of solution quality. In recent years, hybrid algorithms based on exact methods and heuristic algorithms show outstanding performance in solving large-scale combinatorial optimization problems. The hybridization not only overcomes the shortcomings from single algorithm but also fully utilizes the search ability for population-based approaches as well as the interpretability in exact methods, which promotes the application of combinatorial optimization in real-world problems. This paper reviews existing studies on hybrid algorithms combining exact method and evolutionary computation, summarizes the characteristics of the existing algorithms, and directs the future research.

Branch-and-cut (-and-price) algorithms belong to the most successful techniques for solving mixed integer linear programs and combinatorial optimization problems to optimality (or, at least, with certified quality). In this unit, we concentrate on sequential branch-and-cut for hard combinatorial optimization problems, while branch-and-cut for general mixed integer linear programming is treated in [? Martin] and parallel branch-and-cut is treated in [? Ladanyi/Ralphs/Trotter]. After telling our most recent story ofa successful application of branch-and-cut in Section 1, we give in Section 2 a brief review ofthe history, including the contributions of pioneers with an emphasis on the computational aspects of their work. In Section 3, the components ofa generic branch-and-cut algorithm are described and illustrated on the traveling salesman problem. In Section 4, we first elaborate a bit on the important separation problem where we use the traveling salesman problem and the maximum cut problem as examples, then we show how branch-and-cut can be applied to problems with a very large number of variables (branch-and-cut-and-price). Section 5 is devoted to the design and applications of the ABACUS software framework for the implementation of branch-and-cut algorithms. Finally, in Section 6, we make a few remarks on the solution of the exercise consisting of the design of a simple TSP-solver in ABACUS.

In this paper, we adapt a genetic algorithm for constrained optimization problems. We use a dynamic penalty approach along with some form of annealing, thus forcing the search to concentrate on feasible solutions as the algorithm progresses. We suggest two different general-purpose methods for guaranteeing convergence to a globally optimal (feasible) solution, neither of which makes any assumptions on the structure of the optimization problem. The former involves modifying the GA evolution operators to yield a Boltzmann-type distribution on populations. The latter incorporates a dynamic penalty along with a slow annealing of acceptance probabilities. We prove that, with probability one, both of these methods will converge to a globally optimal feasible state.

Abstract. The characteristic form of the Saint-Venant equations is solved in a stochastic setting by using a newly proposed Fokker–Planck Equation (FPE) methodology. This methodology computes the ensemble behavior and variability of the unsteady flow in open channels by directly solving for the flow variables' time–space evolutionary probability distribution. The new methodology is tested on a stochastic unsteady open-channel flow problem, with an uncertainty arising from the channel's roughness coefficient. The computed statistical descriptions of the flow variables are compared to the results obtained through Monte Carlo (MC) simulations in order to evaluate the performance of the FPE methodology. The comparisons show that the proposed methodology can adequately predict the results of the considered stochastic flow problem, including the ensemble averages, variances, and probability density functions in time and space. Unlike the large number of simulations performed by the MC approach, only one simulation is required by the FPE methodology. Moreover, the total computational time of the FPE methodology is smaller than that of the MC approach, which could prove to be a particularly crucial advantage in systems with a large number of uncertain parameters. As such, the results obtained in this study indicate that the proposed FPE methodology is a powerful and time-efficient approach for predicting the ensemble average and variance behavior, in both space and time, for an open-channel flow process under an uncertain roughness coefficient.

In this paper, we study a stochastic variant of the celebrated k-server problem. In the k-server problem, we are required to minimize the total movement of k servers that are serving an online sequence of t requests in a metric. In the stochastic setting we are given t independent distributions <P_1, P_2,..., P_t> in advance, and at every time step i a request is drawn from Pi. Designing the optimal online algorithm in such setting is NP-hard, therefore the emphasis of our work is on designing an approximately optimal online algorithm. We first show a structural characterization for a certain class of non-adaptive online algorithms. We prove that in general metrics, the best of such algorithms has a cost of no worse than three times that of the optimal online algorithm. Next, we present an integer program that finds the optimal algorithm of this class for any arbitrary metric. Finally, by rounding the solution of the linear relaxation of this program, we present an online algorithm for the stochastic k-server problem with the approximation factor of 3 in the line and circle metrics and O(log n) in a general metric of size n. Moreover, we define the Uber problem, in which each demand consists of two endpoints, a source and a destination. We show that given an a-approximation algorithm for the k-server problem, we can obtain an (a+2)-approximation algorithm for the Uber problem. Motivated by the fact that demands are usually highly correlated with the time we study the stochastic Uber problem. Furthermore, we extend our results to the correlated setting where the probability of a request arriving at a certain point depends not only on the time step but also on the previously arrived requests.

The paper presents a general-purpose algorithm for solving stochastic combinatorial optimization problems with the expected value of a random variable as objective and deterministic constraints. The algorithm follows the Ant Colony Optimization (ACO) approach and uses Monte-Carlo sampling for estimating the objective. It is shown that on rather mild conditions, including that of linear increment of the sample size, the algorithm converges with probability one to the globally optimal solution of the stochastic combinatorial optimization problem. Contrary to most convergence results for metaheuristics in the deterministic case, the algorithm can usually be recommended for practical application in an unchanged form, i.e., with the ”theoretical” parameter schedule.KeywordsAnt colony optimizationcombinatorial optimizationconvergence resultsmetaheuristicsMonte-Carlo simulationstochastic optimization