Optimizing Over Serial Dictatorships

Combinatorial Optimization (CO) problems have been intensively studied for decades with a wide range of applications. For some classic CO problems, e.g., the Traveling Salesman Problem (TSP), both traditional planning algorithms and the emerging reinforcement learning have made solid progress in recent years. However, for CO problems with nested sub-tasks, neither end-to-end reinforcement learning algorithms nor traditional evolutionary methods can obtain satisfactory strategies within a limited time and computational resources. In this paper, we propose an algorithmic framework for solving CO problems with nested sub-tasks, in which learning and planning algorithms can be combined in a modular way. We validate our framework in the Job-Shop Scheduling Problem (JSSP), and the experimental results show that our algorithm has good performance in both solution qualities and model generalizations.

In this Chapter, we review the virtues and limitations of the Hopfield neural network for tackling NP-hard combinatorial optimization problems (COPs). Then we discuss two new neural network models based on the noisy chaotic neural network, and applied the two methods to solving two different NP-hard COPs in communication networks. The simulation results show that our methods are superior to previous methods in solution quality. We also point out several future challenges and possible directions in this domain.

Combinatorial optimization is one of the areas that has been studied intensively in computer science and operations research. Solving a Combinatorial Optimization Problem (COP) is NP-hard [55] and it involves finding a set of feasible solutions with the least-cost in a discrete search space [90]. It becomes complicated when the COP being solved is with large scale data, constraints and variables. The significance of COPs can be found in various types of industrial applications. Examples include finding an optimized scan chains route in integrated chips testing and job scheduling in manufacturing industries. This thesis presents a generic Bee Colony Optimization (BCO) framework for solving different COPs. The framework realizes computationally the collective intelligence shown in bee foraging behaviour. In a bee colony, bees are sent out of their beehive and travel across different locations to discover and to collect food. When a bee found a food source, it would perform waggle dance upon flying back to its hive. Its intension is to recruit more bees towards the newly discovered food source. Waggle dance operates as an important communication tool among bees. This collective intelligence shown in the bee foraging behaviour is modelled algorithmically as a generic framework to solve different COPs. The generic BCO framework is designed using the object-oriented approach. A set of domain independent and abstract classes are defined. The domain independent classes implement the bee foraging behaviour and the abstract classes act as an interface to interact with domain specific classes. To solve a COP with the proposed framework, the implementation of domain specific tasks as defined in the abstract classes needs to be supplied to the framework. Besides the advantage of solving different COPs, the proposed BCO framework can be modified or enhanced flexibly and the effect of the enhancement will be applicable across all different COPs being solved.

Many combinatorial optimization problems can be reduced to the LP problems using the results from the field called “polyhedral combinatorics”. The main goal of polyhedral combinatorics is to represent the convex envelope of feasible points of the given combinatorial problem in the form of a system of linear equalities and inequalities. Obtained by this way LP problems often have an exponentially growing number of constraints. We know that at each step of ellipsoid method for LP we have to use no more than one constraint from the set of constraints which are not fulfilled at current point. In many cases the problem of finding such a constraint can be formulated in the form of a new combinatorial (in some sense polar to the original) optimization problem (so-called separation problem).

Various conic relaxations of quadratic optimization problems in nonnegative variables for combinatorial optimization problems, such as the binary integer quadratic problem, quadratic assignment problem (QAP), and maximum stable set problem have been proposed over the years. The binary and complementarity conditions of the combinatorial optimization problems can be expressed in several ways, each of which results in different conic relaxations. For the completely positive, doubly nonnegative and semidefinite relaxations of the combinatorial optimization problems, we discuss the equivalences and differences among the relaxations by investigating the feasible regions obtained from different representations of the combinatorial condition which we propose as a generalization of the binary and complementarity condition. We also study theoretically the issue of the primal and dual nondegeneracy, the existence of an interior solution and the size of the relaxations, as a result of different representations of the combinatorial condition. These characteristics of the conic relaxations affect the numerical efficiency and stability of the solver used to solve them. We illustrate the theoretical results with numerical experiments on QAP instances solved by SDPT3, SDPNAL+ and the bisection and projection method.

The cake cutting problem is concerned with the fair allocation of a divisible good among agents whose preferences vary over it. Recently, designing strategy-proof cake cutting mechanisms has caught considerable attention from AI and MAS researchers. Previous works assumed that an agent's utility fu nction is additive so that theoretical analysis becomes tractable. However, in practice, agents have non-additive utility over a resource. In this paper, we consider the all-or-nothing utility function as a representative example of non-additive utility because it can widely cover agents' preferences for such real-world resources as the usage of meeting rooms, time slots for computational resources, bandwidth usage, and so on. We first show the incompatibility between envy-freeness and Pareto efficiency when each agent has all-or-nothing utility. We next propose two strategy-proof mechanisms that satisfy Pareto efficiency, which are based on the serial dictatorship mechanism, at the sacrifice of envy-freeness. To address computational feasibility, we propose a heuristic-based allocation algorithm to find a near-optimal allocation in time polynomial in the number of agents, since the problem of finding a Pareto efficient allocation is NP-hard. As another approach that abandons Pareto efficiency, we develop an envy-free mechanism and show that one of our serial dictatorship based mechanisms satisfies proportionality in expectation, which is a weaker definition of proportionality. Finally, we evaluate the efficiency obtained by our proposed mechanisms by computational experiments.

Combinatorial optimizations are widely adopted in scientific and engineering applications, such as VLSI design, automated machine learning (AutoML), and compiler design. Combinatorial optimization problems are notoriously challenging to exactly solve due to the NP-hardness. Scientists have long discovered that numerically simulating classical nonlinear Hamiltonian systems can effectively solve many well-known combinatorial optimization problems. However, such physical simulation typically requires a massive amount of computation, which even outstrips the logic capability of modern reconfigurable digital fabrics. In this work, we proposed an FPGA-based general combinatorial optimization problem solver which achieved ultra-high performance and scalability. Specifically, we first reformulated a broad range of combinatorial optimization problems with a general graph-based data structure called the Ising model. Second, instead of utilizing classical simulated annealing to find an approximate solution, we utilized a new heuristic algorithm, simulated bifurcation, to search for solutions. Third, we designed an efficient hardware architecture to fully exploit FPGAs' potentials to accelerate the algorithm, and proposed three hardware-software co-optimizations to further improve the performance. By experimenting on benchmarks, our proposal outperformed the state-of-the-art simulated annealing optimization solver by up to 10.91 times.

Ant Colony Optimization (ACO) algorithms are a new branch of swarm intelligence. They have been applied to solve different combinatorial optimization problems successfully. Their performance is very promising when they solve small problem instances. However, the algorithms' time complexity increase and solution quality decrease for large problem instances. So, it is crucial to reduce the time requirement and at the same time to increase the solution quality for solving large combinatorial optimization problems by the ACO algorithms. This paper introduces a Local Search based ACO algorithm (LSACO), a new algorithm to solve large combinatorial optimization problems. The basis of LSACO is to apply an adaptive local search method to improve the solution quality. This local search automatically determines the number of edges to exchange during the execution of the algorithm. LSACO also applies pheromone updating rule and constructs solutions in a new way so as to decrease the convergence time. The performance of LSACO has been evaluated on a number of benchmark combinatorial optimization problems and results are compared with several existing ACO algorithms. Experimental results show that LSACO is able to produce good quality solutions with a higher rate of convergence for most of the problems.

Knowledge representation of mathematical optimization problems and constructs for modeling

In the intelligent era, combinatorial optimization problems—such as those in logistics scheduling, resource allocation, and intelligent transportation—have become increasingly prominent as core issues in numerous fields. Meanwhile, the rapid development of intelligent technologies has brought new opportunities for solving combinatorial optimization problems. This paper first introduces the basic concepts and classifications of combinatorial optimization problems. It then uses the bibliometric tool Citespace to analyze literature on digital-intelligent combinatorial optimization. Additionally, it comprehensively reviews and analyzes recent research progress from two core dimensions: intelligent optimization algorithms and artificial intelligence algorithms. The paper also summarizes application cases of combinatorial optimization methods in the intelligent era, such as logistics supply chain optimization, transportation route planning, manufacturing optimization, and financial prediction optimization, and discusses the shortcomings of current research. Finally, it prospects future research directions, providing important references for the theoretical development of combinatorial optimization.

A review on the studies employing artificial bee colony algorithm to solve combinatorial optimization problems

By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown promise in parametrizing and automating this laborious design process. The inductive bias of GNNs allows for learning solutions to mixed-integer programming (MIP) formulations of constrained CO problems with a relational representation of decision variables and constraints. The trained GNNs can be leveraged with primal heuristics to construct high-quality feasible solutions to CO problems quickly. However, current GNN-based end-to-end learning approaches have limitations for scalable training and generalization on larger-scale instances; therefore, they have been mostly evaluated over small-scale instances. Addressing this issue, our study builds on supervised learning of optimal solutions to the downscaled instances of given large-scale CO problems. We introduce several improvements on a recent GNN model for CO to generalize on instances of a larger scale than those used in training. We also propose a two-stage primal heuristic strategy based on uncertainty-quantification to automatically configure how solution search relies on the predicted decision values. Our models can generalize on 16x upscaled instances of commonly benchmarked five CO problems. Unlike the regressive performance of existing GNN-based CO approaches as the scale of problems increases, the CO pipelines using our models offer an incremental performance improvement relative to CPLEX. The proposed uncertainty-based primal heuristics provide 6-75% better optimality gap values and 45-99% better primal gap values for the 16x upscaled instances and brings immense speedup to obtain high-quality solutions. All these gains are achieved through a computationally efficient modeling approach without sacrificing solution quality.

By examining the patterns of solutions obtained for various instances, one can gain insights into the structure and behavior of combinatorial optimization (CO) problems and develop efficient algorithms for solving them. Machine learning techniques, especially Graph Neural Networks (GNNs), have shown promise in parametrizing and automating this laborious design process. The inductive bias of GNNs allows for learning solutions to mixed-integer programming (MIP) formulations of constrained CO problems with a relational representation of decision variables and constraints. The trained GNNs can be leveraged with primal heuristics to construct high-quality feasible solutions to CO problems quickly. However, current GNN-based end-to-end learning approaches have limitations for scalable training and generalization on larger-scale instances; therefore, they have been mostly evaluated over small-scale instances. Addressing this issue, our study builds on supervised learning of optimal solutions to the downscaled instances of given large-scale CO problems. We introduce several improvements on a recent GNN model for CO to generalize on instances of a larger scale than those used in training. We also propose a two-stage primal heuristic strategy based on uncertainty-quantification to automatically configure how solution search relies on the predicted decision values. Our models can generalize on 16x upscaled instances of commonly benchmarked five CO problems. Unlike the regressive performance of existing GNN-based CO approaches as the scale of problems increases, the CO pipelines using our models offer an incremental performance improvement relative to CPLEX. The proposed uncertainty-based primal heuristics provide 6-75% better optimality gap values and 45-99% better primal gap values for the 16x upscaled instances and brings immense speedup to obtain high-quality solutions. All these gains are achieved through a computationally efficient modeling approach without sacrificing solution quality.

The N-Queen problem is a classical combinatorial optimization and constraint satisfaction problem that involves placing N queens on an N × N chessboard such that no two queens threaten each other. Due to its exponential search space and increasing computational complexity with larger board sizes, the problem serves as an important benchmark for evaluating the efficiency of heuristic and metaheuristic optimization techniques. This study presents a comprehensive performance analysis of the Simulated Annealing (SA) optimization approach for solving the N-Queen problem across varying board dimensions. The proposed methodology models the N-Queen problem as an optimization task by defining an objective function that minimizes the number of conflicting queen pairs. Simulated Annealing is employed to iteratively explore the solution space, allowing probabilistic acceptance of suboptimal solutions to escape local minima. Key parameters, including initial temperature, cooling schedule, and termination criteria, are systematically tuned to evaluate their impact on convergence speed and solution quality. Experimental results demonstrate that the SA-based approach effectively finds valid solutions for large values of N with high success rates and reduced computational effort compared to exhaustive and deterministic methods. Performance metrics such as solution accuracy, convergence time, number of iterations, and conflict reduction rate are analyzed and discussed. The results indicate that Simulated Annealing offers a robust and scalable solution for the N-Queen problem, particularly for higher-dimensional instances where traditional methods become impractical. Overall, this study highlights the suitability of Simulated Annealing as an efficient metaheuristic for solving complex combinatorial optimization problems and provides insights into its performance characteristics when applied to the N-Queen problem.

Contributors