Deep Policy Dynamic Programming for Vehicle Routing Problems

Formulations for Minimizing Tour Duration of the Traveling Salesman Problem with Time Windows

GVNS, which stands for General Variable Neighborhood Search, is an established and commonly used metaheuristic for the expeditious solution of optimization problems that belong to the NP-hard class. This paper introduces an expansion of the standard GVNS that borrows principles from quantum computing during the shaking stage. The Traveling Salesman Problem with Time Windows (TSP-TW) is a characteristic NP-hard variation in the standard Traveling Salesman Problem. One can utilize TSP-TW as the basis of Global Positioning System (GPS) modeling and routing. The focus of this work is the study of the possible advantages that the proposed unconventional GVNS may offer to the case of garbage collector trucks GPS. We provide an in-depth presentation of our method accompanied with comprehensive experimental results. The experimental information gathered on a multitude of TSP-TW cases, which are contained in a series of tables, enable us to deduce that the novel GVNS approached introduced here can serve as an effective solution for this sort of geographical problems.

Several issues related to the logistics field can be recognized as applications of the renowned Traveling Salesman Problem with Time Windows (TSPTW); examples of these issues include, among others, instance planning deliveries, managing internal logistics, bank couriers, material handling, but also production scheduling. In the light of such numerous applications, in this paper a hybrid algorithm based on the Divide-And-Conquer (DAC) technique and the Biased Randomized heuristic Algorithm (BRA) for solving the mentioned problem is presented. The aim is to propose a flexible solution suitable for implementation in many contexts where the TSPTW is relevant, thus improving performance and key indicators. The quality and reliability of the tool are validated on several benchmark problems through a comparison with a different algorithm already proposed in literature. In the light of the simulations carried out, it turned out to be effective and efficient when dealing with problems similar to those that characterize real applications, even in terms of computational time efficiency.

A general-purpose, simulation-based algorithm S-ACO for solving stochastic combinatorial optimization problems by means of the ant colony optimization (ACO) paradigm is investigated. Whereas in a prior publication, theoretical convergence of S-ACO to the globally optimal solution has been demonstrated, the present article is concerned with an experimental study of S-ACO on two stochastic problems of fixed-routes type: First, a pre-test is carried out on the probabilistic traveling salesman problem. Then, more comprehensive tests are performed for a traveling salesman problem with time windows (TSPTW) in the case of stochastic service times. As a yardstick, a stochastic simulated annealing (SSA) algorithm has been implemented for comparison. Both approaches are tested at randomly generated problem instances of different size. It turns out that S-ACO outperforms the SSA approach on the considered test instances. Some conclusions for fine-tuning S-ACO are drawn.

Vehicle routing problems (VRPs) are a fundamental class of combinatorial optimization problems (COPs) in computer science and operations research, with diverse real-world applications in logistics, transportation, and manufacturing. The intrinsic NP-hard nature makes VRPs exponentially expensive to be solved by exact solvers. As an alternative, heuristic solvers deliver suboptimal solutions within reasonable time but require extensive hand-crafted rules and domain expertise tailored to each specific problem. Recently, neural combinatorial optimization (NCO), which leverages machine learning to learn heuristics in a data-driven manner, has gained significant attention. These neural solvers demonstrate strong performance while reducing computational overhead and reliance on domain expertise compared to traditional solvers. However, they face significant generalization challenges. For example, their performance may degrade substantially when applied to instances with different data distributions, problem scales, or constraints than those encountered during training. This generalization issue severely limits their practical applicability. This thesis aims to systematically address these challenges and advance the development of generalizable neural solvers for VRPs. The thesis begins by enhancing the cross-distribution generalization of neural VRP solvers through an adversarial training approach. We introduce an ensemble-based Collaborative Neural Framework (CNF) that adversarially trains multiple models in a collaborative manner, promoting robustness against adversarial attacks while also boosting performance on clean instances. In this framework, an attacker generates hard instances by perturbing the data distribution, challenging the models to adapt and thereby strengthening cross-distribution generalization. Additionally, a neural router is designed to efficiently distribute training instances among the models, thereby improving load balancing and collaborative efficacy. Extensive experiments on two classic routing problems -- the travelling salesman problem (TSP) and the capacitated vehicle routing problem (CVRP) -- demonstrate the effectiveness and versatility of CNF in enhancing the cross-distribution generalization. Next, the thesis further expands neural solvers to simultaneously address both cross-size and cross-distribution generalization challenges. We propose Omni-VRP, a generic meta-learning framework that enables effective training of an initial model with the capacity for rapid adaptation to new tasks during inference, where each task corresponds to a set of instances with the same problem scale and data distribution. Additionally, we introduce a simple yet efficient approximation method to reduce the training overhead associated with second-order derivative calculations. Extensive experiments on both synthetic and benchmark instances of TSP and CVRP validate the effectiveness of our approach. Finally, the thesis pioneers the study of cross-problem generalization, pushing the boundaries of neural VRP solvers in learning generalizable representations across diverse constraints. We propose MVMoE, a unified neural solver capable of solving 16 VRP variants simultaneously in a zero-shot manner through attribute composition. To efficiently enhance model capacity, we employ a mixture-of-experts architecture with a hierarchical gating mechanism, striking a balance between empirical performance and computational complexity. Experimentally, our method significantly promotes zero-shot generalization on 10 unseen VRP variants, and showcases decent results on the few-shot setting and real-world benchmark instances. Overall, this thesis represents a pioneering exploration and substantial advancement in developing generalizable neural solvers for VRPs. It investigates innovative designs across various components, including model architectures, training algorithms, and problem settings, to enhance the generalization of neural solvers across diverse contexts such as varying data distributions, problem scales, and constraints. These contributions encourage neural solvers to learn more robust and generalizable representations, paving the way for the first generation of foundation models capable of addressing a wide range of COPs. Ultimately, the insights gained here not only propel the field of NCO forward but also enrich the broader landscape of learning-based optimization methods.

Combinatorial optimization problems are broadly studied in the literature. On the one hand, their challenging characteristics, such as the constraints and number of potential solutions, inspires their use to test new solution techniques. On the other hand, the practical application of these problems provides support of daily tasks of people and companies. Vehicle routing problems constitute a well-known class of combinatorial optimization problems, from which the Traveling Salesman Problem (TSP) is one of the most elementary ones. TSP corresponds to finding the shortest route that visits all cities within a path returning to the start city. Despite its simplicity, the difficulty in finding its exact solution and its direct application in practical problems in multiple areas make it one of the most studied problems in the literature. Algorithms inspired by biological phenomena are being successfully applied to solve optimization tasks, mainly combinatorial optimization problems. Those inspired by the collective behavior of insects produce good results for solving such problems. This article proposes the VRoptBees, a framework inspired by honeybee behavior to tackle vehicle routing problems. The framework provides a flexible and modular tool to easily build solutions to vehicle routing problems. Together with the framework, two examples of implementation are described, one to solve the TSP and the other to solve the Capacitated Vehicle Routing Problem (CVRP). Tests were conducted with benchmark instances from the literature, showing competitive results.

In order to improve the efficiency of logistics express delivery, and address the delivery and pick up problems in the process of logistics, this paper introduces the TSP problem algorithm into the research of terminal logistics path planning for the first time. The TSP problem model and the TSP problem model are established according to the different characteristics of delivery and pick-up process. The dynamic programming equations of delivery and pick-up process are obtained by using dynamic programming algorithm and they provide theoretical basis for the next step of intelligent delivery and pick-up.

This article concerns a green Time Dependent Capacitated Vehicle Routing Problem (TDCVRP) which is confronted in urban distribution planning. The problem is formulated as a Markovian Decision Process and a dynamic programming (DP) approach has been used for solving the problem. The article presents a performance comparison of two recent heuristics for the green TDCVRP that explicitly accounts for time dependent vehicle speeds and fuel consumption (emissions). These heuristics are the classical Restricted Dynamic Programming (RDP) algorithm, and the Simulation Based RDP that consists of weighted random sampling, RDP heuristic and simulation. The numerical experiments show that the Simulation Based Restricted Dynamic Programming heuristic can provide promising results within relatively short computational times compared to the classical Restricted Dynamic Programming for the green TDCVRP.

Vehicle Routing Problems (VRPs) are usually treated as static routing problem, since all the information is known before the routes been constructed. However, the real world VRPs are dynamic, where information relevant to the routing can change after the initial scheme has been done. This paper treats the dynamic VRPs with real time traffic information. We formulate the mathematic model of the VRPs, and propose a dynamic programming algorithm based on chaos optimization, whereas the route calculating module and dynamic programming module are introduced. Finally, the efficiency of the proposed algorithm is demonstrated by computer simulations.

Quantum computing is offering a novel perspective for solving combinatorial optimization problems. To fully explore the possibilities offered by quantum computers, the problems need to be formulated as unconstrained binary models, taking into account limitation and advantages of quantum devices. In this work, we provide a detailed analysis of the travelling salesman problem with time windows (TSPTW) in the context of solving it on a quantum computer. We introduce quadratic unconstrained binary optimization and higher-order binary optimization formulations of this problem. We demonstrate the advantages of edge-based and node-based formulations of the TSPTW problem. Additionally, we investigate the experimental realization of the presented methods on a quantum annealing device. The provided results pave the path for utilizing quantum computer for a variety of real-world tasks which can be cast in the form of travelling salesman problem with time windows.

In the aftermath of a large disaster, the routing of vehicles carrying critical supplies can greatly impact the arrival times to those in need. Because it is critical that the deliveries are both fast and fair to those being served, it is not clear that the classic cost-minimizing routing problems properly reflect the relevant priorities in disaster relief. In this paper, we take the first steps toward developing new methodologies for these problems. We focus specifically on two alternative objective functions for the traveling salesman problem (TSP) and the vehicle routing problem (VRP): one that minimizes the maximum arrival time (minmax) and one that minimizes the average arrival time (minavg). To demonstrate the potential impact of using these new objective functions, we bound the worst-case performance of optimal TSP solutions with respect to these new variants and extend these bounds to include multiple vehicles and vehicle capacity. Similarly, we examine the potential increase in routing costs that results from using these alternate objectives. We present solution approaches for these two variants of the TSP and VRP, which are based on well-known insertion and local search techniques. These are used in a series of computational experiments to help identify the types of instances in which TSP and VRP solutions can be significantly different from optimal minmax and minavg solutions.

The multiple traveling salesman problem: an overview of formulations and solution procedures

A note on the lifted Miller–Tucker–Zemlin subtour elimination constraints for routing problems with time windows

The garment cutting is a key process during the garment production. Most companies apply the manual labor or simple mechanical aids to achieve the goals. While these methods cost much time and labor. More and more automatic cutting equipment is applied to the garment cutting so as to save time, labor and materials. During the process of cutting, some problems are coming up, especially the cutting path. The cutting path of the garment numerical control cutter is regarded as generalized travelling salesman problem (GTSP). The garment contours can be regarded as the set of cities, and the nodes of a single contour can be regarded as cities. The cutter visits every contour exactly once. A hybrid intelligence algorithm was proposed to solve the problem. The ant colony algorithm was applied to a selected cutting path arbitrarily, an optimal contour sequence was found. Then the garment contour sequences shortest path was transformed into multi-segment graph shortest problem which is solved with the dynamic programming algorithm in order to optimize the knifes in-out point. The final optimal cutting path was constructed with ant colony optimization algorithm and dynamic programming algorithm. The practical application shows that the hybrid intelligence algorithm has satisfactory solution quality.

Recent works using deep learning to solve routing problems such as the traveling salesman problem (TSP) have focused on learning construction heuristics. Such approaches find good quality solutions but require additional procedures such as beam search and sampling to improve solutions and achieve state-of-the-art performance. However, few studies have focused on improvement heuristics, where a given solution is improved until reaching a near-optimal one. In this work, we propose to learn a local search heuristic based on 2-opt operators via deep reinforcement learning. We propose a policy gradient algorithm to learn a stochastic policy that selects 2-opt operations given a current solution. Moreover, we introduce a policy neural network that leverages a pointing attention mechanism, which can be easily extended to more general k-opt moves. Our results show that the learned policies can improve even over random initial solutions and approach near-optimal solutions faster than previous state-of-the-art deep learning methods for the TSP. We also show we can adapt the proposed method to two extensions of the TSP: the multiple TSP and the Vehicle Routing Problem, achieving results on par with classical heuristics and learned methods.