Value function approximation for dynamic multi-period vehicle routing

Dynamic vehicle routing problem (DVRP) with single depot has received increasing interest from engineers and scientists. Dynamic multi-depot vehicle routing problem (DMDVRP), an extension of DVRP, however, has not received much attention. In our paper, a distance-based clustering approach is introduced to simplify the DMDVRP by allocating each customer to its nearest depot. Thus, DMDVRP is decomposed to a sequence of DVRPs. An improved ant colony optimisation (IACO) with ant-weight strategy and mutation operation is presented to optimise vehicle routing problem (VRP) in this paper. Moreover, to satisfy the real-time feature of DMDVRP, the nearest addition approach is used to handle the new orders occurring during a time slice on the basis of VRP solution. Finally, the computational results for 17 benchmark problems are reported to validate that IACO with the distance-based clustering approach is more suitable for solving DMDVRP.

Stochastic and dynamic vehicle routing problems gain increasing attention in the research community. In these problems, routing plans are dynamically updated based on realizations of stochastic information. Due to the complexity of the corresponding Markov decision processes (MDPs), the calculation of optimal policies for these problems is usually not possible and researchers draw on heuristical methods of approximate dynamic programming (ADP). These methods use simulation to approximate the value of a state and decision in the MDP. The simulations are either conducted offline or online. Offline methods such as value function approximations (VFAs) generally neglect the full detail of the state space due to aggregation. Online methods such as rollout algorithms (RAs) are often not able to capture decision and transition space sufficiently due to runtime limitations. In this paper, we alleviate this tradeoff by combining two methods of ADP, an online RA and an offline VFA in two ways. In addition to the integration of the VFA as a base policy into the online RA to strengthen the RA’s simulations, we also limit the RA’s simulation horizon, estimating the remaining reward-to-go again via the VFA. For two stochastic dynamic routing problems from the literature, we show how this combination outperforms state-of-the-art solutions while simultaneously reducing the required time for online calculations.

The Vehicle Routing Problem (VRP) is a complex and high-level set of routing problems. Two of its important variants are the Dynamic Vehicle Routing Problem (DVRP) and the Green Vehicle Routing Problem (GVRP). The first one has become a challenging research topic in the last two decades, in which not all informations are known in advance, but are revealed as the system progresses. The second one is seen as a new application and solution in new logistics patterns, more specifically, for finding routes of vehicles to serve a set of customers while minimizing the total amount of CO 2 emissions, by increasing the loading rate and reducing the number of empty trips could reduce from 10% to 40% km traveled and therefore CO 2 emissions [1]. In this paper, we combine these two variants (DVRP and GVRP), where we try to minimize, in the dynamic environment, the greenhouse gas especially the carbon dioxide CO 2 , which is the immediate consequence of the depletion of the ozone layer and time duration. We will call this combination the Dynamic Green Vehicle Routing Problem (DGVRP). To study this new problem, we present the technique employed to estimate the amount of CO 2 emissions, the emission matrix and integrate them into the DGVRP model, then we propose the hybridization based on Ant Colony Optimization (ACO) algorithm with a Large Neighborhood Search (LNS) algorithm to solve our problem. The effectiveness of this approach is tested on a set of the dynamic green problems instances, which are adopted in this work from the static GVRP benchmark datasets.

In the field of dynamic vehicle routing, the importance to integrate stochastic information about possible future events in current decision making increases. Integration is achieved by anticipatory solution approaches, often based on approximate dynamic programming (ADP). ADP methods estimate the expected mean values of future outcomes. In many cases, decision makers are risk-averse, meaning that they avoid “risky” decisions with highly volatile outcomes. Current ADP methods in the field of dynamic vehicle routing are not able to integrate risk-aversion. In this paper, we adapt a recently proposed ADP method explicitly considering risk-aversion to a dynamic vehicle routing problem with stochastic requests. We analyze how risk-aversion impacts solutions’ quality and variance. We show that a mild risk-aversion may even improve the risk-neutral objective.

Research on dynamic and stochastic vehicle routing problems received increasing interest in the last decade. It considers a novel problem class, aiming at an appropriate handling of dynamic events combined with the incorporation of stochastic information about possible future events. This survey summarises the recent literature in this area. Besides, the classification according to the available stochastic information, a new classification based on the point in time where substantial computational effort for determining decisions or decision policies arises, is introduced. Furthermore, the difference in solution quality is analysed between approaches which consider either purely dynamic or stochastic problems compared to those which consider both, stochastic and dynamic aspects. A graphical representation demonstrates the strength of the reviewed approaches incorporating dynamic and stochastic information. The survey also gives an overview on the intensity of research for the different problem classes and its benefit in recent years. Finally, guidelines and promising directions for further research are presented.

Dynamic vehicle routing problems (DVRPs) arise in several applications such as technician routing, meal delivery, and parcel shipping. We consider the DVRP with stochastic customer requests (DVRPSR), in which vehicles must be routed dynamically with the goal of maximizing the number of served requests. We model the DVRPSR as a multi-stage optimization problem, where the first-stage decision defines route plans for serving scheduled requests. Our main contributions are knapsack-based linear models to approximate accurately the expected reward-to-go, measured as the number of accepted requests, at any state of the stochastic system. These approximations are based on representing each vehicle as a knapsack with a capacity given by the remaining service time available along the vehicle’s route. We combine these approximations with optimal acceptance and assignment decision rules and derive efficient and high-performing online scheduling policies. We further leverage good predictions of the expected reward-to-go to design initial route plans that facilitate serving dynamic requests. Computational experiments on very large instances based on a real street network demonstrate the effectiveness of the proposed methods in prescribing high-quality offline route plans and online scheduling decisions.

On dynamic pickup and delivery vehicle routing with several time windows and waiting times

Recent advances in commercial vehicle operations (CVO), especially in communication and information technologies, allow the study of dynamic vehicle routing problems under new and updated information, such as traffic conditions and new customers. Two major operational benefits of CVO include ( a) dynamically assigning vehicles to time-sensitive demands, and ( b) efficiently rerouting vehicles according to current traffic conditions. In this research, stochastic vehicle routing problems (SVRP) are considered and extended to incorporate real-time information for dynamic vehicle routing problems. The SVRP model is formulated by a chance-constrained model and is solved by CPLEX with branch-and-bound techniques. Numerical experiments are conducted in a Taichung city network to investigate dynamic vehicle routing strategies under real-time information supply strategies and to assess the effectiveness of such strategies in a dynamic perspective.

Over the years, several variations of the dynamic vehicle routing problem (DVRP) have been considered due to its similarities with many real-world applications. Several methods have been applied to address DVRPs, in which ant colony optimization (ACO) has shown promising results due to its adaptation capabilities. In this chapter, we generate another variation of the DVRP with traffic factor and propose a memetic algorithm based on the ACO framework to address it. Multiple local search operators are used to improve the exploitation capacity and a diversity scheme based on random immigrants is used to improve the exploration capacity of the algorithm. The proposed memetic ACO algorithm is applied on different test cases of the DVRP with traffic factors and is compared with other peer ACO algorithms. The experimental results show that the proposed memetic ACO algorithm shows promising results.

Dynamic rerouting of a fleet of vehicles in agricultural operations through a Dynamic Multiple Depot Vehicle Routing Problem representation

: In this paper we consider the dynamic routing of unmanned aerial vehicles (UAVs) currently in operational use with the US Air Force. Dynamic vehicle routing problems (VRP) have always been challenging, and the airborne version of the VRP adds dimensions and difficulties not present in typical ground-based applications. Previous UAV routing work has focused on primarily on static, pre-planned situations; however, scheduling military operations, which are often ad-hoc, drives the need for a dynamic route solver that can respond to rapidly evolving problem constraints. With these considerations in mind, we examine the use of a Java-encoded metaheuristic to solve these dynamic routing problems, explore its operation with several general problem classes, and look at the advantages it provides in sample UAV routing problems. The end routine provides routing information for a UAV virtual battlespace simulation and allows dynamic routing of operational missions.

Demand coverage diversity based ant colony optimization for dynamic vehicle routing problems

While Approximate Dynamic Programming has successfully been used in many applications involving discrete states and inputs such as playing the games of Tetris or chess, it has not been used in many continuous state and input space applications. In this paper, we combine Approximate Dynamic Programming techniques and apply them to the continuous, nonlinear and high dimensional dynamics of a quadcopter vehicle. We use a polynomial approximation of the dynamics and sum-of-squares programming techniques to compute a family of polynomial value function approximations for different tuning parameters. The resulting approximations to the optimal value function are combined in a point-wise maximum approach, which is used to compute the online policy. The success of the method is demonstrated in both simulations and experiments on a quadcopter. The control performance is compared to a linear time-varying Model Predictive Controller (MPC). The two methods are then combined to keep the computational benefits of a short horizon MPC and the long term performance benefits of the Approximate Dynamic Programming value function as the terminal cost.

While Approximate Dynamic Programming has successfully been used in many applications involving discrete states and inputs such as playing the games of Tetris or chess, it has not been used in many continuous state and input space applications. In this paper, we combine Approximate Dynamic Programming techniques and apply them to the continuous, nonlinear and high dimensional dynamics of a quadcopter vehicle. We use a polynomial approximation of the dynamics and sum-of-squares programming techniques to compute a family of polynomial value function approximations for different tuning parameters. The resulting approximations to the optimal value function are combined in a point-wise maximum approach, which is used to compute the online policy. The success of the method is demonstrated in both simulations and experiments on a quadcopter. The control performance is compared to a linear time-varying Model Predictive Controller (MPC). The two methods are then combined to keep the computational benefits of a short horizon MPC and the long term performance benefits of the Approximate Dynamic Programming value function as the terminal cost.

An Innovative Method to Solve Classic and Dynamic Multi-Depot Vehicle Routing Problem