key: cord-0045944-gcy4f402
authors: Borowski, Michał; Gora, Paweł; Karnas, Katarzyna; Błajda, Mateusz; Król, Krystian; Matyjasek, Artur; Burczyk, Damian; Szewczyk, Miron; Kutwin, Michał
title: New Hybrid Quantum Annealing Algorithms for Solving Vehicle Routing Problem
date: 2020-05-25
journal: Computational Science - ICCS 2020
DOI: 10.1007/978-3-030-50433-5_42
sha: 7e6ef9e7a113e47d418ed3340a74ea042ca68371
doc_id: 45944
cord_uid: gcy4f402

We introduce new hybrid algorithms, DBSCAN Solver and Solution Partitioning Solver, which use quantum annealing for solving Vehicle Routing Problem (VRP) and its practical variant: Capacitated Vehicle Routing Problem (CVRP). Both algorithms contain important classical components, but we also present two other algorithms, Full QUBO Solver and Average Partitioning Solver, which can be run only on a quantum processing unit (without CPU) and were prototypes which helped us develop better hybrid approaches. In order to validate our methods, we run comprehensive tests using D-Wave’s Leap framework on well-established benchmark test cases as well as on our own test scenarios built based on realistic road networks. We also compared our new quantum and hybrid methods with classical algorithms - well-known metaheuristics for solving VRP and CVRP. The experiments indicate that our hybrid methods give promising results and are able to find solutions of similar or even better quality than the tested classical algorithms.

Vehicle Routing Problem (VRP) is an important combinatorial optimization problem in which the goal is to find the optimal setting of routes for a fleet of vehicles which should deliver some goods from a given origin (depot) to a given set of destinations (customers) [1] . It is a generalization of the Travelling Salesman Problem (TSP) (introduced first as the Truck Dispatching Problem [1] ) in which one vehicle has to visit some number of destinations in the optimal way [2] . Both problems are proven to be NP-hard [3] . There exist the exact algorithms able to find optimal solutions in a reasonable time for relatively small instances, but generally, those problems are computationally difficult and the state-of-the-art approaches applied in practice are based on heuristics (constructive, improvement and composite) and metaheuristics [4, 5] .

Recently, we can observe a noticeable progress in the development of quantum computing algorithms and it turned out that they may be particularly successful in solving combinatorial optimization problems, such as TSP and VRP [6] . The first quantum algorithms for TSP and VRP already exist and in the scientific literature we can find algorithms which can be run on gate-based quantum computers [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] as well as quantum annealing algorithms which can be run on adiabatic quantum computers [18] [19] [20] [21] [22] [23] [24] .

In this paper, we present new methods for solving VRP and its more practical variant, CVRP (Capacitated Vehicle Routing Problem), in which all vehicles have a limited capacity. The algorithms introduced in this paper are based on quantum annealing, because due to the number of available qubits, those algorithms have currently a greater chance to give any practical improvement over classical algorithms.

We developed and present four algorithms: Full QUBO Solver (FQS), Average Partition Solver (AVS), DBSCAN Solver (DBSS) and Solution Partitioning Solver (SPS). The first and second one are designed only for solving VRP, DBSCAN solver can also solve CVRP if capacities of all vehicles are equal, SPS is able to solve CVRP with arbitrary capacities. It is also important to add that the last two methods are hybrid algorithms and they contain important components which should be run on classical processors.

In order to evaluate different algorithms for solving VRP using quantum annealing, we carried out series of experiments using D-Wave's Leap framework [25] which contains implementations of built-in solvers and allows to implement new solvers. We used QBSolv [26] run on quantum processing unit (QPU) and simulating quantum annealing on classical processors (CPU), as well as hybrid solver [27] run on both, QPU and CPU.

Beside quantum algorithms, we also wanted to test and compare several wellknown classical algorithms which gave good results in previous studies. Based on a comprehensive literature review [5] and further analysis, we selected 4 metaheuristics: based on simulated annealing [28] , bee algorithm [29] , evolutionary annealing [30] and recursive DBSCAN with simulated annealing [31] , respectively.

In order to reliably compare different algorithms, we conducted experiments on well-established benchmark datasets [32, 33] , as well as on datasets created by us, with realistic road networks (taken from the OpenStreetMap service) and artificially generated orders.

The rest of the paper is organized as follows: in Sect. 2, we describe in details all the quantum annealing solvers which we used in our experiments. Sections 3 and 4 present the design and results of our experiments, respectively. Section 5 outlines possible future research directions and concludes the paper.

In this section, we describe QUBO formulations and solvers which we developed for different variants of VRP: general VRP, CVRP with equal capacities and CVRP with arbitrary capacities. Before that, we introduce our notation and assumptions.

We assume that in each instance of VRP (or CVRP) we have a road network represented as a directed connected graph with vertices and edges. We also assume that the depots and destinations to which the orders of customers should be delivered are always located in vertices of the road network (in the case of benchmark instances and artificial networks, it may be even assumed that the road network is defined by locations of orders, while in the case of realistic road networks, real locations of orders are usually close enough to vertices determining the road network graph).

Let M be the number of available vehicles and N the number of orders. Let's denote the vehicles as V = {v 1 , v 2 , . . . , v M } and the orders as O = {o 1 , o 2 , . . . , o N }. We assume that there is always a depot located in one of the vertices (we also assume that destinations of orders are not located in the depot -such orders can be just served immediately so are not interesting) and all vehicles are initially located in the depot and should finish all routes back in the depot. Therefore, we have in total N + 1 significant vertices and without any loss of generality, we can assume that our graph has exactly N + 1 vertices and N * (N +1) directed edges (we can just consider edges built based on the shortest paths between every pair of vertices in the original graph), destination of the order o i is located in the vertex i and the depot is located in the vertex N + 1. We can also denote the cost of the direct travel from the vertex i (destination of the order o i ) to the vertex j (destination of the order o j ) as C i,j . We can also define C N +1,i and C i,N +1 for i ∈ {1, 2, . . . , N} as the costs of direct travels from the depot to the destinations of orders and from the destinations of orders to the depot, respectively.

Let's assume that x i,j,k = 1 if in a given setting the vehicle number i visits the vertex number j as k-th location on its route (for j ∈ {1, 2, . . . , N + 1} and k ∈ {0, 1, 2, . . . , N + 1}), otherwise x i,j,k = 0. We always have x i,N +1,0 = 1 and x i,j,0 = 0 for j < N + 1 (the depot is always the first location), and if x i,N +1,K = 1 for some K then for k > K x i,j,k = 1 (each vehicle stays in the depot after reaching it).

First, we defined a basic QUBO formulation used for solving VRP instances. The formulation is based on a similar formulation for TSP in [20] .

Let's define the binary function

It is easy to prove that the minimum value of A(y 1 , y 2 , ..., y n ) is equal to −1 and this value can be achieved only if exactly one of y 1 , y 2 , ..., y n is equal to 1.

By definition of VRP, the problem of minimizing the total cost can be defined as minimizing the function:

The first component of the sum C is a sum of all costs of travels from the depot -the first section of each cars' route. The second is a cost of the last section of a route (to depot) in a special case when a single car serves all N orders (only in such a case the component can be greater than 0). The last part is the cost of all other sections of routes.

To assure that each delivery is served by exactly one vehicle and exactly once, and that each vehicle is in exactly one place at a given time, the following term (in which all A components are equal to −1 only for such desired cases) should be included in our QUBO formulation:

By definition of VRP, QUBO representation of this optimization problem is

for some constants A 1 and A 2 , which should be set to ensure that the solution found by quantum annealer minimizes Q (which should be −N −NM) to ensure satisfiability of the aforementioned constraints (after running initial tests we set A 1 = 1, A 2 = 10 7 ).

APS is a variation of Full QUBO Solver for which we decrease the number of variables for each vehicle by assuming that every vehicle serves approximately the same number of orders. This means, every vehicle can serve up to A+L deliveries, where A is the total number of orders divided by the number of vehicles and L is a parameter (called "limit radius"), which controls the number of orders. The QUBO formulation is the same as in case of Full QUBO Solver but the number of variables x i,j,k is lower (M (A + L)N ), which simplifies computations.

DBSS allows us to use quantum approach combined with a classical algorithm. This particular algorithm is inspired by recursive DBSCAN [31] . DBSS uses recursive DBSCAN as a clustering algorithm with limited size of clusters. Then, TSP is solved for each cluster separately by FQS (just by assuming in the QUBO formulation that the number of vehicles equals 1). If the number of clusters is equal to the number of vehicles, the answer is known immediately. Otherwise, the solver runs recursively considering clusters as deliveries, so that each cluster contains orders which in the final result are served one after another without leaving the cluster. What is more, we concluded that by limit the total sum of weights of deliveries in clusters, this algorithm can solve CVRP if all capacities of vehicles are equal.

While adding capacity constraints is not simple, we were looking for the solution that can use results generated by DBSS. Therefore, we developed SPS. It is a simple algorithm which divides TSP solution found by another algorithm (e.g., DBSS) into consecutive intervals, which are the solution for CVRP. The idea is as follows:

. . , d j (in the order corresponding to TSP solution) and let cost i,j be the total cost of serving only orders d i , d i+1 , . . . , d j . Also, let dp i,S be the cost of the best solution for orders d 1 , d 2 , d 3 , ..., d i and for the set of vehicles S. Now, the dynamic programming formula for solving CVRP is given by:

where cost i,j = 0 and w i,j = 0 for i > j. Formula (6) returns a plenty of possible routes, but it also finds the optimal solution. We can speed it up by noticing that if two vehicles have the same capacity, it doesn't matter which one of them we choose, but pessimistically, capacities can be pairwise distinct. We propose the following heuristic to optimize this solution:

1. Instead of set S of vehicles, consider a sequence v 1 , v 2 , . . . , v M of vehicles and assume that we attach them to deliveries in such an order. 2. Now, our dynamic programming formula is given by:

3. To count this dynamic effectively, we can observe that:

So if we have counted dp for fixed k, then for counting dp for k+1 we can store all dp values for k and increase them, one by one (starting from i = j + 1), by a right side of Eq. 10. Using monotonic queue, we can get minimum in O(1) time.

We can now select some random permutations of vehicles and perform dynamic programming for each of them. The number of permutations can be regulated by additional parameter. With optimization of dynamic programming, the complexity of this algorithm is O(NMR), where R is the number of permutations. The greatest limitation of SPS is that it considers only one TSP solution. Nonetheless, we observed that DBSS for more than one vehicle works in a similar way.

The goal of our experiments was to test and compare different formulations of QUBO (solving different variants of VRP) on different datasets and with different solvers and settings (number of qubits and quota of time on quantum processor). We ran them using D-Wave's Leap platform [25] and its 2 solvers: qbsolv [26] and hybrid solver [27] . To run comprehensive and comparable experiments, we prepared several datasets: -Christofides1979 -a standard benchmark dataset for CVRP, well-known and frequently investigated by the scientific community [32,33], -A dataset built by us based on a realistic road network of Belgium, acquired from the OpenStreetMap service.

Christofides1979 consists of 14 tests, where each test instance is described by three files. The first one provides the number of vehicles and their capacity (the same for all vehicles). The second file describes the orders, i.e. their coordinates in 2−dimensional plane and the demand. The last file reports the time matrix (times of travel between various vertices in a graph). For a purpose of running our experiments and compare the results, we selected only 9 out of 14 tests because in case of other tests some hybrid or classical algorithms were not able to find any good solutions. All the important parameters describing Christofides1979 instances are given in Table 1 . In the case of the second dataset, we generated in total 51 tests. Each test was characterized by the number of orders. Table 2 presents a description of this dataset. Basically, it consists of 4 groups of test cases: small test (small number of orders), medium tests (medium number of orders), big tests (large number of order), mixed tests (various number of orders with some additional conditions). In every experiment, our programs computed the minimal cost of serving all orders. D-Wave's quantum annealing machine is naturally nondeterministic, so are the returned results, so for every algorithm and on every test case we ran 5 experiments. The code of programs used in our experiments is publicly available at [34] .

In this section, we present results of experiments conducted using QBSolv and hybrid solver built-in D-Wave's Leap framework and using algorithms described in Sect. 3.

First, we investigated Full QUBO Solver (FQS) on test cases small-0 -small-9. On every test except small-0, we ran experiments for 3 different numbers of vehicles (1, 2, 3) on quantum processor (FQS QPU [26]), its classical simulator (FQS CPU) and using a hybrid solver (FQS Hybrid [27] ). On small-0 there were only 2 orders so we tested only 1, 2 vehicles.

As we can see in Table 3 , QBSolv (FQS CPU and FQS QPU) exacerbates final results in test cases with more vehicles. For more vehicles, it can potentially generate the same solution as for less vehicles, because some vehicles can be just ignored. Solutions generated with hybrid solver (FQS Hybrid) confirm that. However, the size of QUBO makes the solutions with more vehicles unavailable for QBSolv. In hybrid solver, we have such a problem in only one case (small-9). However, in only 1 test case (small-3) QBSolv was able to improve the solution returned for smaller number of vehicles. In addition, in most cases QBSolv was not able to find a solution on QPU, the size of the instance and the number of the required variables and qubits was just too large. Also the required time of computations on QPU was worse than in case of CPU or hybrid approach. Therefore, we concluded that it doesn't make sense to run more experiments on QPU for larger test cases (with more cars and more orders) and we conducted next tests only using QBSolv on CPU and using a hybrid solver.

For larger VRP instances (medium-0 -medium-9), we observed that the transition from one vehicle to two vehicles is difficult. QBSolv usually returns much worse results (there is only 1 exception, test case medium-8). For the hybrid solver, in only one case the result for two vehicles is better (medium-6) but the results are usually still better than in case of QBSolv. We also noticed that the order of deliveries in tests with one vehicle was not optimal for majority of test cases. Only the least instances -with up to 15 orders -seem to be solved optimally. An interesting thing is that differences between results for two vehicles and one vehicle are very discrepant and it is not caused by the number of orders. By analyzing full results, we concluded that for 2 vehicles the solvers divided deliveries evenhandedly and for some tests it is a good way to build the optimal solution. We came up with an idea that since solvers found only these solutions, we can ask them to optimize only that kind of solutions, so we implemented Average Partition Solver, which demands less qubits.

We extended Full QUBO Solver with an option of changing the maximum difference between the number of deliveries attached to the vehicles, i.e., a deflection from the average number of deliveries per one vehicle. We found out experimentally that it should be 1 10 of the number of deliveries, which gives maximum difference in our test cases equal to 5. Having 1 vehicle, APS works exactly the same as Full QUBO Solver, so we ran experiments only for more vehicles (but we also included the results for 1 vehicle in Table 3 , just for comparison).

In most test cases, the results found using APS were better than results found by FQS. We can also notice that differences between results for 3 vehicles and results for 2 vehicles generated by APS are lower than the differences between results for 2 vehicles and 1 vehicle generated by FQS. However, in case of 3 vehicles, QBSolv on CPU still can't find better solutions with only 2 vehicles. The hybrid solver can find better solutions in cases with 3 vehicles than in cases with only 2 vehicles in 4 (out of 10) test cases. 

We can see in Table 3 that DBSS usually gives worse results than the APS, but we expected that it may change in case of tests with more orders thanks to utilizing the power of recursive DBSCAN. Indeed, on big test cases with a larger number of orders, DBSS gives much better results than APS (Table 4) . Additionally, DBSS can be run on larger instances and don't need assumption that every vehicle serves approximately the same number of deliveries (as it is in case of APS). 

At the beginning, we tested SPS on test cases where all capacities are equal, in order to compare results with DBSS which can solve this problem. The results are presented in Table 5 . In some cases, our solvers were not able to find the proper solutions (we mark such cases as "Not valid") but in general, SPS outperformed DBSS. Based on those experiments, we decided to test further only SPS and compare it with 4 classical algorithms -simulated annealing (SA), bee algorithm (BEE), evolutionary annealing (EA) and recursive DBSCAN with simulated annealing (DBSA). We ran next experiments with even more orders on mixed test cases generated by us (Table 2 ) and on benchmark datasets Christofides1979 ( Table 1) . The results are presented in Table 6 and Table 7 . Table 6 . Comparison of results achieved by Solution Partitioning Solver (SPS) and classical algorithms (SA -simulated annealing, BEE -Bee algorithm, EA -evolutionary annealing, DBSA -DBSCAN with simulated annealing) on a benchmark dataset Christofides79. 

We introduced new hybrid algorithms for solving VRP and CVRP and ran tests using D-Wave's Leap framework on well-established benchmark test cases and on our own test scenarios built based on realistic road networks. We also compared our new quantum and hybrid methods with classical algorithms -well-known metaheuristics for solving VRP and CVRP. The results indicate that our hybrid methods give promising results and are able to find solutions of a similar quality to the tested classical algorithms. Our primary future research direction is extending QUBO formulations to solve even more realistic variant of VRP -the Vehicle Routing Problem with Time Windows (VRPTW). Also, we are planning to compare our hybrid algorithms with even more classical algorithms for solving VRP and its variants.

The truck dispatching problem

XVIII. On the representation of polyedra

Reducibility among combinatorial problems

Vehicle routing: historical perspective and recent contributions

On a road to optimal fleet routing algorithms: a gentle introduction to the state-of-the-art. In: Smart Delivery Systems, Solving Complex Vehicle Routing Problems, Intelligent Data-Centric Systems

Combinatorial optimization on gate model quantum computers: a survey

Quantum-inspired evolutionary algorithm for travelling salesman problem

A novel hybrid column generation-metaheuristic approach for the vehicle routing problem with general soft time window

A quantum evolutionary algorithm for the vehicle routing problem with delivery time cost

Finding solutions to NP problems: philosophical differences between quantum and evolutionary search algorithms

A hybrid natural computing approach for the VRP problem based on PSO, GA and quantum computation

Efficient quantum algorithm for solving travelling salesman problem: an IBM quantum experience

A new improved quantum evolution algorithm with local search procedure for capacitated vehicle routing problem

Multiobjective quantum evolutionary algorithm for the vehicle routing problem with customer satisfaction

Bi-direction quantum crossover-based clonal selection algorithm and its applications

Quantum computing-based Ant Colony Optimization algorithm for TSP

A novel quantum swarm evolutionary algorithm and its applications

Quantum annealing of the travelingsalesman problem

Optimization using quantum mechanics: quantum annealing through adiabatic evolution

Ising formulations of many NP problems

A Near-Term Quantum Computing Approach for Hard Computational Problems in Space Exploration

Quantum adiabatic computation and the travelling salesman problem

A hybrid solution method for the capacitated vehicle routing problem using a quantum annealer

A Hybrid Solution Method for the Capacitated Vehicle Routing Problem Using a Quantum Annealer

A new capacitated vehicle routing problem with split service for minimizing fleet cost by simulated annealing

An artificial bee colony algorithm for the capacitated vehicle routing problem

A hybrid meta-heuristic for multi-objective vehicle routing problems with time windows

Solving high volume capacitated vehicle routing problem with time windows using recursive-DBSCAN clustering algorithm

The vehicle routing problem

Code used in our experiments

Acknowledgment. The presented research was carried out within the frame of the project "Green LAst-mile Delivery" (GLAD) realized at the University of Warsaw with the project partners: Colruyt Group, University of Cambridge and Technion. The project is supported by EIT Food, which is a Knowledge and Innovation Community (KIC) established by the European Institute for Innovation & Technology (EIT), an independent EU body set up in 2008 to drive innovation and entrepreneurship across Europe.

 small-1  1  10643  10643  10643  10643  10643  -2  10643  10643  10643  12379  12379  -3  10643  -10643  --small-2  1  21311  21311  21311  21311  21311  -2  21311  -21311  24508  24508  -3  22192  -21311 ---