key: cord-0262798-bogkj4bq authors: Ruvzivcka, Marek; Volovsin, Marcel; Gazda, Juraj; Maksymyuk, Taras; Han, Longzhe; Dohler, Mischa title: Fast and computationally efficient generative adversarial network algorithm for unmanned aerial vehicle-based network coverage optimization date: 2022-03-25 journal: nan DOI: 10.1177/15501477221075544 sha: 9b94e535e41b80f6f91977739582743c2f3c6074 doc_id: 262798 cord_uid: bogkj4bq The challenge of dynamic traffic demand in mobile networks is tackled by moving cells based on unmanned aerial vehicles. Considering the tremendous potential of unmanned aerial vehicles in the future, we propose a new heuristic algorithm for coverage optimization. The proposed algorithm is implemented based on a conditional generative adversarial neural network, with a unique multilayer sum-pooling loss function. To assess the performance of the proposed approach, we compare it with the optimal core-set algorithm and quasi-optimal spiral algorithm. Simulation results show that the proposed approach converges to the quasi-optimal solution with a negligible difference from the global optimum while maintaining a quadratic complexity regardless of the number of users. We are rapidly developing a digitized, highly connected, and data-driven society, where most of the business and industrial operations and citizen's daily routines rely on ubiquitous wireless connectivity. To date, the growing demand for data-hungry mobile applications has forced operators to deploy many additional small cells to accommodate the ever-increasing traffic volumes. However, the gains achieved by the small cells are very sensitive to the mobility pattern of user equipments (UEs). Typically, each city has its own unique social and economic features, which affect the movement of citizens and are considered by mobile operators when deploying small cells in the most effective way. However, there are also various exogenous factors that can temporary change the typical mobility patterns, such as big social events or sports tournaments. The most recent example is the coronavirus disease 2019 (COVID- 19) pandemic, which has caused a massive change in the lifestyle and typical mobility patterns of people worldwide due to massive lockdowns and quarantine restrictions. According to the Google Mobility Trends Report, in 2020, the activity of UEs in business areas has dropped by 30%, while the activity in residential areas and parks has increased by 20% and 109%, respectively. 1 Such challenges of abrupt changes in UEs mobility can be effectively solved by moving cells based on unmanned aerial vehicles (UAVs). Recently, UAV-based wireless networks have been gaining momentum for various applications such as disaster recovery, telemetry, or military communications. 2 However, to be used effectively as an alternative to the cellular mobile network infrastructure, UAVs need to be optimally placed within the target coverage area. There are many proposed solutions for the deployment of UAV-based communication systems, which are covered in recent surveys and research articles. [3] [4] [5] [6] [7] [8] [9] In this article, we propose a novel solution for optimal coverage design based on a conditional generative adversarial network (cGAN). cGANs belong to the class of generative deep learning models, which are widely adopted for various challenges in fifthgeneration (5G) systems. 10, 11 The basic motivation for using a cGAN is its ability to learn a mapping from input data to a target data distribution. While most of the cGAN research targets image or video generation, there is tremendous potential for cGAN technology in other fields of computer science, such as clustering 12 and intrusion detection. 13 In our contribution, the input data are represented by the UEs distribution in the space, and the target data are represented by the UAVs distribution providing the coverage for UEs. The computation of the optimal UAVs distribution is generally NP-hard 9 and thus, due to the extreme time limitations, not feasible in practice. We believe that through the fast and computationally efficient cGAN application, we can provide a sufficiently accurate approximation of the distributions of UAVs with a reduced time complexity. As the baseline for comparison, we chose two algorithms, which provide either the optimal solution for the UAV-based coverage, termed the core-set algorithm, 7, 8 or a quasi-optimal solution, termed the spiral algorithm. 9 Note that in the article, we do not target the specific mobility of UAVs but focus our attention on the computation of the UAVs positions for a given snapshot of UEs positions. Nevertheless, an important characteristic is that our algorithm is capable of cooperating with specific algorithms that address the mobility patterns of UAVs (e.g. based on the application of reinforcement learning (RL)) to achieve joint superior performance even in very complex and dynamic scenarios. The remainder of this article is organized as follows. The Related work section briefly summarizes the recent advancements in UAV coverage optimization. In the System model section, we briefly describe the setup of the system model, basic assumptions, and constraints. In the Proposed approach section, we provide a detailed explanation of the proposed approach and implementation of the cGAN workflow for UAV-based coverage optimization. We split this section into two main subsections covering the training phase and the deployment phase. In the Performance evaluation and discussion section, we simulate and discuss the performance of the proposed approach. Finally, we conclude the article in the Conclusion section. The problem of optimal placement of UAVs can be formulated as a mixed integer nonlinear problem, which is generally NP-hard. Thus, we seek approximate solutions based on the application of either differential evolution strategies or machine learning algorithms. Among the former methods, we can appreciate the contribution of Plachy et al., 14 where the authors elaborated on the application of particle swarm optimization (PSO) for UAVs placement, reflecting the instantaneous positions of UEs. Later, the model was extended by including the interference between the base stations and UEs by leveraging the fundamental electricity forces and well-known Coulomb's law. 15 The optimal path of UAVs that maintains a fixed operational altitude and avoids obstacles was provided by Gonzales et al. 16 The authors suggested the application of the differential evolution concept for a feasible UAV path. The proposed approach is of special relevance to 5G/ sixth-generation (6G) communication systems, as it deals with different vision fields of the UAV, which is very common in 6G communication systems supported by UAVs in urban areas. Finally, self-organization of a large fleet of UAVs that follows the specific mobility pattern of UEs was recently proposed in Horva´th et al. 17 The authors employ an evolutionary strategy based on the existence of virtual forces and show the superiority of the proposed approach compared to the conventional rule-based methods. Machine learning algorithms for UAV coverage mainly rely on the application of unsupervised clustering algorithms 9 and RL methods. A promising solution in this domain is the combination of both: deep neural nets (NNs) for large state space approximation and RL rules applied in the discrete-time stochastic process controlled by a Markov decision process (MDP). The advantage of a deep RL-based approach is that it captures the system evolution trajectory and does not require any tuning in the validation phase once the control policy is trained. Pioneering works addressing UAVs coverage optimization based on the application of deep RL were proposed by Wang et al. and Cui et al. 18, 19 The authors elaborated on the application of a conventional single-agent RL strategy represented by the stateaction-reward-state-action (SARSA) algorithm, 18 and the concept was also extended to the multiagent learning scenario, which is of more relevance in the presence of a large fleet of UAVs. 19 Furthermore, Khan and Yau 20 proposed the application of deep RL in determining the optimal UAV trajectory while addressing the limited energy resources of UAVs, and thus, optimization criteria were proposed for when the target was to increase the network lifetime of the UAV fleet. Finally, the work presented in Liu et al. 21 proposes a multiagent greedymodel-based RL approach that accounts for multiple UAVs with different parameters to explore environments in parallel to accelerate training. In our work, we aim to propose a solution based on a cGAN application that could act as a complementary mechanism to the algorithms given above. As long as there exists high uncertainty related to the UAV position in the fleet, the proposed algorithms can only act weakly, providing substantial performance gaps compared to the maximum theoretical expectations. For example, RL-based algorithms determine the positions of UAVs in fleet indirectly by proposing a specific reward function, which cannot generalize well in complex state spaces. Hence, our proposed approach intends to implement computation of quasi-optimal locations of UAVs in a fleet for a given snapshot of UEs positions in a very short time frame compared to the state-of-the-art algorithms (e.g. the core-set algorithm and spiral algorithm). While both algorithms are characterized by high time complexity that limits their application in the practice, we believe that the low time complexity of our proposed approach is of paramount importance. Thus, our proposed low-complexity approach (computation of the UAVs positions in the fleet) can co-exist and cooperate with the complex deep RL approach (computation of the trajectories of individual UAVs in the fleet) to jointly achieve truly highprecision and low-complexity solutions, targeting the theoretically achievable upper-bound performance of UAV-based communication systems. Note that we only address the former problem (computation of the UAVs positions in the fleet) throughout the article, and the combination of deep RL and our proposed approach is left for our future work. To perform coverage optimization, we split the investigated area into nonoverlapping segments of size n 3 n. Thus, we represent the scenario in matrix form, where each matrix element represents the number of UEs in a segment and their corresponding locations in the area. The same procedure is applied to the UAV-based aerial base stations. Segmentation of the investigated region allows application of powerful deep learning techniques and fast matrix processing to determine the quasioptimal coverage for UEs. Here, we assume that the UE is covered if it is positioned within the dedicated perimeter of the UAV. Our proposed approach is aimed at determining the minimum required number of UAVs and their corresponding positions to attain the quasioptimal coverage of the target area using a cGAN. Throughout the article, we use the following notations. The matrix X = (x ij ) 2 N n 3 n represents the distribution of UEs in the area, while x ij denotes the number of UEs in the (i, j) segment. Similarly, K = (k ij ) 2 N n 3 n and Y = (y ij ) 2 N n 3 n denote the UAVs placement in the segmented region computed using the analytical core-set strategy and heuristicbased cGAN, respectively. The proposed approach consists of two phases-the training phase of the cGAN and the deployment phase. The training phase encompasses the training of both architectures of the cGAN: generator G and discriminator D. The deployment phase includes the joint application of the generator G and a unique correction procedure avoiding blurring of the UAVs positions. The training is executed only once and offline; thus, it does not affect the time complexity of the proposed approach in practical deployments. In this section, we introduce the assumptions considered throughout the article, cGAN model training/characteristics and unique multilayer sum-pooling loss that extends the conventional cGAN loss definition. Definitions and assumptions. The cGAN 22 uses two deep neural networks simultaneously, that is, the generator (G) and discriminator (D). The training process of the cGAN proceeds by alternately updating the parameters of G and D. In conventional GAN models, 23 the generator's task is to generate realistic samples, and the discriminator's task is to distinguish synthetic (generated) samples from real samples. For our problem, we use a cGAN, which allows adding an external condition of the spatial UEs distribution X in the coverage area for both the generator and the discriminator. Thus, a target objective of the generator G is to compute the required minimum number of UAVs and their quasi-optimal locations Y for the given cGAN input. The role of the discriminator D is to decide whether the generated positions of UAVs are produced by the optimal core-set algorithm or by the heuristics-based generator. In this case, the optimal locations of UAVs represent positions such that all UEs are covered with the minimal number of UAVs. We compute the optimal solution K based on the Euclidean k-center problem employing the mixed breadth-depth traversing strategy. In general, the solution referred to as the core-set algorithm is based on the existence of a small subset of points called core-sets, which results in a polynomial time approximation scheme for a fixed number of UAVs. cGAN model. We represent the coverage optimization problem in matrix form. Formally, the generator G learns a mapping from input data X n 3 n to generate output data Y n 3 n , G : X ! Y. The real data samples (optimal UAVs positions) computed by the core-set algorithm are denoted by the template matrix K n 3 n . The objective function of the generator G aims to minimize the probability that the generated samples G(X) ! Y are classified by the discriminator as synthetic. Thus, the objective function is presented in the following form The discriminator D outputs a single scalar value, which represents the probability that Y is a sample produced by the generator G rather than a sample from the template matrix K. The objective function of the discriminator D is defined as Both the discriminator D and generator G are trained in parallel with the aim to minimize L cGAN (G) and at the same time maximize L cGAN (D). Isola et al. 24 further extended the training process with the L1 norm (Manhattan distance, i.e., mean absolute error (MAE)). The L1 norm is defined as the MAE between target K and generated data Y. Due to its nature, we can formulate the iterative optimization problem as a minimax problem over L cGAN (G, D) as follows arg min where l = 100 is the weighting coefficient. 24 Generator and discriminator. The generator G and discriminator D are implemented according to the scheme shown in Figure 1(a) , which depicts the training process for the proposed cGAN-based solution. For brevity, we refer to the detailed descriptions of the generator G and discriminator D in Isola et al. 24 In particular, generator G is implemented as the U-net with an input shape of 256 3 256, according to the initial segmentation of the target area. The shape of each successive layer is decreased twice in both rows and columns, with the eventual bottleneck of 2 3 2 in the seventh layer. The discriminator D is implemented as a conventional convolutional neural network, with the respective input shape of 256 3 256. Multilayer sum-pooling loss. Since UAVs occupy only a few segments in the investigated area, the target matrix Y is extremely sparse. Therefore, application of the conventional loss function for a cGAN such as the L1 norm, as shown in equation (3), results in weak performance of the proposed approach. To overcome this problem, we propose a custom multilayer sum-pooling loss function, which is computed according to Algorithm 1. First, the algorithm applies pooling with a filter of size 1 and the same padding, which corresponds to the conventional mean squared error (MSE) calculation between the target template K and the output of the generator Y. Note that the target template K is the sample taken from the extensive dataset consisting of both UEs and the corresponding optimal UAVs locations generated by the core-set algorithm. Then, by doubling the pooling size during each iteration, we provide important feedback to the generator about the spatial dislocation of the generated UAVs positions Y with respect to their optimal positions K. Finally, we formulate the optimization function of the cGAN with the proposed multilayer sum-pooling loss L pool as follows arg min The deployment phase is executed online to determine the UAVs positions in real time, respecting the UEs positions in the space. Here, we rely on the application of the reduced cGAN architecture, namely, its trained generator G (see Figure 1 (b)). By its nature, generator G generates blurred images, resulting in the presence of a larger number of UAVs in the fleet than necessary. To tackle this issue, a correction algorithm is developed and included in the deployment phase. Correction algorithm for coping with UAV blurring. The conventional application of a cGAN in the area of image processing does not require accurate image precision and often suffers from image blurring. This is not of major concern in the original image processing domain and could be suppressed by advanced image processing techniques. However, in our case, as a side effect, the blurriness of the output results in an increased number of UAVs slightly displaced from each other in the relatively sparse matrix Y. To tackle this challenge, we employed a correction mechanism to address this issue, which is designed as follows. The proposed correction algorithm takes the sparse matrix Y, and to accelerate the computations, it transforms it into a scalable coordinate storage scheme represented by the set Y , 25 solely encompassing the space coordinates ½x, y where UAVs are present. Note that the transformation f of the sparse matrix to the compressed set f : Y ! Y allows for faster computations than that with regular sparse matrices. In the first step, the algorithm randomly takes a UAV and its position ½x, y from Y . Then, it determines all other UAVs positioned in Y in the perimeter of size e and calculates the mutual distance. The selected UAV and neighboring UAVs within the perimeter e are withdrawn from Y . The group of the dedicated UAVs is replaced by the new particular UAV stored in the new set L with Algorithm 1. Multilayer sum-pooling loss 1: Inputs: Y n3n , K n3n 2: Initialize the loss matrix L pool = ; 3: Set the sum-pooling filter size to f = 1, stride = 1, and padding = 1 4: while f ł 64 do 5: Calculate the pooled version of the optimal position matrix with filter size f : P 1 = pool(K, f , padding) 6: Calculate the pooled version of the generated position matrix with filter size f : P 2 = pool(Y, f , padding) 7: Calculate the squared error for the pooled matrices Loss = (P 1 À P 2 ) 2 8: Add the loss to the total loss L pool = L pool + Loss 9: f = 23f , padding = 23padding 10: Output: L pool ½Dx, Dy coordinates computed as the ensemble average of all mutual distances calculated in this time step. The output of the algorithm is represented by the coordinate storage scheme L that contains the final number of UAVs along with their corresponding coordinates in the space. In our case, the hyperparameter e was tuned by a large set of computational experiments, with the aim of maximizing the cost function presented in equation (1). The detailed steps of the algorithm are summarized in Algorithm 2. Figure 2 graphically illustrates the issue of UAV blurring (red (thin) circles) and the correction mechanism presented in Algorithm 2 applied on top of the blurred UAVs positions (black bold circles). The simulation model is defined in the form of a geometric disk coverage problem. 26 The objective of the problem is to cover a set of UEs in a target area by the minimum number of UAVs with a given radius of coverage. The performance is evaluated from two viewpoints: coverage quality and computational efficiency. As the reference for the coverage quality, we use the core-set algorithm, which provides the optimal solution but is not computationally efficient. The computational efficiency is assessed versus the spiral algorithm, which is the current best practice among quasi-optimal solutions in terms of computational efficiency. 9 We focus our analysis on the complexity of the generator G and proposed routine to filter the number of UAVs at the output of generator G, thus leaving the complexity of the training procedure out of the investigation. Note that the training procedure is realized prior to the UAVs deployment (offline) and thus does not negatively affect the computational requirements of the proposed approach in practical realizations. Thus, in what follows, we use the term proposed approach to indicate the joint application of the generator function G and proposed correction algorithm (i.e. deployment phase). As the baseline scenario, we present the complexity of the core-set and spiral algorithms and the conventional K-means clustering algorithm. Let us denote the number of UAVs at the output of the generator G as y and number of filtered UAVs at the output of the correction mechanism as k. Based on massive numerical experiments, we can claim that in general, y ; 2:4k. As indicated by the pseudocode in Algorithm 2, the routine resembles the typical array sorting procedure with a complexity of O(y 2 ). As y ; 2:4k, we can approximate the complexity as O(k 2 ). Note that this is the worst-case scenario since the reduction of the sorting set generally leads to reduced number of executed inner steps. The time complexity of the analytical solution provided by the core-set algorithm is by definition O(p k ), where p is the number of UEs in the area. The time complexity of the spiral algorithm is O(k 3 ), which is significantly higher than that of the proposed approach. Among the conventional wellknown machine learning techniques, the K-means algorithm has a time complexity of O(p dk + 1 ), where d denotes the number of dimensions of the state space (in our case, d = 2). 27 While the time complexity of both the core-set and K-means algorithms depends on the number of UEs in the target area, for the spiral and proposed approaches, the time complexity is insensitive to the number of UEs. The time complexity of the spiral algorithm is cubic, while our approach runs quadratic in time as the worst-case scenario. As the downside of our approach, the training procedure must be executed prior to UAVs optimization deployment, which is not the case for the other investigated algorithms. A comparison of the time complexities of the given algorithms is shown in Table 1 . Note that for the experiments, we used a workstation equipped with an AMD Ryzen Threadripper 1950X with 64 RAM memory, and the simulations were executed on an NVIDIA GeForce GTX 1080Ti 11 GB GPU. The simulator routines were programmed in Python ver. 3.6, running on MS Windows 10 Pro. We define a coverage ratio metric R = D=r, where D is a side length of the target square coverage area and r is the radius of the coverage circle of a single UAV. We tested three coverage ratios R = 2, 4, 6 for two different numbers of UEs: 400 and 1000. The cGAN was trained only once for each R, and the same weights were transferred to predict UAVs positions for different number of UEs. Additionally, the correction parameter e was adjusted during simulations to remove the effect of UAV blurring. In Table 2 , we can see the averaged simulation results of the proposed algorithm in comparison with the spiral and core-set algorithms. The statistics were averaged over the dataset with a size of 1000 samples (snapshots of various instantaneous UEs positions). We can appreciate the time complexity reduction of the proposed approach across all investigated scenarios. In addition, the number of required UAVs for the proposed approach is in general closer to the optimal coreset solution than to the spiral algorithm solution. Graphical representations of the coverage solutions of the proposed approach, spiral algorithm, and core-set algorithm are shown in Figure 3 . As the downside of the proposed approach, we need to note that the coverage is not optimal and there are very few UEs that are not covered (also noted in Table 2 ). These are mostly located at the edges of the UEs clusters or are outliers not captured by the generator G. As long as our interest is to cover high-density clusters of UEs, we do not see this downside as a major problem. Obviously, these UEs can be easily covered by traditional ground infrastructure, that is, eNodeB network elements. Considering the growing demand for mobile applications and exponential growth in connected devices in smart cities, we can envision that the typical number of UEs in the foreseeable future will be an order of magnitude higher than those simulated in this article. Thus, we assume that the proposed approach with an asymptotic complexity of O(k 2 ) can be adopted for the deployment of aerial wireless communication systems, as well as for other UAV applications that are facing the disk coverage problem. Finally, considering that the cGAN can be trained over any type of data (i.e. more Figure 3 . Computational comparison of the proposed approach, core-set algorithm, and spiral algorithm, R = 6, 1000 UEs. Note that in addition to the fastest processing capabilities, the proposed approach requires in general less UAVs than the quasi-optimal spiral algorithm. complex datasets), the proposed algorithm can be easily extended to complex coverage optimization based on multiple wireless network performance criteria, such as achievable throughput and area spectral efficiency. Thus, the solution based on the application of a cGAN proposed in this article appears to be feasible, opening further promising research directions in this area. To improve the deployment of moving cells based on UAVs, we have proposed a new computationally efficient algorithm for coverage optimization. The proposed approach is based on a cGAN, with a unique multilayer sum-pooling loss function that allows convergence to the quasi-optimal coverage solution in the presence of high-density clusters of UEs. Simulations have been conducted to assess the performance of the proposed system versus the optimal solution provided by the core-set algorithm and the most computationally efficient quasi-optimal solution provided by the spiral algorithm. The results show that the proposed approach provides similar results to those provided by other state-of-the-art algorithms while being an order of magnitude better in terms of computational time. In future research, we will enhance the coverage optimization by using deep RL with the reward function determined by the cGAN algorithm based on multiple network performance indicators. The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. For the spiral and core-set algorithms, by definition, the coverage is 100%. In comparison, the proposed approach provides a tremendous time complexity improvement accompanied by only a slight performance degradation in terms of coverage (see also Figure 3 ). Note that the time complexity requirements for the spiral and core-set algorithms significantly increase with increasing number of UEs, while that of the proposed approach remains almost constant. The best achieved time for a given setting is highlighted in bold. Google mobility trends: how has the pandemic changed the movement of people around the world? Survey on UAV cellular communications: practical aspects, standardization advancements, regulation, and security challenges A tutorial on UAVs for wireless networks: applications, challenges, and open problems Survey of important issues in UAV communication networks On-demand density-aware UAV base station 3D placement for arbitrarily distributed users with guaranteed data rates Joint 3D UAV placement and resource allocation in software-defined cellular networks with wireless backhaul Approximate clustering via core-sets A mixed breadth-depth first strategy for the branch and bound tree of Euclidean k-center problems Placement optimization of UAV-mounted mobile base stations Generative adversarial networks enhanced location privacy in 5G networks Generative adversarial learning for machine learning empowered self organizing 5G networks Using improved conditional generative adversarial networks to detect social bots on Twitter Intrusion detection system using voting-based neural network Joint positioning of flying base stations and association of users: evolutionarybased approach Energy efficient positioning of flying base stations via Coulomb's law Coverage mission for UAVs using differential evolution and fast marching square methods Evolutionary coverage optimization for a self-organizing UAV-based wireless communication system Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing Adaptive UAV-trajectory optimization under quality of service constraints: a modelfree solution Route selection in 5G-based flying ad-hoc networks using reinforcement learning A greedy-modelbased reinforcement learning algorithm for Beyond-5G cooperative data collection Conditional generative adversarial nets Generative adversarial networks Image-to-image translation with conditional adversarial networks Sparse matrix computations with application to solve system of nonlinear equations Construction and maintenance of wireless mobile backbone networks Delaunay refinement algorithms for triangular mesh generation The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: