Reinforcement Learning Aided UAV Base Station Location Optimization for Rate Maximization

Unmanned aerial vehicle (UAV) base stations play a significant role in many scenarios like disaster relief and terrestrial traffic offloading. To put UAV base stations into operation, deployment is a key issue to be addressed. Existing works do not take into account different user Quality-of-Service (QoS) requirements in the deployment of multiple unmanned aerial vehicles (UAVs). Therefore, in this paper, we propose a 3D deployment scheme which minimizes the number of UAVs to cover all users with different QoS requirements. First, we derive the relationship between the altitude and coverage of a UAV at different user QoS requirements. Then, we formulate the deployment problem on the basis of the relationship. Next, we propose a 3D UAV deployment algorithm which considers both altitude and horizontal location. Finally, numerical results show that the proposed algorithm can deploy less UAVs to achieve full coverage of all users.

Ground-based communication facilities are at risk of being destroyed after natural disasters, such as earthquakes, floods, tsunamis, hurricanes, fires, or terrorist attacks. Unmanned aerial vehicles (UAV) can be used as air base stations to support user equipment (UE). UAV base station (UAV-BS) development plays a major part in rescue operations and post-disaster reconstruction. Improving network throughput in the UAV-BS signal coverage area and reducing deployment costs while maintaining effective communication are important issues that need to be addressed. In view of the problem, this paper demonstrates the problem of maximizing network throughput under the constraint of UAV-BS capacity by deploying UAV-BS. We proposed a UAV-artificial bee colony (U-ABC) algorithm to solve the problem of UAV-BS deployment. U-ABC algorithm can calculate the optimal flight position of each UAV-BS and maximum network throughput in the disaster area. In performance evaluation, we compared U-ABC algorithm with genetic algorithm, Greedy-ABC algorithm, PSO algorithm, DI-PSO algorithm and PSO-GWO algorithm. We analyzed the flight height altitude of UAV-BSs, the interference factor of UAV-BS, and the influence of the number of UE on network throughput. Results show that the proposed method improved the overall network throughput and achieved a high UE coverage rate under a given number of UAV-BSs.

Unmanned aerial vehicles (UAVs) are being integrated as an active element in 5G and beyond networks. Because of their flexibility and mobility, UAV base stations (UAV-BSs) can be deployed according to the ground user distributions and their quality-of-service (QoS) requirement. Although there has been quite some prior research on the UAV deployment, no work has studied this problem in a 3-D setting and taken into account the UAV-BS capacity limit and the QoS requirements of ground users. Therefore, in this article, we focus on the problem of deploying UAV-BSs to provide satisfactory wireless communication services, with the aim to maximize the total number of covered user equipment subject to user data-rate requirements and UAV-BSs' capacity limit. First, we model the relationship between the air-to-ground path loss (PL) and the location of UAV-BSs in both horizontal and vertical dimensions, which has not been considered in previous works. Unlike the conventional UAV deployment problem formulation, the 3-D deployment problem is decoupled into a 2-D horizontal placement and altitude determination connected by PL requirement and minimization. Then, we propose a novel genetic algorithm-based 2-D placement approach in which UAV-BSs are placed to have maximum coverage of the users with consideration of data rate distribution. Finally, numerical and simulation results show that the proposed approach has enabled a better coverage percentage comparing with other schemes.

The unmanned aerial vehicles (UAVs) have been widely considered as one of the key applications for future wireless communication systems, where UAVs can be used as aerial base stations (BSs) for coverage extension, transmission improvement, emergency communication, and etc. Against this background, each UAV BS is expected to select the optimal codeword to form directional analog beams, and it is capable of achieving concurrent transmissions from multiple other UAV BSs simultaneously. However, in such a kind of UAV networks, due to the vast number of connected mobile users (MUEs), UAV BSs cannot timely and preciously select the codeword from the pre-defined codebook. Fortunately, machine learning (ML) is suitable for decreasing complexity in codeword selection, because ML could extract features from the data samples acquired in real environments. In this paper, we propose an ML approach to achieve an efficient and low complexity codeword selection for UAV networks. Specifically, we first derive the probabilities that multiple UAV BSs serve one MUE to obtain the average sum rate (ASR) in UAV networks. On that basis, we develop an ML approach to maximize the ASR, where we design a classifier based on support vector machine (SVM), where our ML approach is used for selecting the optimal codeword and maximizing the ASR in UAV networks. Third, we proposed an iterative sequential minimal optimization (SMO) training algorithm to train the data of all links between UAV BSs and MUEs, where the algorithm convergence is also discussed. Finally, we show the comparison between our proposed algorithm and the traditional methods by the simulation results. The simulation at last demonstrate our method is a more efficient solution for obtain a higher performance, where a much lower computational complexity can achieved than the traditional algorithm based on channel estimation.

We propose a novel formation control method considering the challenges of formation keeping, unmanned aerial vehicle (UAV) control, and position adjustment in UAV Base Station (UAV-BS) formation control scenarios. The method is based on a loose coupling structure inspired by the traditional virtual structure method. Firstly, a type of new formation is designed for UAV-BS group. When the UAV-BS group keep a stable network, the centers of all UAV-BS moving trajectory, e.g., circular, are treated as virtual points to form a rigid body. When the UAV-BS group transforms between different formations, the real positions of the UAV-BS group are used to form a rigid body. Secondly, we redesign a trajectory tracking method for UAV-BS to control the flight direction. A flight direction controller is designed for UAV-BS, based on nonlinear guidance logic. According to the empirical model of error feedback, an error corrector is designed to control the position error between the UAV-BS and the virtual point. Simulation results show that the proposed algorithm can satisfactorily complete the formation and control of UA-BS. Besides, the algorithm can reduce the difficulty of formation control of UAV-BS and the requirements of formation control for computing power and network communication.

Unmanned aerial vehicle (UAV) with flexible mobility and low cost has been a promising technology for wireless communication. Thus, it can be used for wireless data collection in Internet of Things (IoT). In this article, we consider millimeter-wave (mmWave) communication on a UAV platform, where the UAV base station (UAV-BS) serves multiple ground users, which generate big sensor data. Both the deployment of the UAV-BS and the beamforming design have essential impact on the throughput of the system. Thus, we formulate a problem to maximize the achievable sum rate of all the users, subject to a minimum rate constraint for each user, a position constraint of the UAV-BS, and a constant-modulus (CM) constraint for the beamforming vector. We solve the nonconvex problem with two steps. First, by introducing the approximate beam pattern, we solve the deployment and beam gain allocation subproblem. Then, we utilize the artificial bee colony (ABC) algorithm to solve the beamforming subproblem. For the global optimization problem, we find the near-optimal position of the UAV-BS and the beamforming vector to steer toward each user, subject to an analog beamforming structure. The simulation results demonstrate that the proposed solution can achieve a more superior performance than the present random steering beamforming strategy in terms of achievable sum rate.

To address the ever-growing connectivity demand in communications, the adoption of ingenious solutions, such as utilization of unmanned aerial vehicles (UAVs) as mobile base stations, is imperative. In general, the location of a UAV base station (UAV-BS) is determined by optimization algorithms, which have high computationally complexities and are hard to run on UAVs due to energy consumption and time constraints. In this article, we overview the UAV-BS trajectory optimization problem for next generation wireless networks and show that a convolutional neural network (CNN) model can be trained to infer the location of a UAV-BS in real time. To this end, we create a framework to determine the UAV-BS locations considering the deployment of mobile users (MUs) to generate labels by using the data obtained from an optimization algorithm. Performance evaluations reveal that once the CNN model is trained with the given labels and locations of MUs, the proposed approach is, indeed, capable of approximating the results given by the adopted optimization algorithm with high fidelity, outperforming reinforcement learning-based approaches in resource-constrained settings. We also explore future research challenges and highlight key issues.

With the rapid development of unmanned aerial vehicles assisted applications enabled with a communication system, they are open to various malicious attacks. As a new form of flying things, they can access the network for better communication via the aerial base station. Most of the Unmanned aerial vehicles assisted flying objects' optimal path selection schemes does not consider the path deviation. In path deviation attacks, secure data transmission are not addressed in existing works. The secure communication process between Unmanned aerial vehicles and base station are exploited through security-based attacks. Moreover, path loss issue leads to multicast packet loss and unsecured broadcast. The existing network architecture setup does not fulfill the secure data communication and privacy issues. In this paper, Blockchain is utilized to investigate the secure communication between Unmanned aerial vehicles to Wireless Unmanned aerial vehicles Base stations. Since the destination information is dynamic under an uncertain environment, it will cause a delay in data communication. Unmanned aerial vehicles are more vulnerable to security attacks. The proposed blockchain-based architecture supports secure data communication in Unmanned aerial vehicles uncertain environments. To improve network security, this paper designs a modified particle swarm optimization method for better path selection. Through these experimental results, a blockchain-based data communication scheme is outer performed concerning network security.

This paper proposes a method for the trajectory planning of an unmanned aerial vehicle (UAV) acting as an aerial base station of a communication network. The UAV base station provides wireless communication for moving ground vehicles with different quality-of-service (QoS) requirements. The ground vehicles follow locally planned trajectories, which are communicated to the aerial base station. The UAV possesses a network estimator which determines the channel properties between the UAV and each ground vehicle. Whenever the QoS deceeds a threshold the UAV trajectory is changed based on this estimate by a trajectory planning unit in a way to fulfil all individual QoS requirements of the ground vehicles if possible. In contrast to literature, this approach considers the communication relay between moving vehicles with different QoS requirements. The performance of the method is demonstrated in a simulation study.

The unmanned aerial vehicle (UAV) base station plays a significant role in enhancing the terrestrial network, when the ground base station (GBS) is destroyed in emergent cases or its load exceeds the capacity of the terrestrial network. Presently, many papers focus on optimizing the UAV position deployment and user access, while ignoring the optimization about the spectrum resource management. To solve this problem, we formulate a joint optimization problem of the spectrum resource management and the position placement for UAVs with the constraint of the limited backhaul capacity. Later, the joint optimization problem is modeled as a hierarchical game decision architecture comprised of a UAV position placement game and a spectrum resource management game. Further, we analyze the equilibrium property of the two games and propose two best response- (BR) based optimization algorithms to reach the Nash Equilibriums (NEs) of the two games, respectively. Specifically, the proposed algorithm about the UAV deployment considers the variable granularity local exploration and global random exploration. Simulation results show that the proposed UAV deployment algorithm can improve the total throughput by 7% and 20% at least in comparison with the K-means deployment algorithm and the fixed granularity exploration algorithm, respectively.

In this paper, we study the resource allocation and trajectory design for secure unmanned aerial vehicle (UAV)-enabled communication systems, where multiple multi-purpose UAV base stations are dispatched to provide secure communications to multiple legitimate ground users (GUs) in the existence of multiple eavesdroppers (Eves). Specifically, by leveraging orthogonal frequency division multiple access (OFDMA), active UAV base stations can communicate to their desired ground users via the assigned subcarriers while idle UAV base stations can serve as jammer simultaneously for communication security provisioning. To achieve fairness in secure communication, we maximize the average minimum secrecy rate per user by jointly optimizing the communication/jamming subcarrier allocation policy and the trajectory of UAVs, while taking into account the constraints on the minimum safety distance among multiple UAVs, the maximum cruising speed, the initial/final locations, and the existence of cylindrical no-fly zones (NFZs). The design is formulated as a mixed integer non-convex optimization problem which is generally intractable. Subsequently, a computationally-efficient iterative algorithm is proposed to obtain a suboptimal solution. Simulation results illustrate that the performance of the proposed iterative algorithm can significantly improve the average minimum secrecy rate compared to various baseline schemes.

Unmanned aerial vehicle (UAV)-assisted wireless communication is considered to be an important technique in 6G. However, the joint optimization of trajectory and beamforming for UAV is a challenging problem. In this paper, we merge an improved reinforcement learning scheme in a UAV-enabled MIMO communication. Our goal is to maximize the sum of received signal energy within a finite time horizon through trajectory design and efficient beamforming of the UAV base station (UAV-BS). Firstly, we model the communication process as a Markov decision process (MDP). Secondly, we propose a general framework of Q-learning in combination with multi-armed bandit (MAB) to solve the problem. On one hand, the Q-learning algorithm is used to optimize the UAV trajectory by learning optimal actions and recording the learned strategies. On the other hand, the MAB is used to find the beamforming vector at the UAV. Simulation results are provided to show that the proposed MAB-Qlearning scheme can effectively optimize the UAV wireless transmission scheme with a mobile ground user.

The rapid growth of data traffic due to the demands of new services and applications poses new challenges to the wireless network. Unmanned aerial vehicles (UAVs) can be a solution to support wireless networks during congestion, especially in scenarios where the region has high traffic peaks due to the temporal and spatial flow of users. In this paper, an intelligent machine-learning-based system is proposed to deploy UAV base stations (UAV-BS) to temporarily support the mobile network in regions suffering from the congestion effect caused by the high density of users. The system includes two main steps, the load prediction algorithm (LPA) and the UAV-BSs clustering and positioning algorithm (UCPA). In LPA, the load history generated by the mobile network is used to predict which macrocells are congested. In UCPA, planning is performed to calculate the number of UAV BSs needed based on two strategies: naïve and optimized, in addition to calculating the optimal positioning for each device requested to support the overloaded macrocells. For prediction, we used two models, generalized regression neural networks (GRNN) and random forest, and the results showed that both models were able to make accurate predictions, and the random forest model was better with an accuracy of over 85%. The results showed that the intelligent system significantly reduced the overhead of the affected macrocells, improved the quality of service (QoS), and reduced the probability of blocking users, as well as defined the preventive scheduling for the UAV BSs, which benefited the scheduling and energy efficiency.

The unmanned aerial vehicles base stations (UAV-BSs) have great potential in being widely used in many dynamic application scenarios. In those scenarios, the movements of served user equipments (UEs) are inevitable, so the UAV-BSs needs to be re-positioned dynamically for providing seamless services. In this paper, we propose a system framework consisting of UEs clustering, UAV-BS placement, UEs trajectories prediction, and UAV-BS reposition matching scheme, to serve the UEs seamlessly as well as minimize the energy cost of UAV-BSs' reposition trajectories. An Echo State Network (ESN) based algorithm for predicting the future trajectories of UEs and a Kuhn-Munkres-based algorithm for finding the energy-efficient reposition trajectories of UAV-BSs is designed, respectively. We conduct a simulation using a real open dataset for performance validation. The simulation results indicate that the proposed framework achieves high prediction accuracy and provides the energy-efficient matching scheme.

The use of unmanned aerial vehicle (UAV)- based communication in millimeter-wave (mmWave) frequencies to provide on-demand radio access is a promising approach to improve capacity and coverage in beyond-5G (B5G) systems. There are several design aspects to be addressed when optimizing for the deployment of such UAV base stations. As traffic demand of mobile users varies across time and space, dynamic algorithms that correspondingly adjust the UAV locations are essential to maximize performance. In addition to careful tracking of spatio-temporal user/traffic activity, such optimization needs to account for realistic backhaul constraints. In this work, we first review the latest 3GPP activities behind integrated access and backhaul system design, support for UAV base stations, and mmWave radio relaying functionality. We then compare static and mobile UAV-based communication options under practical assumptions on the mmWave system layout, mobility and clusterization of users, antenna array geometry, and dynamic backhauling. We demonstrate that leveraging the UAV mobility to serve moving users may improve the overall system performance even in the presence of backhaul capacity limitations.