Enhanced Dynamic Spectrum Access in UAV Wireless Networks for Post-Disaster Area Surveillance System: A Multi-Player Multi-Armed Bandit Approach

In order to fully utilize the scarce spectrum resources and improve the efficiency of wireless spectrum resource utilization, dynamic spectrum access is proposed as a solution. There are two kinds of dynamic spectrum access system, one is overlay dynamic spectrum, and the other is dynamic spectrum access under interference temperature constraints. In this paper, we use a three-dimensional Markov chain to model the dynamic spectrum access under interference temperature constraints, and analyze the performance of secondary users in the system. Two important parameters, i.e., secondary user interrupting probability and blocking probability are derived using the proposed model. The performance of secondary users in different dynamic spectrum access systems are analyzed and compared. The numerical results show that the performance of secondary users can be improved significantly in dynamic spectrum access under interference temperature constraints comparing to overlay dynamic spectrum access. Specially, the advantage of interference temperature constraints based dynamic spectrum access is more notable on the condition that there are not enough resources to provide to secondary users for access.

Multi-Armed Bandit (MAB) algorithms are classic algorithms that address sequential decision-making under uncertainty by solving the exploration-exploitation trade-off dilemma. This study investigates the performance comparison of multiple MAB algorithms using a simulated multi-armed bandit machine with Bernoulli reward distribution as the experimental environment. This study compares the performance differences among Upper Confidence Bound (UCB), Thompson Sampling (TS) and -Greedy Thompson Sampling (-TS) in this environment, and attempts to set different parameters, namely the number of arms and the number of experimental rounds and calculate the corresponding cumulative regret of the algorithm under various conditions. The size of the cumulative regret reflects the performance of each algorithm in the simulated slot machine model with rewards that conform to the Bernoulli distribution. In addition, the algorithm running time under the same conditions is also recorded to analyze from the perspective of algorithm efficiency. The experimental results show that under the experimental environment of this study, the cumulative regret produced by the UCB algorithm is more than three times that of the other two algorithms. When the number of trials is small, the cumulative regret generated by the TS algorithm is small, but overall, the performance of the TS algorithm and the -TS algorithm set in this experiment in minimizing the cumulative regret is not much different. However, TS runs in a shorter time under the same conditions. The results of this experiment show that after the number of rounds of experimental operation reaches a large enough number, the operating efficiency of the TS algorithm will be significantly higher than that of the -TS algorithm. TS algorithms have higher randomness, so they show better performance under this experimental condition. The -TS under the parameter setting of this study encourages exploration more in the early stage of the experiment, so it will produce greater cumulative regret than the traditional TS algorithm. In the long run, the performance difference between the two in the multi-armed bandit problem with Bernoulli distribution of rewards is very small. However, the TS algorithm has more advantages in algorithm execution efficiency, so when solving similar problems, the TS algorithm is a better choice.

Unmanned aerial vehicle (UAV) has greatly extended the scope of wireless broadband access by serving as a relay between the base station and user devices. In the paradigm of dynamic spectrum access (DSA), a UAV needs to traverse the flight area without interfering with incumbent users, and takes tune to serve the secondary users in the user area. It is important to minimize the service period by planning the UAV’s trajectory. In this paper, we study the trajectory planning problem of optimizing the flight route of the UAV, given the incumbent protection zone where the UAV cannot enter, for DSA. We divide the user area and the flight area into multiple 2D grids. Then, we formulate the problem as finding the optimal trajectory in the flight area to cover the user area. We propose an algorithm that first finds the trajectory in each divided 2D grid by generating the minimum dominating path and selecting necessary critical flight locations within the path, and then concatenates the obtained trajectories to the trajectory in the flight area. Experimental results show that the UAV takes 32% less time to finish trajectory under proposed algorithm, and as the user area and the flight area become larger, the gap among algorithms increases.

In this paper we study the problem of quantifying the value of spectrum opportunities to secondary users in Dynamic Spectrum Access (DSA) systems. We especially focus on estimating the impact of key channel parameters, namely the activity patterns of the primary users on the expected quality of experience for secondary users accessing video streams over a DSA system. In our study we consider three basic types of video representing typical video content categorized according to scene change rate ranging from low-activity newscast to high-activity sports video. Through extensive simulations we show that given some information on the expected level of activity in the video, the duty cycle of the primary user alone can yield good predictors for the expected quality of experience of secondary users. Knowledge of the precise distributions of the primary user ON and OFF periods can be used to further enhance the precision of the prediction, but at least for exponential and log-normal channel access patterns the differences are rather small. Finally, we study the problem of determining the channel statistics that are needed to apply the predictor in an optimization setting. Our simulations show that high accuracy can be achieved in matter of minutes of estimation time, which is more than enough for practical deployments in typical urban environments.

Unmanned aerial vehicles (UAVs) are expected to be integrated into future wireless networks to offer services, especially in unreachable or congested areas. To improve the spectral efficiency, non-orthogonal multiple access (NOMA) scheme can be utilised within the UAV communication to allow more users to be covered and associated. The performance of the NOMA-UAVs network is governed by several factors including power allocation, user association and pairing methods. This paper presents an approach that uses multi-armed bandit (MAB) and two-sided matching frameworks to maximize the throughput of multi-UAV-assisted NOMA networks in a decentralized manner. The approach enables the UAVs to propose to the ground users (GUs) without explicit cooperation among the UAVs while the GUs can accept or reject the proposals. To this end, we propose a modified Thompson sampling algorithm that we named decaying epsilon Thompson sampling (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text {D}\epsilon \text {TS}$ </tex-math></inline-formula>) MAB algorithm that is designed to improve the exploration-exploitation tradeoff in the MAB. The performance of the proposed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text {D}\epsilon \text {TS}$ </tex-math></inline-formula> MAB algorithm is evaluated against other existing MAB techniques. Simulation results show that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text {D}\epsilon \text {TS}$ </tex-math></inline-formula> algorithm attains faster convergence and improved performance in terms of smaller regret and increased achievable system throughput. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text {D}\epsilon \text {TS}$ </tex-math></inline-formula> MAB algorithm particularly excels in regards of the convergence rate when the number of available action spaces increases.

This paper considers a resource allocation problem where several Internet-of-Things (IoT) devices send data to a base station (BS) with or without the help of the reconfigurable intelligent surface (RIS) assisted cellular network. The objective is to maximize the sum rate of all IoT devices by finding the optimal RIS and spreading factor (SF) for each device. Since these IoT devices lack prior information of the RISs or the channel state information (CSI), a distributed resource allocation framework with low complexity and learning features is required to achieve this goal. Therefore, we model this problem as a two-stage multi-player multi-armed bandit (MPMAB) framework to learn the optimal RIS and SF sequentially. Then, we put forth an exploration and exploitation boosting (E2Boost) algorithm to solve this two-stage MPMAB problem by combining the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula> -greedy algorithm, Thompson sampling (TS) algorithm, and non-cooperation game method. We derive an upper regret bound for the proposed algorithm, i.e., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}(\log ^{1+\delta }_{2} T)$ </tex-math></inline-formula> , increasing logarithmically with the time horizon <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> . Numerical results show that the E2Boost algorithm has the best performance among the existing methods and exhibits a fast convergence rate. More importantly, the proposed algorithm is not sensitive to the number of combinations of the RISs and SFs thanks to the two-stage allocation mechanism, which can benefit the high-density networks.

Introduction/purpose: This paper provides an overview of Unmanned Aerial Vehicles (UAVs) from their early days to their integration into modern wireless networks. Methods: It analyzes, synthesizes and compares the UAVs development technologies throughout their history with significant aspects of their integration in wireless communication networks. Results: Important aspects of wireless communications as one of the key technologies for UAVs are presented. Next, energy efficiency as a research path for UAVs is considered. The paper also emphasizes the present state in this area as well as what the future of UAVs in communications will be. Conclusion: UAVs are important not only for widespread military usage in various combat operations and warfare environment, but also for other purposes such as their integration in 5G networks.

The application of unmanned aerial vehicles (UAV) as base station (BS) is gaining popularity. In this paper, we consider maximization of the overall data rate by intelligent deployment of UAV BS in the downlink of a cellular system. We investigate a reinforcement learning (RL)-aided approach to optimize the position of flying BSs mounted on board UAVs to support a macro BS (MBS). We propose an algorithm to avoid collision between multiple UAVs undergoing exploratory movements and to restrict UAV BSs movement within a predefined area. Q-learning technique is used to optimize UAV BS position, where the reward is equal to sum of user equipment (UE) data rates. We consider a framework where the UAV BSs carry out exploratory movements in the beginning and exploitary movements in later stages to maximize the overall data rate. Our results show that a cellular system with three UAV BSs and one MBS serving 72 UE reaches 69.2% of the best possible data rate, which is identified by brute force search. Finally, the RL algorithm is compared with a K-means algorithm to study the need of accurate UE locations. Our results show that the RL algorithm outperforms the K-means clustering algorithm when the measure of imperfection is higher. The proposed algorithm can be made use of by a practical MBS–UAV BSs–UEs system to provide protection to UAV BSs while maximizing data rate.

The escalating demand for wireless communication services has led to a critical shortage of available radio spectrum. Cognitive radio networks emerge as a promising solution to this challenge by enabling dynamic and intelligent spectrum access. This paper presents an in-depth exploration of cognitive radio networks, focusing on spectrum management and coexistence strategies. We delve into the fundamental concepts of cognitive radio networks, highlighting their ability to opportunistically exploit underutilized frequency bands while ensuring minimal interference with licensed primary users. The paper covers essential topics such as spectrum sensing techniques, dynamic spectrum access, spectrum mobility, coexistence protocols, and spectrum sharing models. Case studies illustrate practical implementations in wireless sensor networks, 5G systems, and Internet of Things (IoT) applications. Moreover, we address challenges, including security and privacy concerns, and discuss the integration of cognitive radio networks with emerging technologies like AI, machine learning, and blockchain. This comprehensive overview underscores cognitive radio networks' potential to revolutionize spectrum utilization, offering a path toward efficient and harmonious wireless communication.

&#x0D; &#x0D; &#x0D; &#x0D; &#x0D; &#x0D; The Ministry of Public Works and Public Housing has the task of carrying out government affairs in the field of public works and public housing to assist the President in administering the state government. One of the mandated functions in carrying out this task is to formulate and implement policies in the field of road administration in accordance with the provisions of laws and regulations, including the regulation of toll roads in Indonesia in accordance with Government Regulation No. 15 of 2005 concerning Toll Roads. The toll road is one of the vital means needed to increase efficiency in public services, logistics and industry which leads to economic efficiency. According to data released by the World Bank in 2019, losses due to traffic jams in Indonesia amounted to USD 4 billion, equivalent to an estimated IDR 56 trillion, more than that due to queuing at toll gates there is potential for losses for all business entities. To&#x0D; &#x0D; &#x0D; &#x0D; &#x0D; &#x0D; &#x0D; overcome this problem, Indonesia is preparing to implement a non-stop (touchless) toll payment system on the Multi Lane Free Flow (MLFF) system using RFID-based ETC (Electronic Toll Collection) Total Solution technology and the transaction process is carried out through a special application that can be accessed from the Internet. Integrate all departments with an IoT (Internet of thing) system. This payment system can eliminate the GTO function of toll entry and exit gates. In addition, the MLFF system using RFID-based ETC will recognize and identify vehicles and users. In addition to the RFID-based ETC system to increase the productivity and effectiveness of traffic management on toll roads, maintenance of toll roads and bridges is also very vital, therefore detecting road surface damage directly using sensors, CCTV cameras and UAV unmanned aerial vehicles. (Unmanned Aerial Vehicle) which has the advantages of high.

Owing to the rapid development of unmanned aerial vehicles (UAVs), UAV-enabled data collection has emerged as a promising technology. However, for the scenarios where the UAVs are dispatched to gather surrounding information actively and dynamically, the existing data collection methods can hardly fulfill the corresponding demands. To this end, we aim to study the paradigm data acquisition, where the UAV dynamically gathers information by on-board sensors. In this paper, we consider the scenario where a UAV acquires data in real time, which needs to be timely processed with the assistance of a mobile edge computing server. To address the real-time data acquisition characteristics, we construct a novel data acquisition rate model, which is with respect to the UAV speed. Next, we formulate a UAV energy consumption minimization problem that jointly considers UAV trajectory, transmission power, and CPU frequency. To tackle the highly complex problem, we propose an efficient iterative algorithm, and rigorously derive a closed-form solution for the UAV transmission power and computation resources allocation. With the obtained numerical results, we further validate the superiority of the proposed system design and the effectiveness of our algorithm against benchmark schemes.

Unmanned aerial vehicle (UAV) swarms are extensively used in emergency communications, area monitoring, and disaster relief. Their operations are coordinated by control centers, making them well-suited for federated learning (FL) frameworks. However, current UAV FL methods ignore the rich information contained in UAV images and the potential of using a single dataset to accomplish multiple tasks. For instance, in disaster relief scenarios, images acquired by UAVs can support tasks like crowd detection, road passability analysis, and disaster impact assessment. These tasks exhibit time-varying demands and may have potential correlations. To meet these requirements, this paper introduces two core mechanisms: a dynamic task attention mechanism to evaluate task importance for efficient resource allocation, and a task affinity (TA) metric to capture inter-task correlations for knowledge sharing. Building on these innovations, we propose FedDya, a novel UAV swarm-based one-dataset multi-task FL framework, where ground emergency vehicles (EVs) collaborate with UAVs to accomplish multiple tasks leveraging a single dataset. To optimize resource allocation, we formulate a two-layer optimization problem to jointly optimize UAV transmission power, computation frequency, bandwidth allocation, and UAV-EV associations. For the inner problem, we derive closed-form solutions for transmission power, computation frequency, and bandwidth allocation and apply the block coordinate descent method for optimization. For the outer problem, a novel two-stage algorithm is designed to determine optimal UAV-EV associations. Furthermore, theoretical analysis reveals a trade-off between UAV energy consumption violation and multi-task performance, characterized by an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i>( √ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</i> , 1/<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</i> ) relationship. Extensive simulation results further validate the effectiveness of the proposed scheme.

To cover remote areas where terrestrial cellular networks may not be available, non-terrestrial infrastructures such as satellites and unmanned aerial vehicles (UAVs) can be utilized in the upcoming sixth-generation (6G) era. Considering the spectrum scarcity problem, satellites and UAVs need to share the spectrum to save costs, leading to a cognitive satellite-UAV network. Due to the openness of both satellite links and UAV links, communication security has become a major concern in cognitive satellite-UAV networks. In this paper, we safeguard a cognitive satellite-UAV network from a physical layer security (PLS) perspective. Using only the slowly-varying large-scale channel state information (CSI), we jointly allocate the transmission power and subchannels to maximize the secrecy sum rate of UAV users. The optimization problem is a mixed integer nonlinear programming (MINLP) problem with coupling constraints. We propose a heuristic algorithm which relaxes the coupling constraints by the penalty method and obtains a sub-optimal low-complexity solution by utilizing random matrix theory, the max-min optimization tool, and the bipartite graph matching algorithm. The simulation results corroborate the superiority of our proposed scheme.

Unmanned aerial vehicles (UAVs)-based applications, such as surveillance systems and wireless relays, are attracting increasing attention from academia and industrial fields. The high-performance aerial communication system is one of the key enablers for them. However, due to the low attenuation of radio waves in the air-to-ground channels, the interference between aerial and terrestrial communication systems would significantly deteriorate their communication performance and greatly limit the potential UAV applications. To address the problem, in this paper, the spectrum sharing strategy between a multiple UAV communication system, in which both UAVs and ground station (GS) are equipped with directional antennas, and terrestrial systems is proposed. The GS position is selected and the flyable areas of the UAVs using certain spectrum resources are defined in advance using prior knowledge from spectrum monitoring on terrestrial communication systems to minimize interference and maximize the flyable areas of the UAVs instead of the low-efficient dynamic channel sensing and allocation for interference elimination. The simulations are conducted through a case study of the spectrum sharing between a multi-UAV video transmission system and the terrestrial wireless local area network (WLAN) system in the 5.7GHz band. The simulation results show that thanks to the proposed system the entire area can be enabled for UAV flight.

The improvement of energy efficiency and spectral efficiency in networks is achieved by seamlessly integrating energy harvesting, cognitive radio technologies, and Non-Orthogonal Multiple Access (NOMA) techniques. These complementary strategies optimize resource usage and address challenges related to energy consumption. Additionally, the adaptability and versatility of Unmanned Aerial Vehicles (UAVs) offer innovative solutions for enhancing coverage performance, thereby improving connectivity, efficiency, and reliability. We introduce a novel approach called the Deep Reinforcement Learning-Random Walrus (DRL-RW) algorithm, which combines Deep Reinforcement Learning (DRL) with the Random Walrus optimization (RWO) technique for efficient spectrum resource allocation and energy harvesting management in dynamic environments. The DRL-RW algorithm enables UAVs to learn optimal spectrum-sharing strategies and energy harvesting policies, while the RWO enhances the algorithm's adaptability and speed in exploring diverse solutions. Simulation results demonstrate the effectiveness of the DRL-RW algorithm, showing significant improvements in several performance metrics, including reduced energy consumption, enhanced computation time, improved convergence, increased signal-to-noise ratio, higher throughput, extended network lifetime, and increased harvested energy. These findings underscore the efficacy of the DRL-RW approach in addressing challenges associated with energy management in cognitive radio networks. The integration of UAVs, NOMA networks, and this novel algorithm represents a promising direction for developing energy-efficient communication systems.