Intelligent joint trajectory design and resource allocation in UAV-based data harvesting system

With the ongoing advancement of electric power Internet of Things (IoT), traditional power inspection methods face challenges such as low efficiency and high risk. Unmanned aerial vehicles (UAVs) have emerged as a more efficient solution for inspecting power facilities due to their high maneuverability, excellent line-of-sight communication capabilities, and strong adaptability. However, UAVs typically grapple with limited computational power and energy resources, which constrain their effectiveness in handling computationally intensive and latency-sensitive inspection tasks. In response to this issue, we propose a UAV task offloading strategy based on deep reinforcement learning (DRL), which is designed for power inspection scenarios consisting of mobile edge computing (MEC) servers and multiple UAVs. Firstly, we propose an innovative UAV-Edge server collaborative computing architecture to fully exploit the mobility of UAVs and the high-performance computing capabilities of MEC servers. Secondly, we established a computational model concerning energy consumption and task processing latency in the UAV power inspection system, enhancing our understanding of the trade-offs involved in UAV offloading strategies. Finally, we formalize the task offloading problem as a multi-objective optimization issue and simultaneously model it as a Markov Decision Process (MDP). Subsequently, we proposed a task offloading algorithm based on a Deep Deterministic Policy Gradient (OTDDPG) to obtain the optimal task offloading strategy for UAVs. The simulation results demonstrated that this approach outperforms baseline methods with significant improvements in task processing latency and energy consumption.

Unmanned aerial vehicle (UAV) has been widely deployed in efficient data collection for Internet of Things (IoT). UAV can not only act as a relay, but also as an energy source to provide information and energy transmission for ground sensor nodes (SNs). This paper studies the efficient multi-UAV-assisted data collection problem in wireless powered IoT. Specifically multiple UAVs wirelessly charge SNs using radio frequency (RF) energy transfer, and the SNs then use the harvested energy to upload the updates of the sensed information to the UAVs, thus improving the freshness of collected data and extending the service time of the SNs. The problem is modeled as a partially observed Markov decision process (POMDP) with a large observation and action space, where each UAV acts as an intelligent agent to learn the environment and make decisions independently. The value-decomposition network (VDN) algorithm is employed to find the optimal strategy in the multi-agent deep reinforcement learning framework. Simulation results validate the effectiveness of the proposed data collection approach compared to two baseline policies.

Wireless sensor networks (WSN) play a key role in enabling wireless communication technology among several nodes to remotely communicate and exchange information. WSN consists of tiny sensor nodes equipped with battery, scattered in an area to gather information around an environment and send to data collection node known as sink or base station (BS). WSN have been widely used in various applications like agriculture, fire detection, health care and military and has become imperative necessity for future revolutionary area like UAV (unmanned aerial vehicles), IoT (Internet of things) and smart cities which employs large scale sensor nodes. However sensor nodes are limited to battery, memory, low computational power, resource and bandwidth. Continues sensing of events, makes node to drain its battery faster and goes dead fast. For resource constrained WSN, hierarchical cluster based approaches are considered as energy efficient and improves network performance for large scale WSN. Minimizing energy consumption and extending network lifetime are major challenging issues of WSN, clustering methods with optimized routing have offered solution to optimize energy utilization. To balance energy consumption and improve network lifetime many existing hierarchical clustering approaches have been proposed, however existing method does not consider rotation of cluster head (CH) and considers cluster head selection based on residual energy and distance parameter. In this paper we propose an improved energy efficient cluster tree (IEECT) based routing to improve energy efficiency of hierarchical cluster. IEECT considers modification of existing LEACH (Low energy adaptive clustering hierarchy) protocol to improved energy efficient LEACH by considering energy parameters like residual node energy and average network energy. IEECT accounts optimal number of cluster head (CH) and selection of CH is done using threshold value among sensor nodes. Proposed IEECT combines tree based routing and data aggregation scheme to maintain desirable quality of service. Simulation experiments are carried out by using network simulator. Performance of IEECT is evaluated in terms of PDR, delay, energy consumption, network lifetime and overhead.

Unmanned aerial vehicle (UAV)-enabled communication has emerged as an appealing technology for efficient data collection in wireless sensor networks (WSNs). This paper considers a scenario where multiple UAVs collect data from a group of sensor nodes (SNs) on the ground. We study the fundamental tradeoff between the aerial cost , which is defined by the propulsion energy consumption and operation costs of all UAVs, and the ground cost , which is defined as the energy consumption of all SNs. To characterize such a tradeoff, an optimization problem is formulated to minimize the weighted sum of the above two costs, by optimizing the UAV trajectory jointly with wake-up time allocation, as well as the transmit power of all SNs. As the formulated problem is non-convex, it is difficult to be optimally solved in general. To tackle this issue, we decouple it into two sub-problems: UAV trajectory and wake-up time allocation optimization, as well as SN transmit power optimization. We propose an iterative algorithm to solve the two sub-problems by leveraging successive convex approximation and alternating optimization techniques. In addition, a new approach is proposed to design the UAV initial trajectory with multiple travelling salesman problem (MTSP) technique. Simulations are conducted to corroborate our study and show the flexible tradeoff achieved by the proposed design for cost balance between UAVs and SNs.

The flexibility and maneuverability of unmanned aerial vehicles (UAVs) lend themselves to tracking users and operating as an aerial base station carrying out communication enhancement functionality. A core challenge neglected by most existing works is that the true-but-unknown obstacles can jeopardize UAV flight security and shadow its communication links with users, resulting in poor achievable rate and high collision risks. In this paper, a deep-reinforcement-learning (DRL)-based UAV target tracking and digital-twin (DT)-assisted collision avoidance method is proposed to optimize UAV’s communication performance while tracking moving users. Toward this end, Twin Delayed Deep Deterministic policy gradient (TD3) as a novel and policy-based DRL algorithm is used to construct an agent responsible for adaptive deciding UAV flying control actions. To efficiently detect unknown obstacles in a flight environment, an orthogonal frequency division multiple (OFDM)-based integrated sensing and communication (ISAC) system is investigated, endowing UAV’s agent with real-time obstacle distance. Finally, we present a DT obstacle model construction mechanism and integrate it with TD3 agent training. The extensive simulations demonstrate the reward convergence of the TD3 algorithm and the communication improvement with stable user tracking and reliable collision avoidance, compared with conventional approaches.

Wireless sensor networks (WSNs) are an important means of collecting data in a variety of situations, such as the monitoring of large or hazardous areas. The retrieval of WSN data can yield better results through the use of unmanned aerial vehicles (UAVs), for example, concerning the increase in the amount of data collected and the decrease in the time between the collection and use of the data. In particular, disaster areas may be left without communication resources and with high residual risk to humans, at which point a WSN can be quickly launched by air to collect relevant data until other measures can be established. The set of rules of each problem’s component (e.g., number of UAVs, UAVs dislocation control, sensors, communication) is considered the approaches to solve the problem. In this meaning, some studies present approaches for the use of UAVs for the collection of WSN data, focusing primarily on optimizing the path to be covered by a single UAV and relying on long-range communication that is always available; these studies do not explore the possibility of using several UAVs or the limitations on the range of communication. This work describes DADCA, a distributed scalable approach capable of coordinating groups of UAVs in WSN data collection with restricted communication range and without the use of optimization techniques. The results reveal that the amount of data collected by DADCA is similar or superior to path optimization approaches by up to 1%. In our proposed approach, the delay in receiving sensor messages is up to 46% shorter than in other approaches, and the required processing onboard UAVs can reach less than 75% of those using optimization-based algorithms. The results indicate that the DADCA can match and even surpass other presented approaches, since the path optimization is not a focus, while also incorporating the advantages of a distributed approach.

The unmanned aerial vehicle (UAV) is widely used in some scenes with high requirements for information freshness. Due to the limited endurance of the UAV, especially in the scenes with large area and dense sensor nodes (SNs), it is difficult for one UAV to complete the data collection task under the condition of ensuring the freshness of SNs’ information. Therefore, multiple UAVs are required to cooperate to participate in data collection. In this paper, we study the multi-UAV assisted data collection problem to improve information freshness. We use the Age of Information (AoI) to measure the freshness of information, mainly including the SNs’ uploading time, the UAVs’ flight time and the data offloading time. The data collection problem is formulated to minimize the SNs’ peak AoI and average AoI in multi-UAV assisted wireless sensor networks. Since the problem is complex, we introduce a start-to-end strategy comprising of association and planning to minimize two SNs’ AoIs through an iterative three-step process. Firstly, the locations of data collection points (CPs) at which the UAVs hover to collect data and the SN-CP association are determined based on a density-based clustering algorithm. Secondly, the CPs are clustered to form CP clusters, and the CP-UAV association is established. Finally, based on the results of the above two steps, the flight trajectories of the UAVs are optimized by improved ant colony (ACO) algorithm subject to the limited endurance capability. The simulation results show the proposed strategy can optimize the peak-AoI and ave-AoI of SNs to improve the freshness of information.

A new consensus theory-based method for formation control and obstacle avoidance of UAVs

UAV-assisted task offloading for IoT in smart buildings and environment via deep reinforcement learning

Unmanned aerial vehicles (UAVs) have emerged as a promising platform for supporting integrated sensing, computing, and communication (ISCC) functionality in Internet of Things (IoT) applications. This paper investigates a UAV-enabled ISCC network, where the UAV performs radar sensing and onboard edge computing with the computational assistance of ground access points (APs). Given the limited onboard energy, ensuring energy-efficient operation of UAVs is crucial to support the long-term sustainability of network performance. In this paper, we define computing efficiency as the ratio between the total number of successfully processed computational bits and the overall UAV energy consumption, under the constraint of a required sensing threshold. To maximize this performance metric, this paper jointly optimizes the beamforming vector, the CPU frequency, and the trajectory of the UAV. This optimization problem is modeled as a Markov decision process (MDP) and solved using a deep reinforcement learning (DRL) approach based on a memory mechanism. Specifically, a long short-term memory (LSTM) and twin delayed deep deterministic policy gradient (TD3)-based trajectory design and resource allocation (LTTDRA) algorithm is proposed. LSTM units are integrated into the actor and critic to effectively capture the temporal correlations in dynamic environments, thereby enhancing policy stability and accelerating algorithm convergence. The reward function is meticulously designed to alleviate sparse-penalty effects and learn high-performance strategies in complex environments with multiple constraints. Extensive simulations are conducted under various settings and network scenarios, and the results consistently indicate that the proposed approach substantially outperforms the baseline schemes.

As the development of unmanned aerial vehicle (UAV) communication in the residential and commercial areas, different from the traditional wireless sensor networks (WSNs), in this paper we consider a two-layer wireless networks for data gathering, one is sensor nodes to UAVs layer, the other one is UAVs to the outside layer. The sensor nodes sense data and transmit to UAVs, and then UAVs send data to the outside. In this paper, we only focus on the first layer, and we suppose the capacity is sufficient for UAVs to transmit their gathered data in the second layer. The sensor nodes aim to maximize the total data transmitting rate. Because of the limited resources (i.e. limited bandwidth, limited energy of each node, and so on) in the system, thus how to allocate the limited resources to maximize the total data transmitting rate is challenging, and it is necessary to propose an efficient algorithm to allocate the resources. We formulate the problem that jointly considers bandwidth allocation and energy allocation to maximize the total transmitting rate while guarantees the rate in each time slot for each sensor node. Then an optimal algorithm based on dynamic programming named DPBA is proposed. Finally, we conduct the extensive simulation to compare our proposed algorithm and the benchmark algorithm (i.e. equal resource allocation algorithm, denoted by ERAA). Simulation results show that our proposed algorithm outperforms the benchmark algorithm.

The proliferation of unmanned aerial vehicles (UAVs) has driven the emergence of urban air mobility (UAM) as a solution to alleviate the increasing demand for UAVs applications in various domains. The concept of UAM envisions a future where UAVs play a pivotal role in short-distance transportation, delivery services, surveillance, emergency rescue, and more. However, the rapid growth of UAVs poses significant challenges arising from the need to ensure the safe coexistence of UAVs with other manned/unmanned aerial vehicles. To ensure the safe integration of UAVs into low-altitude urban airspace and support high-density on-demand mobility, this research focuses on handling the safety-centric challenges related to UAV collision risk assessment, obstacle avoidance, and conflict resolution in complex urban environments.
\nOne of the main challenges is the intrusion of UAVs into airport runways. Airports serve as critical hubs for civil aircraft takeoff and landing operations. However, the increasing presence of UAVs has raised concerns about intruding into airports, leading to flight delays and temporary runway closures. To quantify the potential interference of UAVs on civil aircraft, a collision-course based trajectory planning model is proposed to evaluate the collision risk between manned civil aircraft and intruding UAVs. In this risk assessment model, the trajectory of the intruding UAV is formulated using a collision-course concept, taking into account the limited trajectory information available about intruding UAVs. The planned trajectory of the UAVs is then used to predict its relative position concerning civil aircraft. To account for the possible disturbances of the relative distance, a stochastic position prediction model based on Brownian motion is developed. The collision risk is assessed based on this stochastic kinematic model, providing a quantitative measure of the potential collisions. The proposed model is validated through extensive simulations involving different UAV initial positions, position updates, and different sizes of collision zone tests, as well as comparative tests with the Monte-Carlo algorithm, demonstrating its robustness in assessing the collision risk of the airport restricted area. To facilitate the application, the model is finally used to develop a platform to generate runway heatmaps, accounting for airport runway risk distributions. 
\nAnother safety-centric challenge is ensuring autonomous obstacle avoidance and conflict resolution of UAVs. To enable UAV autonomous safe operations in uncontrolled urban low-altitude airspace, a comprehensive collision-free path planning framework is developed to address the avoidance of both static obstacles and dynamic threats. Firstly, to handle static obstacles, a novel algorithm of 3D voxel jump point research is introduced to generate a global reference path for UAV navigation to avoid static obstacles, particularly for a free flight at different altitudes. Additionally, to optimize the reference path for specific mission requirements, techniques such as trajectory de-diagonalization, reconstruction, and smoothness are proposed to reduce the risk of static collisions, shorten the fly range, and enable UAVs to negotiate turns more efficiently. Secondly, real-time conflict resolution actions based on the Markov decision process are developed to avoid local dynamic threats encountered along the optimized desired path. This allows the host UAV to dynamically adjust its course to a resolution trajectory while maintaining its planned speed without hovering to avoid the potential collision with the approaching intruders. By avoiding near mid-air collisions (NMAC) and preventing the UAV from hovering, the proposed method ensures the timely completion of UAV missions. Simulation results demonstrate that the developed method successfully achieves UAV autonomous global path planning and local real-time conflict resolution in complex urban low-altitude environments, thereby avoiding both static obstacles and dynamic threats. 
\nMoreover, to bridge the research gap of UAV hard conflict resolution in mitigating NMAC risks arising from local dynamic threats, a novel situational NMAC risk assessment approach is developed by integrating the conditional random field (CRF) algorithm with safety metrics and conflict resolution performance to facilitate UAV tiered aerial passage operations. To improve UAV operating environments in urban airspace, a concept of tiered aerial passages is first designed to provide a practical urban airspace planning scheme that addresses UAV heterogeneity and optimizes the utilization of urban airspace. For UAV tiered aerial passage operations, a situational NMAC risk assessment module coupled with a conflict resolution decision-making module is formulated, enabling the host UAV to evaluate and respond to real-time conflicts, ensuring avoidance of potential collisions. To assess NMAC risks, a general risk assessment model is proposed based on the CRF algorithm, considering UAV safety metrics and resolution performance. Then the overtaking, approaching head-on, and converging UAV encounters are analyzed, and situational risk potential scores are created using the Gaussian kernel function in the CRF algorithm, considering advisory, caution, and warning risk alert levels. To resolve conflict, acceptable, tolerable, and unacceptable scenarios are proposed to enhance the flexibility and personalization of the resolution decision-making. The proposed scheme is validated by simulating geometric encounter situations and comparative tests with the previous hard resolution technique. The results indicate that the proposed method outperforms UAV NMAC risk assessment and conflict resolution. Even in the most dangerous unacceptable resolution decision-making scenarios across three conflict encounters, the proposed algorithm achieves NMAC rates of 0.036, 0.070, and 0.043, which are lower than the corresponding NMAC rates associated with the hard resolution approach. Together with the acceptable and tolerable scenarios, the proposed negotiation resolution effectively tailors different risk requirements to accommodate diverse UAV missions. 
\nWith a primary focus on ensuring the safety of the UAM architecture, this research endeavors to assess UAV NMAC risks and efficiently manage potential conflicts. The proposed work are thoroughly validated to demonstrate their effectiveness in real-world scenarios. As a result, this study significantly contributes to the establishment of a robust UAM architecture, facilitating the realization of safe and reliable UAM.

The rapid development of the agricultural Internet of Things (IoT) is inseparable from the support of wireless sensor networks (WSNs) in recent years. To further facilitate the adaptation of WSNs to agricultural applications, reducing the energy consumption of sensor nodes in agricultural environments has become a crucial problem. Sleep scheduling in randomly deployed WSNs is an effective method to reduce the energy consumption of sensor nodes, which can extend the network lifetime while ensuring network coverage. However, most of the existing sleep scheduling algorithms require frequent information exchange (such as broadcasting to find neighboring nodes and collecting nodes’ residual energy), which will inevitably lead to massive energy consumption. To address this problem, in this article, an unmanned aerial vehicle (UAV)-assisted sleep scheduling algorithm (UAVSS) is proposed, which avoids excessive information exchange among nodes and ensures sufficient network coverage with the least number of sensor nodes. The UAV traverses all the sensor nodes along the shortest path to help information exchange and gather sensing data. Furthermore, the UAV path is planned by using the latest monarch butterfly optimization (MBO) algorithm. Simulation results indicate that the proposed UAVSS can prolong the network lifetime while ensuring the required area coverage.

A wireless sensor network (WSN) is usually deployed in a field of interest (FoI) for detecting or monitoring some special events and then forwarding the aggregated data to the designated data center through sink nodes or gateways. Traditionally, the WSN requires the intensive deployment in which the extra sensor nodes are deployed to achieve the required coverage level. Fortunately, depending on the developments of the unmanned aerial vehicle (UAV) techniques, the UAV has been widely adopted in both military and civilian applications. Comparing with the traditional mobile sensor nodes, the UAV has much faster moving speed, longer deployment range and relatively longer serving time. Consequently, the UAV can be considered as a perfect carrier for the existing sensing equipment and used to form a UAV-based WSN (UWSN). In this paper, we theoretically analyse the coverage problem in the UWSN. Based on the integral geometry, we solve the aforementioned question. The experimental results further verifies our theoretical results.

Unmanned Aerial Vehicles (UAVs) can collaborate to collect and relay data for ground sensors in remote and hostile areas. In multi-UAV-assisted wireless sensor networks (MA-WSN), the UAVs' movements impact on channel condition and can fail data transmission, this situation along with newly arrived data give rise to buffer overflows at the ground sensors. Thus, scheduling data transmission is of utmost importance in MA-WSN to reduce data packet losses resulting from buffer overflows and channel fading. In this paper, we investigate the optimal ground sensor selection at the UAVs to minimize data packet losses. The optimization problem is formulated as a multi-agent Markov decision process, where network states consist of battery levels and data buffer lengths of the ground sensor, channel conditions, and waypoints of the UAV along the trajectory. In practice, an MA-WSN contains a large number of network states, while the up-to-date knowledge of the network states and other UAVs' sensor selection decisions is not available at each agent. We propose a Multi-UAV Deep Reinforcement Learning based Scheduling Algorithm (MUAIS) to minimize the data packet loss, where the UAVs learn the underlying patterns of the data and energy arrivals at all the ground sensors. Numerical results show that the proposed MUAIS achieves at least 46 % and 35% lower packet loss than an optimal solution with single-UAV and an existing non-learning greedy algorithm, respectively.