Deep Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Control Using 3D Hand Gestures

Unmanned Aerial Vehicle (UAV) has become a preferred choice due to it robustness, reliability and efficiency, at an economically lower cost. The present study is intended to develop a UAV that is capable of navigating autonomously, using path planning algorithm, incorporating a collision avoidance approach at the lowest possible cost. This paper addresses the current issues of auto-piloting UAVs with collision avoidance mechanisms, that are commercially available, however, these systems are too large and expensive. The framework design of the UAV consists of a quadcopter frame, a flight controller, four motors, a Global Positioning System (GPS) system, an on-board telemetry system, a battery source and a raspberry pi camera processor board with a camera module. In the proposed UAV design, a preliminary software is designed to enable the UAV to navigate autonomously, on a pre-set designated flight path. The designed UAV provides a birds-eye view of its designated flight path to monitor the traffic flow.

3D Human Computer Interaction (HCI) becomes more and more popular thanks to the emergence of commercial depth cameras. Moreover, hand gestures provide a natural and attractive alternative to cumbersome interface devices for HCI. In this paper, we present an Image-to-Class Dynamic Time Warping (I2C-DTW) approach for 3D hand gesture recognition. Themain idea is that we divide the time-series curve of a 3D hand gesture into various finger combinations, called `fingerlets', which can either be learned or be set manually to represent each gesture and to capture inter-class variations. Furthermore, the I2C-DTW approach searches for the minimal path to warp two fingerlets, which are fromone test image and the specific class, respectively. Then the gesture recognition is to use the ensemble of multiple image-to-class DTW distance of fingerlets to obtain better performance. The proposed approach is evaluated on two 3D hand gesture datasets and the experiment results show that the proposed I2C-DTW approach significantly improves the recognizing performance.

Continuous trajectory control of fixed-wing unmanned aerial vehicles (UAVs) is complicated when considering hidden dynamics. Due to UAV multi degrees of freedom, tracking methodologies based on conventional control theory, such as Proportional-Integral-Derivative (PID) has limitations in response time and adjustment robustness, while a model based approach that calculates the force and torques based on UAV's current status is complicated and rigid. We present an actor-critic reinforcement learning framework that controls UAV trajectory through a set of desired waypoints. A deep neural network is constructed to learn the optimal tracking policy and reinforcement learning is developed to optimize the resulting tracking scheme. The experimental results show that our proposed approach can achieve 58.14% less position error, 21.77% less system power consumption and 9.23% faster attainment than the baseline. The actor network consists of only linear operations, hence Field Programmable Gate Arrays (FPGA) based hardware acceleration can easily be designed for energy efficient real-time control.

The increasing usage of Unmanned Aerial Vehicles (UAVs) in diverse applications necessitates the development of effective flight control systems. A major difficulty, nevertheless, is maintaining robust flight control under complex and dynamic environmental conditions, such as obstacle and non-flying zones, wind disturbances, and sensor noise. Traditional control techniques fail to achieve required flight control in these environments. To address this point, an adaptive control strategy is proposed based on a Deep Reinforcement Learning (DRL) model to enhance the flight performance of quadcopters. The learning and adaptation of the DRL-based control strategy are implemented in real-time through continuous interaction with the environment. This is to improve flight control and achieve consistent UAV performance under varying conditions. Proximal policy optimization with a reward function is used to minimize positional errors, ensure collision-free flight paths, and reduce energy consumption. The developed DRL model for quadcopters is trained in a simulated environment and then tested in three complex environmental scenarios, including urban, forest and mountain terrains. Experimental results demonstrate remarkable improvements in UAV flight performance. In the training phase, the reported training reward increased from 10 to 110 and the train loss is dropped from 0.85 to 0.05, which indicated successful model learning. Also, during system verification, rewards increased from 12 to 115 and UAV flight path deviations were decreased from 0.5 to 0.08m. The proposed controller outperforms conventional approaches in urban environments by lowering average trajectory deviations to 0.2m from 0.35m for MPC and 0.6m for PID. Also, the developed DRL-based controller outperformed the PID and MPC controllers, with path deviations of 0.18m in mountains and 0.12m in forests. In addition, fewer collision rates with obstacles are achieved with the model, 3% in forest, 1.8% in urban areas, and 4.5% in mountains. Furthermore, the consumed energy is reduced to 950J as compared to 1200J for PID and 1050J for MPC. The results show the strength of deploying the proposed controller in following the intended UAV flight path with high precision, effectively avoiding detected conflicts and minimizing consumed energy by the UAV.

In this paper, we develop an alternative navigation system for Unmanned Aerial Vehicle (UAV) in Global Positioning Systems (GPS)-denied environment. We use two image inputs, one is acquired with an on-board camera placed on the UAV (which is the large-area image) and the other is from satellite images (which is small known image) with GPS information. We use a convolutional neural network (CNN) architecture based on Oxford’s Visual Geometry Group network (VGG-16) and utilize normalized variant mutual information between these two images to obtain position of the UAV. Satellite images are labelled and given to the UAV. When GPS information is lost, our algorithm starts to function and images from UAV camera are searched whether satellite image is seen by cameras on UAV image or not. If the UAV is in that area, our algorithm finds the GPS information from satellite image data.

Unmanned aerial vehicles (UAVs) are envisioned to play a key role in intelligent transportation systems to complement the communication infrastructure in future smart cities. UAV-assisted vehicular networking research typically adopts throughput and latency as the main performance metrics. These conventional metrics, however, are not adequate to reflect the freshness of the information, an attribute that has been recently identified as a critical requirement to enable services such as autonomous driving and accident prevention. In this paper, we consider a UAV-assisted single-hop vehicular network, wherein sensors (e.g., LiDARs and cameras) on vehicles generate time sensitive data streams, and UAVs are used to collect and process this data while maintaining a minimum age of information (AoI). We aim to jointly optimize the trajectories of UAVs and find scheduling policies to keep the information fresh under minimum throughput constraints. The formulated optimization problem is shown to be mixed integer non-linear program (MINLP) and generally hard to be solved. Motivated by the success of machine learning (ML) techniques particularly deep learning in solving complex problems with low complexity, we reformulate the trajectories and scheduling policies problem as a Markov decision process (MDP) where the system state space considers the vehicular network dynamics. Then, we develop deep reinforcement learning (DRL) to learn the vehicular environment and its dynamics in order to handle UAVs' trajectory and scheduling policy. In particular, we leverage Deep Deterministic Policy Gradient (DDPG) for learning the trajectories of the deployed UAVs to efficiently minimize the Expected Weighted Sum AoI (EWSA). Simulations results demonstrate the effectiveness of the proposed design and show the deployed UAVs adapt their velocities during the data collection mission in order to minimize the AoI.

In this paper, one of the advanced technology in the field of human-computer interaction is known as the virtual hand. At the first, we have to know the grasping behavior. The virtual hand simulation can give experience grasping behavior and natural interaction. Many of HCI researcher said that one day the interaction between human and computer will be more natural, easy to use and less error. We only focus on 3D hand gesture. We use the engaging virtual application that supports 3D hand gesture in order to test whether the combination of grasping will provide the desired effect or not. We present the result from usability evaluation experiments. And finally, we want to shows that the 3D hand gesture can be grasping naturally look like the human hand.

Unmanned aerial vehicles (UAVs) are an emerging technology in the 5G-and-beyond systems with the promise of assisting cellular communications and supporting IoT deployment in remote and density areas. Safe and secure navigation is essential for UAV remote and autonomous deployment. Indeed, the opensource simulator can use commercial software-defined radio tools to generate fake global positioning system (GPS) signals and spoof the UAV GPS receiver to calculate wrong locations, deviating from the planned trajectory. Fortunately, the existing mobile positioning system can provide additional navigation for cellular-connected UAVs and verify the UAV GPS locations for spoofing detection, but it needs at least three base stations (BSs) at the same time. In this article, we propose a novel deep-ensemble-learning-based, mobile-network-assisted UAV monitoring and tracking system for cellular-connected UAV spoofing detection. The proposed method uses path losses between BSs and UAVs communication to indicate the UAV trajectory deviation caused by GPS spoofing. To increase the detection accuracy, three statistics methods are adopted to remove environmental impacts on path losses. In addition, deep ensemble learning methods are deployed on the edge cloud servers and use the multilayer perceptron (MLP) neural networks to analyze path losses statistical features for making a final decision, which has no additional requirements and energy consumption on UAVs. The experimental results show the effectiveness of our method in detecting GPS spoofing, achieving above 97% accuracy rate under two BSs, while it can still achieve at least 83% accuracy under only one BS.

Unmanned Aerial Vehicles (UAV) have become more popular for usage due to the low cost of deployment and maintenance. Single UAV employment allows remote area monitoring and transferring different payloads to inaccessible or dangerous zones for human. In order to deal with flight tasks that are more complex, UAV swarms are applied. The main challenge of UAV swarm formation and flight control is to avoid vehicle collisions. In this case, artificial intelligence is responsible for flight performance in the airspace in such way that collision is avoided. The main requirements to the method, which will provide conflict-free maneuvers, are safety (collision avoidance), liveness (decentralized control, destination area reachability) and flyability (UAV flight performance constraints are satisfied). Artificial force field method fulfills all of these demands. It allows to detect a potential conflict between multiple UAVs in a swarm and other static or moving obstacles found in airspace, to provide collision resolution by changing UAVs flight parameters through maintaining minimum separation distance, including cases when manned vehicles are found in the same airspace. There can be distinguished by a wide range of obstacles: static (buildings, restricted areas and bed weather conditions) and dynamic ones (other UAVs, manned aircraft). Method allows keeping UAV swarm shape on the flight path, taking into account ground speed and turn bank angle values restrictions according to UAV’s flight performance characteristics.

In this paper, we propose a systematic framework for the autonomous navigation system based on distributed filtering for an Unmanned Aerial Vehicle (UAV). The proposed framework consists of the design and algorithm of the autonomous navigation. Therein, the camera mounted on the UAV functions as a navigation sensor targeted for navigation and positioning. In order to reduce the computational complexity and exclude the risk caused by Global Positioning System (GPS) outage, an autonomous navigation system based on distributed filtering is designed and realized. When GPS is available by monitoring the GPS integrity, sensor information from Strapdown Inertial Navigation System (SINS) and GPS is fused using a 7-state Conventional Kalman Filter (CKF) to estimate the full UAV state (position, velocity and attitude); when GPS is unavailable, sensor information from gyroscopes, accelerometers and magnetometer is fused using a 4-state Extended Kalman Filter (EKF) to estimate the attitude and heading of the UAV, and sensor information from SINS and vision positioning system is fused using a 7-state Incoordinate Interval Kalman Filter (IIKF) to estimate the position and velocity of the UAV. In addition, the second-order vertical channel damping loop is adopted to fuse measurements from a barometer with those of SINS, which suppresses the divergence of the vertical channel error and makes the altitude information calculated by SINS trustable. Both ground and flight experiments of the autonomous navigation system have been carried out. The test results show that the system can provide stabilized attitude information in long durations, and can realize the automatic flight control of UAV.

Unmanned Aerial Vehicles (UAVs) play a crucial role in navigation, requiring accurate sensors to determine position, speed, and orientation, especially in unknown environments. Direct navigation systems like the Global Positioning System (GPS) provide positional data, while indirect systems, such as the Inertial Measurement Unit (IMU), integrate accelerometer and gyroscope data to supply speed and orientation information. This study investigates the integration of GPS and IMU sensors using the Unscented Kalman Filter (UKF) to improve localization accuracy on a cost-effective UAV Quadrotor equipped with LiDAR Livox. The research methodology involved collecting raw data from GPS, IMU, and LiDAR sensors during UAV flights. These data were processed using a UKF-based mathematical model to fuse sensor inputs and generate accurate point cloud models. Results show that the UKF fusion method achieved a localization accuracy of 0.403 m, with maximum residuals recorded as 1.332 m for the X axis, 20.421 m for the Y axis

Unmanned Aerial Vehicles (UAVs), or drones, have recently become a favorable solution for fast parcel delivery due to their maneuverability and advances in navigation technologies. With the limitation of battery capacity and payload of drones, it is crucial to consider both efficiency and cost while conducting the tasks. Meanwhile, UAVs should not collide with each other while traveling to customers. In this article, we propose a UAV parcel delivery system involving deep reinforcement learning (DRL) approach for collision avoidance and a genetic algorithm for route optimization. Specifically, a delivery center generates near-optimal routes, loading UAV with parcels according to demands. Each UAV takes charge of delivering packages in compliance with the assigned route while avoiding collision with each other. We utilize DRL to achieve collision avoidance without having prior knowledge about the trajectories of other UAVs. Additionally, we adopt a genetic algorithm to obtain the lowest energy cost path for each UAV. To find such an optimized path, we solve a capacitated vehicle routing problem (CVRP) with a modified cost function and extra constraints. Realistic simulations using a physics engine and software-in-the-loop (SITL) are conducted to evaluate the feasibility of the proposed methods.

With the rapid development of aviation technology, especially in intelligent situation awareness, autonomous flight control and electric propulsion, the number of unmanned aerial vehicles (UAVs) has grown explosively, bringing challenges to low-altitude flight safety. Autonomous UAV detect and avoid (DAA) technology is a key future direction. Firstly, this paper focuses on UAV multi-sensor data fusion, proposing an asynchronous trajectory fusion method via comprehensive weighting that accounts for multiple influencing factors. Secondly, for trajectory tracking of ownship and intruders, improved Kalman filter (KF) and enhanced Sage-Husa adaptive unscented Kalman filter (AUKF) algorithms are used for vertical and horizontal tracking to enhance accuracy. Then, a hybrid method combining detect and avoid alerting logic for unmanned aircraft systems (DAIDALUS) with Markov decision process (MDP) is proposed to solve the UAV collision avoidance decision-making problem. It calculates collision avoidance strategies via the state space, takes them as MDP’s action space, sets up reward functions and state transition probabilities to build an MDP model, and explores the influence of discount factors. Finally, through experimental tests, results show the proposed UAV detection and avoidance decision method has a 27.2% efficiency increase compared to standalone DAIDALUS, the loss of well clear (LoWC) rate is only 5.8%, and the nearest distance between the intruder and the ownship is 343 m, meeting the collision avoidance requirements for UAV flight.

The operations of unmanned aerial vehicles (UAVs) are susceptible to cybersecurity risks, mainly because of their firm reliance on the Global Positioning System (GPS) and radio frequency (RF) sensors. GPS and RF sensors are vulnerable to potential threats, such as spoofing attacks that can cause the UAVs to behave erratically. Since these threats are widespread and potent, it is imperative to develop effective intrusion detection systems. In this paper, we propose an image-based intrusion detection system for detecting GPS spoofing cyberattacks based on a deep learning methodology. We combine convolutional neural networks with Principal Component Analysis (PCA) to reduce the dimensionality of the dataset features, data augmentation to increase the size and diversity of the training dataset, and transfer learning to improve the proposed model’s performance with limited data to design a fast, accurate, and general method. Extensive numerical experiments demonstrate the effectiveness of the proposed solution carried out using benchmark datasets. We achieved an accuracy of 100% within a running time of 120.64 s at 0.3529 ms latency and a detection time of 2.035 s in the case of the training dataset. Further, using this trained model, we achieved an accuracy of 99.25% within a detection time of 2.721 s on an unseen dataset that was unrelated to the one used for training the model. In contrast, other models, such as Inception-v3, showed lower accuracy on unseen datasets. However, Inception-v3 performance improved significantly after Bayesian optimization, with the Tree-structured Parzen Estimator reaching 99.06% accuracy. Our results demonstrate that the proposed image-based intrusion detection method outperforms the existing solutions while providing a general model for detecting cyberattacks included in unseen datasets.

The autonomous UAV (unmanned aerial vehicle) navigation has recently gained an increasing interest from both academic and industrial sectors due to its potential uses in various fields and especially, the need for social distancing during the pandemic. Many works have adopted a deep reinforcement learning (RL) method with experience replay called deep deterministic policy gradient (DDPG) to control the motion of UAV, and gain high accuracy results in static and simplified environments. However, they are still far from being ready for real world adoption in that the UAVs have to operate under complex and dynamic conditions. We also found that using only DDPG makes the learning process prone to oscillation and is inefficient for tasks having high dimensional action-state spaces. Furthermore, the goal reward mechanism in traditional reward functions brings a bias to the state, which resembles the one at the goal area and leads to erroneous action selection. To get closer to being ready for real world adoption, we proposed a novel method that enables UAVs to be capable of handling motion control in realistic environments. The first component of our proposed method is point cloud data (PCD) simplification with truncated icosahedron structure which converts enormous PCD into a few essential data points. In the second component of our method, we replace the traditional goal reward mechanism with a new mechanism called Augmentative Backward Reward (ABR) function to dispense the goal reward to transitions proportionately to its participation. By integrating simplified PCD and ABR, we achieved significantly better results when compared with using only the-state-of-the-art, TD3. In addition, we tested the proposed method with another navigation task, BipedalWalkerHardcore, a testbed for RL, and the result is still better and steadier than of TD3. These results indicate that the proposed method is robust.