Map Aided Visual-Inertial Integrated Navigation for Long Range UAVs

Now a days, Unmanned Aerial Vehicles (UAVs) have been exploited in numerous civil and military aspects such as disaster management and border monitoring. Although most of these UAVs are equipped with Global Navigation Satellite System (GNSS) / Inertial Navigation System (INS) reliable integrated system for localization proposes, they are susceptible to lose their GNSS signals under some environmental and artificial challenging circumstances such as, flying in urban areas, jamming, and spoofing. In such complicated scenarios, the navigation solution will degrade rapidly due to the accumulated INS drift errors. Although various aiding sensors such as cameras, and radar are employed to mitigate the INS drift errors, their estimated positioning accuracy is still affected by some factors. This paper presents a nested optimal filter approach (G-H filter, Extended Kalman Filter (EKF)) for UAVs in GNSS denied environment, to enhance the estimated positioning accuracy by integrating the Radar Odometry (RO)/INS/MAG to sustain a reliable navigation solution during GNSS outage. First, the UAV relative velocity is calculated based on image processing approach from the radar range doppler map, then the G-H filter is deployed to improve the accuracy of the vehicle estimated forward velocities based on the vehicle dynamic characteristics to reduce the effect of unrealistic velocity estimate from the radar. These improved velocities are then fused with the Inertial Measurement Unit (IMU), magnetometer, and barometer via EKF to mitigate the positioning RMS errors during the GNSS signals outages. The experimental results demonstrate the proposed system's ability to notably enhance the 3D positioning accuracy during the GNSS signal outage by more than 99% when compared to a stand-alone IMU dead reckoning solution.

Nowadays, unmanned aerial vehicles (UAVs) play a vital role in many applications and have been successfully employed in a wide variety of applications such as road surveying, precision agriculture, landslide monitoring, cultural heritage mapping, and pipeline monitoring. All of these applications require accurate and stable navigation system. Currently, most of the commercially available UAVs rely on the integration of the Global Satellite Navigation Systems (GNSS) and Inertial Navigation Systems (INS) to estimate the position, velocity, and attitude. The small form factor of modern UAVs allowed them to operate in more challenging environments such as urban and natural canyons. In these environments, the GNSS availability cannot be guaranteed, and hence the navigation solution will deteriorate dramatically during these GNSS signal outages due to the drift exhibited by the inertial navigation solution. Using other aiding sensors is crucial to limit the accumulated errors associated with INS measurements. Onboard cameras can offer a useful clue to support the navigation solution during the GNSS signal outage periods. Varieties of monocular visual odometry based on photogrammetric and Structure from Motion (SfM) approaches have been proposed to assist the navigation estimation process. The main problem of using the monocular VO technique is the loss of scale if neither external measurement nor a priori knowledge about the surrounding environment are available. Moreover, the estimated camera pose from VO is prone to drift with time. This paper introduces a novel approach for estimating the navigation states of an Unmanned Aerial Vehicle (UAV) by integrating the visual information that obtained from a monocular camera with the Inertial Measurement Unit (IMU) observations via an Extended Kalman Filter (EKF). Most of the current monocular VO algorithms rely on a calibrated camera model and apply the conventional photogrammetric and SfM approaches. While these approaches can help towards estimating the relative rotation and translation by tracking image features and applying geometrical constraints, they cannot estimate the motion scale using only the image visual features.

Drones are becoming increasingly significant for vast applications, such as firefighting, and rescue. While flying in challenging environments, reliable Global Navigation Satellite System (GNSS) measurements cannot be guaranteed all the time, and the Inertial Navigation System (INS) navigation solution will deteriorate dramatically. Although different aiding sensors, such as cameras, are proposed to reduce the effect of these drift errors, the positioning accuracy by using these techniques is still affected by some challenges, such as the lack of the observed features, inconsistent matches, illumination, and environmental conditions. This paper presents an integrated navigation system for Unmanned Aerial Vehicles (UAVs) in GNSS denied environments based on a Radar Odometry (RO) and an enhanced Visual Odometry (VO) to handle such challenges since the radar is immune against these issues. The estimated forward velocities of a vehicle from both the RO and the enhanced VO are fused with the Inertial Measurement Unit (IMU), barometer, and magnetometer measurements via an Extended Kalman Filter (EKF) to enhance the navigation accuracy during GNSS signal outages. The RO and VO are integrated into one integrated system to help overcome their limitations, since the RO measurements are affected while flying over non-flat terrain. Therefore, the integration of the VO is important in such scenarios. The experimental results demonstrate the proposed system’s ability to significantly enhance the 3D positioning accuracy during the GNSS signal outage.

Global Navigation Satellite Systems (GNSS) has been the primary positioning solution for Unmanned Aerial Vehicles (UAVs) due to their worldwide coverage, high precision and lightweight receivers. However, GNSS is prone to electromagnetic interference and malicious assaults, including jamming or spoofing because of its low signal-to-noise ratio (SNR). To ensure the continuity and protection of UAV operations, using redundant navigation systems is essential. In recent years, the phased array radio system (PARS) has established itself as a local navigation solution. PARS is robust towards malicious assaults because of an much higher SNR than GNSS regarding directed and encrypted transmission. An essential factor of PARS is that the orientation of the radio antenna at a ground station needs to be precisely determined to obtain the correct positioning of UAVs. This paper presents a method for extending a previously proposed calibration algorithm to estimate the ground antenna orientation with an inertial navigation system (INS) aided by redundant positioning sensors (GNSS, PARS or barometer) using a multiplicative extended Kalman filter (MEKF) so that the calibration can be activated during flights whenever GNSS is available. In other words, the proposed navigation system is essentially an aided-INS which switches between two modes depending on the availability of GNSS: calibration and GNSS aiding mode when GNSS is available (Mode 1) and PARS and barometer aiding mode when GNSS is unavailable (Mode 2). Considering that the navigation system needs to include the effect of Earth’s curvature for a long-distance flight, PARS horizontal measurement and the barometer measurement were treated independently, and the navigation equations were propagated in Earth Centred Earth Fixed (ECEF) frame. The independent treatment of barometer measurement, and the propagation in ECEF frame were also beneficial when using multiple ground antennas to have a common reference point and reference frame. The proposed method was validated using data (Inertial Measurement Unit (IMU), GNSS, PARS, Pixhawk autopilot (including barometer) measurements) collected during a field test. In the validation, GNSS was made available at the middle of the flight and the calibration mode was activated for 200s. The proposed navigation system successfully estimated the precise orientation of multiple ground antennas and the navigation solutions were verified using GNSS and Pixhawk autopilot solutions as ground truth.

Precise timing and precise location information are provided by Global Navigation Satellite Systems (GNSS) and play a crucial role in the positioning, navigation and data acquisition of most Unmanned Aerial Vehicles (UAV). GNSS functions include the following applications in UAVs: time-stamping and geo-referencing of collected data and images, synchronization of swarm flying and follow-me flights, determination of position and attitude in-flight, flight trajectory by following a pre-defined number of waypoints, mission planning, return home automatically without external control, avoidance of obstacles and geo-fencing. 
 Some of these critical operations have implications for the safety of the UAV, the surrounding environment and health and safety of people, for example UAVs threatening to bring down aircrafts at airports, which are no-fly zones for UAVs. The appropriate GNSS based function to avoid this is geo-fencing. Another example is obstacle avoidance to prevent collisions and damages both for the UAV and the obstacle, e.g. anything from a window pane, tree, human being, to a power line.
 In order to ensure health and safety it is thus important to ensure correct function of the navigation and the timing, under a wide variety of circumstances, and in different signal environments. There can be signal disturbances, such as obscurations by buildings or reflected GNSS signals, called multipath. The performance of timing and navigation based on GPS/GNSS can be tested and verified in a controlled and repeatable way in the laboratory with different types of test equipment. We will give an introduction to a wide range of potential threats to GNSS Positioning, navigation and timing and an overview of different test methods. In addition, we are presenting a method for time synchronization of drones to enable safe swarm and follow flights in UAVs.

This paper describes an ultra-tight integration of a Global Navigation Satellite System (GNSS) receiver and an Inertial Navigation System (INS) for small Unmanned Aerial Vehicles (UAVs). The system is based on a low-cost and low-weight GNSS Intermediate Frequency (IF) sampler which has been fitted onto a small UAV. The storage of IF samples together with measurements from an Inertial Measurement Unit (IMU) has allowed the authors to process an Ultra-Tightly Coupled (U.T.C.) GNSS/INS solution from real data collected with a small UAV. The focus of this paper has been to investigate the potential benefits in signal tracking during flights below dense foliage and tree canopies by using the developed U.T.C. system. The obtained results have been compared to a scalar GNSS receiver and a high-grade commercial reference receiver and have showed significant improvements in tracking capabilities.

The object of research is the onboard subsystems of Unmanned Aerial Vehicles (UAVs). The research is aimed at analyzing UAVs, specifically the integration and enhancement of satellite-based positioning systems, including Global Navigation Satellite Systems (GNSS) and Inertial Navigation Systems (INS). The problem concerns traditional satellite-based positioning services, especially those relying solely on medium earth orbit (MEO) satellites, which are insufficient for specific requirements. The study aims to address the limitations of these systems on onboard subsystems of UAVs, especially in challenging environments laden with jammers and interference, and to provide a more accurate, robust, and continuous positioning solution. The research proposes a «multilayer system of systems» approach that integrates signals from various sources, including low Earth orbit (LEO) satellites, ground-based positioning, navigation, and timing (PNT) systems, and user-centric sensors. The combined approach, termed LeGNSS/INS, leverages the strengths of each component, providing redundancy and enhanced accuracy. The system's performance was evaluated using pseudo-real output data, demonstrating its ability to generate quasi-real dynamic trajectories for UAV flight. The error analysis showed that the proposed method consistently outperforms traditional GNSS systems, especially in challenging environments. The enhanced performance of the LeGNSS/INS system can be attributed to integrating multiple satellite systems with INS and applying optimal filtering techniques. The research also employed mathematical modeling to represent the dependencies and interactions when combining data from different sources, such as GPS, LEO, and INS. The Kalman filter is a mechanism to fuse data from multiple sources optimally. The insights from this study apply to various sectors, including aviation, maritime navigation, autonomous drones, and defense. The enhanced positioning accuracy can significantly improve safety, navigation precision, and operational efficiency. However, the study assumes idealized conditions for satellite signal reception, which might not always be accurate in real-world scenarios. Challenges, such as the martial law conditions in Ukraine affecting data collection and potential satellite signal restrictions, were also highlighted. Further research can delve into the impact of more complex environmental factors and the integration of additional satellite systems or sensors to enhance accuracy further.

During the past few decades, the presence of global navigation satellite systems (GNSSs) such as GPS, GLONASS, Beidou and Galileo has facilitated positioning, navigation and timing (PNT) for various outdoor applications. With the rapid increase in the number of orbiting satellites per GNSS, enhancements in the satellite-based augmentation systems (SBASs) such as EGNOS and WAAS, as well as commissioning new GNSS constellations, the PNT capabilities are maximized to reach new frontiers. Additionally, the recent developments in precise point positioning (PPP) and real time kinematic (RTK) algorithms have provided more feasibility to carrier-phase precision positioning solutions up to the third-dimensional localization. With the rapid growth of internet of things (IoT) applications, seamless navigation becomes very crucial for numerous PNT dependent applications especially in sensitive fields such as safety and industrial applications. Throughout the years, GNSSs have maintained sufficiently acceptable performance in PNT, in RTK and PPP applications however GNSS experienced major challenges in some complicated signal environments. In many scenarios, GNSS signal suffers deterioration due to multipath fading and attenuation in densely obscured environments that comprise stout obstructions. Recently, there has been a growing demand e.g. in the autonomous-things domain in adopting reliable systems that accurately estimate position, velocity and time (PVT) observables. Such demand in many applications also facilitates the retrieval of information about the six degrees of freedom (6-DOF - x, y, z, roll, pitch, and heading) movements of the target anchors. Numerous modern applications are regarded as beneficiaries of precise PNT solutions such as the unmanned aerial vehicles (UAV), the automatic guided vehicles (AGV) and the intelligent transportation system (ITS). Hence, multi-sensor fusion technology has become very vital in seamless navigation systems owing to its complementary capabilities to GNSSs. Fusion-based positioning in multi-sensor technology comprises the use of multiple sensors measurements for further refinement in addition to the primary GNSS, which results in high precision and less erroneous localization. Inertial navigation systems (INSs) and their inertial measurement units (IMUs) are the most commonly used technologies for augmenting GNSS in multi-sensor integrated systems. In this article, we survey the most recent literature on multi-sensor GNSS technology for seamless navigation. We provide an overall perspective for the advantages, the challenges and the recent developments of the fusion-based GNSS navigation realm as well as analyze the gap between scientific advances and commercial offerings. INS/GNSS and IMU/GNSS systems have proven to be very reliable in GNSS-denied environments where satellite signal degradation is at its peak, that is why both integrated systems are very abundant in the relevant literature. In addition, the light detection and ranging (LiDAR) systems are widely adopted in the literature for its capability to provide 6-DOF to mobile vehicles and autonomous robots. LiDARs are very accurate systems however they are not suitable for low-cost positioning due to the expensive initial costs. Moreover, several other techniques from the radio frequency (RF) spectrum are utilized as multi-sensor systems such as cellular networks, WiFi, ultra-wideband (UWB) and Bluetooth. The cellular-based systems are very suitable for outdoor navigation applications while WiFi-based, UWB-based and Bluetooth-based systems are efficient in indoor positioning systems (IPS). However, to achieve reliable PVT estimations in multi-sensor GNSS navigation, optimal algorithms should be developed to mitigate the estimation errors resulting from non-line-of-sight (NLOS) GNSS situations. Examples of the most commonly used algorithms for trilateration-based positioning are Kalman filters, weighted least square (WLS), particle filters (PF) and many other hybrid algorithms by mixing one or more algorithms together. In this paper, the reviewed articles under study and comparison are presented by highlighting their motivation, the methodology of implementation, the modelling utilized and the performed experiments. Then they are assessed with respect to the published results focusing on achieved accuracy, robustness and overall implementation cost-benefits as performance metrics. Our summarizing survey assesses the most promising, highly ranked and recent articles that comprise insights into the future of GNSS technology with multi-sensor fusion technique.

An opportunistic navigation framework using low Earth orbit (LEO) satellites is analyzed. This framework, termed simultaneous tracking and navigation (STAN), estimates a navigating vehicle’s state along with the states of orbiting LEO satellites. STAN employs an extended Kalman filter (EKF) to fuse measurements from satellite receivers and an inertial navigation system (INS). The navigation performance is analyzed due to: (i) the vehicle being equipped with (1) different inertial measurement unit (IMU) grades: consumer, industrial, and tactical and (2) different receiver clock quality: temperature-compensated crystal oscillators (TCXO) and oven-controlled crystal oscillators (OCXO) and (ii) the LEO satellites being equipped with different transmitter clock quality: OCXO and chip-scale atomic clock (CSAC). Additionally, the effect of utilizing a large number of LEO satellites for navigation is investigated. This analysis provides insight into the achievable performance of STAN, which can serve as an alternative navigation system in global navigation satellite system (GNSS)-denied environments. The performance predictions from simulations are compared with experimental results with real signals from the Orbcomm LEO constellation. A close match between the simulation and experimental results is demonstrated for an unmanned aerial vehicle (UAV) navigating via the STAN framework with signals from two Orbcomm LEO satellites for 160 seconds, the last 35 seconds of which are without GNSS signals. The UAV’s position root-mean squared error (RMSE) from simulations was 8.6 m, while the experimental position RMSE was 10 m.

In the near future, multi-sensor fusion will be the core component to navigate the autonomous driving platforms. This paper proposes new strategies to cope with the integration of an inertial navigation system (INS), a global navigation satellite system (GNSS), and light detection and ranging (LIDAR) to achieve simultaneous localization and mapping (INS/GNSS/LiDAR SLAM) especially in GNSS challenging environments where GNSS signals are blocked or contaminated with reflected signals. The proposed strategies implement a high level of integration with various information received from multiple sensors to collectively compensate for the specific drawbacks of those sensors included in the integrated system. The first strategy is to solve the divergence and drift problems of SLAM using the initial pose information from INS and the proposed refreshing process using an INS/GNSS integrated system. In addition, an updated mechanization is designed to qualify those received measurements based on cross validation of separate types of data. This mechanization is to ensure all measurements are reliable for the Extended Kalman Filter (EKF) update process. Moreover, the SLAM-derived information plays a major role to recognize the vehicle movement which assists the system to accurately apply those appropriate vehicle motion constraint models. The preliminary results presented in this study illustrate that proposed algorithm performs superior than the traditional INS/GNSS integration scheme and provides absolute navigation accuracy of 2 meters and 0.6% of distance traveled in GNSS-denied as well as 1.2 meters in GNSS-hostile environments, respectively.

Abstract. Multisensor systems are essential for autonomous navigation applications to achieve reliable accuracy. Integrating the Global Navigation Satellite System (GNSS) and the Inertial Navigation System (INS) is the most common integration scheme. However, this integration is unreliable in different scenarios since the GNSS signal may deteriorate in downtown areas or suffer from a blockage in underground and indoor areas. Therefore, other sensors are integrated with INS to compensate for GNSS outages. This paper proposes a novel algorithm for radar/INS tightly-coupled integration for autonomous navigation applications. This algorithm is applied in multiple steps. Radar data analysis is the first and most crucial step to remove the noisy data and the outliers and keep the useful objects. Then, data association is done to match the detected objects between radar frames. The tightly-coupled integration is performed at the measurement level through an Extended Kalman Filter (EKF), where the distance between the INS and the detected objects can be predicted from the INS and measured from the radar. Real data was collected from four Frequency Modulated Continuous Wave (FMCW) radar units in Calgary's suburban areas and Toronto's downtown area. The proposed algorithm was tested and assessed by introducing simulated GNSS single outages with different durations. The results show an enhancement in the vehicle's position by about 94% to 96%.

This paper exploits the idea of integrating the outputs of several optical flow sensors (OFSs) and a micro-electro-mechanical systems (MEMS) inertial measurement unit (IMU). Several optical flow sensors are fixed on an unmanned aerial vehicle (UAV). Their locations and orientations are different so as to detect the optical flow from different directions. By fusing the optical flow information and inertial navigation system (INS) measurements using an extended Kalman filter (EKF), the INS errors can be estimated and compensated. The integration of optical flow sensors and MEMS-based IMU provides a small size, low cost and self-contained scheme for UAVs navigation at low height above the ground in GPS denied environments. The preliminary simulation results presented in this article illustrate the effectiveness of the proposed integrated navigation scheme.

The possible applications and benefits of autonomous Unmanned Aerial Vehicle (UAV) use in urban areas are gaining considerable attention. Before these possibilities can be realized, it is essential that UAVs be able to navigate reliably and precisely in urban environments. The most common means of determining the location of a UAV is to utilize position measurements from Global Navigation Satellite Systems (GNSS). In urban environments, however, GNSS measurements are significantly degraded due to occlusions and multipath. This research analyzes the use of camera Line-of-Sight (LOS) measurements to self-describing fiducials as a replacement for conventional GNSS measurements. An extended Kalman filter (EKF) is developed and validated for the purpose of combining continuous measurements from an Inertial Measurement Unit (IMU) with the discrete LOS measurements to accurately estimate the states of a UAV. The sensitivity of the estimation error covariance to various system parameters is assessed, including IMU grade, fiducial placement, vehicle altitude, and image processing frequency.

Abstract. Typically, in situations where Global Navigation Satellite System (GNSS) signals are unavailable, navigation systems rely on integrating GNSS and inertial navigation system (INS) data. While such integration can provide accurate positioning during short GNSS signal outages, it cannot sustain prolonged GNSS outages. The reason for this is that the system's performance depends solely on the INS and can result in significant errors over time. To address this issue, additional onboard sensors are necessary. This study proposes a navigation system that integrates INS and LiDAR simultaneous localization and mapping (SLAM) using an extended Kalman filter (EKF). The system was tested using the raw KITTI dataset in various outdoor driving scenarios without GNSS signals. It is shown that the proposed system significantly outperformed the INS-only system, with an average RMSE improvement of around 93% and 58% in the horizontal and the up directions, respectively.

The traditional leader-follower Unmanned Aerial Vehicle (UAV) formation cooperative positioning (CP) algorithm, based on relative ranging, requires at least four leader UAV positions to be known accurately, using relative distance with leader UAVs to achieve the unknown position follower UAV’s high-precision positioning. When the number of the known position leader UAVs is limited, the traditional CP algorithm is not applicable. Aiming at the minimum cooperative unit, which consists of a known position leader UAV and an unknown position follower UAV, this paper proposes a CP method based on the follower UAV’s moving vector. Considering the follower UAV can only acquire the single distance with the leader UAV at each distance-sampling period, it is difficult to determine the follower UAV’s spatial location. The follower UAV’s moving vector is used to construct position observation of the follower UAV’s inertial navigation system (INS). High-precision positioning is achieved by combining the follower UAV’s moving vector. In the process of CP, the leader UAV obtains a high-precision position by an INS/Global Positioning System (GPS) loosely integrated navigation system and transmits its position information to the follower UAV. Based on accurate modeling of the follower UAV’s INS, the position, velocity and heading observation equation of the follower UAV’s INS are constructed. The improved extended Kalman filtering is designed to estimate the state vector to improve the follower UAV’s positioning accuracy. In addition, considering that the datalink system based on radio signals may be interfered with by the external environment, it is difficult for the follower UAV to obtain relative distance information from the leader UAV in real time. In this paper, the availability of the relative distance information is judged by a two-state Markov chain. Finally, a real flight test is conducted to validate the performance of the proposed algorithm.