Feature Detection and Description based on ORB Algorithm for FPGA-based Image Processing

In view of the fact that ORB (the Oriented Fast and Rotated Brief) algorithm does not have scale invariance, an improved ORB algorithm, called SURB, was proposed combining the idea in Speed-up Robust Feature (SURF). Firstly, the Hessian matrix was used to detect feature points, which made the extracted feature points have scale invariance. Secondly, the ORB is used to describe feature points, which generated binary-descriptors of radiation invariant. Thirdly, the Hamming matrix is used for rough matching. Lastly, the method of RANSAC is used for purification and the average weighted algorithm is used for image fusion. When the image scale change, the average number of matching successful points of the SURB algorithm is three times of the ORB algorithm. However, when the image scale does not change, the average stitching time of ORB algorithm is same as the SURB algorithm and the average time-consuming of SURB algorithm from the feature detection to matching is six times of the SURF algorithm. The experimental results show that the SURB algorithm not only has the scale invariance, but also improves the matching efficiency, and the SURB can be applied to the video stitching system.

Abstract. In the process of image stitching, the ORB (Oriented FAST and Rotated BRIEF) algorithm lacks the characteristics of scale invariance and high mismatch rate. A principal component invariant feature transform (PCA-ORB, Principal Component Analysis- Oriented) is proposed. FAST and Rotated BRIEF) image stitching method. Firstly, the ORB algorithm is used to optimize the feature points to obtain the feature points with uniform distribution. Secondly, the principal component analysis (PCA) method can reduce the dimension of the traditional ORB feature descriptor and reduce the complexity of the feature point descriptor data. Thirdly, KNN (K-Nearest Neighbor) is used, and the k-nearest neighbor algorithm performs roughly matching on the feature points after dimensionality reduction. Then the random matching consistency algorithm (RANSAC, Random Sample Consensus) is used to remove the mismatched points. Finally, the fading and fading fusion algorithm is used to fuse the images. In 8 sets of simulation experiments, the image stitching speed is improved relative to the PCA-SIFT algorithm. The experimental results show that the proposed algorithm improves the image stitching speed under the premise of ensuring the quality of stitching, and can play a role in fast, real-time and large-scale applications, which are conducive to image fusion.

The feature point method is the mainstream method to accomplish inter-frame estimation in visual Simultaneous Localization and Mapping (SLAM) methods, among which the Oriented FAST and Rotated BRIEF (ORB) feature-based method provides an equilibrium of accuracy as well as efficiency. However, the ORB algorithm is prone to clustering phenomena, and its unequal distribution of extracted feature points is not conducive to the subsequent camera tracking. To solve the above problems, this paper suggests an adaptive feature extraction algorithm that first constructs multiple-scale images using an adaptive Gaussian pyramid algorithm, calculates adaptive thresholds, and uses an adaptive meshing method for regional feature point detection to adapt to different scenes. The method uses Adaptive and Generic Accelerated Segment Test (AGAST) to speed up feature detection and the non-maximum suppression method to filter feature points. The feature points are then divided equally by a quadtree technique, and the orientation of those points is determined by an intensity centroid approach. Experiments were conducted on publicly available datasets, and the outcomes demonstrate the algorithm has good adaptivity and solves the problem of a large number of corner point clusters that may result from using manually set detection thresholds. The RMSE of the absolute trajectory error of SLAM applying this method on four sequences of TUM RGB-D datasets is decreased by 13.88% when compared with ORB-SLAM3. It is demonstrated that the algorithm provides high-quality feature points for subsequent image alignment, and the application to SLAM improves the reliability and accuracy of SLAM.

For the problem of poor image quality captured with the camera on the telescopic arm of the offshore wind turbine boarding bridge due to the wave fluctuation, an image stabilization algorithm based on adaptive Oriented Fast and Rotated Brief (ORB) is proposed. The threshold of feature point decision function and the size of feature point searching box in ORB algorithm are updated in real time to reduce the impact of camera shake on the captured image, and the detection speed is improved by removing the invalid feature points in the image. The image stabilization algorithm proposed in this paper is more suitable for real-time detection of offshore wind turbine logo images than the ORB, SIFT and Harris-based algorithms with an average improvement of 25.9%.

Image stitching technique is to integrate multiple images with overlapping regions into a complete image with a wide viewing angle, less distortion, and no obvious suture. Image stitching could be used for global positioning and robot autonomous navigation without changing the hardware. SIFT feature and SURF feature are the classical algorithm in the image stitching. But they have the long time-consuming problem. Facing this problem, a novel fast image stitching algorithm based on ORB (Oriented FAST and Rotated BRIEF) features is proposed in this paper. The algorithm firstly selects ORB algorithm which adds the direction information to the FAST detector for image feature extracting and matching. Secondly, RASANC (Random Sample Consensus) algorithm is used to eliminate false matching points. Finally, weighted average method is used to speed up the image fusion. Experimental results show that the stitching effect of the algorithm in this paper is the same as that of SIFT algorithm and SURF algorithm. The two obvious advantages of our algorithm include: (1) stitching speed is much faster than that of Shift and Surf;(2)The algorithm is robust to external parameters of camera. So the proposed algorithm is hopefully applied to some handed electrical devices.

Chapter 9 - Monocular simultaneous localization and mapping for a space rover application

Computer vision is one of the most active research fields in information technology today. Giving machines and robots the ability to see and comprehend the surrounding world at the speed of sight creates endless potential applications and opportunities. Feature detection and description algorithms can be indeed considered as the retina of the eyes of such machines and robots. However, these algorithms are typically computationally intensive, which prevents them from achieving the speed of sight real-time performance. In addition, they differ in their capabilities and some may favor and work better given a specific type of input compared to others. As such, it is essential to compactly report their pros and cons as well as their performances and recent advances. This paper is dedicated to provide a comprehensive overview on the state-of-the-art and recent advances in feature detection and description algorithms. Specifically, it starts by overviewing fundamental concepts. It then compares, reports and discusses their performance and capabilities. The Maximally Stable Extremal Regions algorithm and the Scale Invariant Feature Transform algorithms, being two of the best of their type, are selected to report their recent algorithmic derivatives.

Simultaneous Localization and Mapping (SLAM) is a system used to achieve autonomous positioning and navigation. Feature detection is an important part of a SLAM system as fast and robust image matching is required for the task. A typical feature detection algorithm called Speeded-Up Robust Features (SURF) is used in a robot SLAM system with Moving Object Detection (MOD). This paper describes a modified feature detection algorithm based on Field Programmable Gate Array (FPGA) hardware. The paper focuses on implementing the software algorithm on a hardware platform. The advantage of the parallel and pipelining design of FPGA is fully applied to highly improve the performance and efficiency of the system. By using the FPGA hardware platform, the algorithm can also be implemented easily in an FPGA-based SLAM system afterward to finally use for System-On-Chip (SoC) applications.

Object recognition is a key research area in the field of image processing and computer vision, which recognizes the object in an image and provides a proper label. In the paper, three popular feature descriptor algorithms that are Scale Invariant Feature Transform (SIFT), Speeded Up Robust Feature (SURF) and Oriented Fast and Rotated BRIEF (ORB) are used for experimental work of an object recognition system. A comparison among these three descriptors is exhibited in the paper by determining them individually and with different combinations of these three methodologies. The amount of the features extracted using these feature extraction methods are further reduced using a feature selection (k-means clustering) and a dimensionality reduction method (Locality Preserving Projection). Various classifiers i.e. K-Nearest Neighbor, Naive Bayes, Decision Tree, and Random Forest are used to classify objects based on their similarity. The focus of this article is to present a study of the performance comparison among these three feature extraction methods, particularly when their combination derives in recognizing the object more efficiently. In this paper, the authors have presented a comparative analysis view among various feature descriptors algorithms and classification models for 2D object recognition. The Caltech-101 public dataset is considered in this article for experimental work. The experiment reveals that a hybridization of SIFT, SURF and ORB method with Random Forest classification model accomplishes the best results as compared to other state-of-the-art work. The comparative analysis has been presented in terms of recognition accuracy, True Positive Rate (TPR), False Positive Rate (FPR), and Area Under Curve (AUC) parameters.

Under complex lighting conditions, hospital logistics robots are facing serious challenges in positioning and navigation.The traditional ORB (Oriented FAST and rotated BRIEF) algorithm often has problems such as unstable feature point extraction, poor positioning accuracy, and long navigation path planning time in environments with large lighting changes, which greatly affects the robot's navigation efficiency and accuracy. In this paper, an improved ORB-SLAM3 algorithm is used to improve the navigation performance of the robot in a hospital environment with complex lighting. The improved ORB-SLAM3 algorithm extracts feature points of different scales by constructing an image pyramid to ensure that effective visual information can be stably obtained under different lighting conditions; the strategy of combining adaptive threshold and dual threshold methods is used to optimize the accuracy of feature point extraction, and the feature points are efficiently managed through a four-tree model to ensure the uniform distribution of feature points. These measures significantly improve the quality and matching accuracy of feature point extraction; the algorithm also combines visual and inertial information closely by combining inertial measurement unit data, further enhancing the positioning accuracy and system stability.The experimental results show that in the environment of rapid light changes, the average positioning error, navigation map construction integrity and navigation planning time of the improved ORB-SLAM3 algorithm are 0.26 m, 80% and 7.26 ms, respectively.This achievement not only effectively improves the navigation efficiency of the hospital logistics robot, but also enhances its stability and reliability in an environment with severe lighting changes. It is of great practical significance to improve the level of intelligence of the hospital's internal logistics system.

Compared to ordinary camera images, aerial images captured by UAVs mostly consist of small targets in the distance, with complex image information and rich detail. However, aerial images can be easily affected by factors such as lighting differences and atmospheric conditions, which leads to noticeable color variations between images. Traditional image stitching algorithms applied to drone aerial images often suffer from insufficient accuracy. To improve image stitching accuracy and efficiency, this paper proposes an improved ORB (Oriented Fast and Rotated Brief) feature extraction algorithm. The traditional ORB algorithm has shortcomings in rotation invariance, uneven distribution, and few feature points in the image to be registered. Firstly, this paper enhances the FAST feature point detection algorithm by setting an adaptive threshold based on the image's grayscale value to extract more feature point data. Secondly, the selection strategy of surrounding pixels is optimized during the feature point validation process to reduce the calculation time wasted on false feature points. Finally, the gray centroid method is used to add the main direction to the feature points found by the FAST algorithm to enhance matching accuracy under image rotation. Moreover, this paper optimizes the preprocessing stage of the algorithm to quickly solve the image transformation matrix, and thereby further improve the accuracy of image stitching. The L-M (Levenberg-Marquardt) method is combined with RANSAC to refine the obtained transformation matrix and improve the stitching accuracy. Experimental results demonstrate that the optimized algorithm applied to drone aerial image stitching achieves shorter processing times, higher matching accuracy, and better stitching performance.

Adaptive ORB feature detection with a variable extraction radius in RoI for complex illumination scenes

Target tracking and feature detection in 3D registration of augmented reality(AR) system are vulnerable to environment, time-consuming algorithm and low accuracy. An AR tracking registration method based on improved Kernerlized Correlation Filter(KCF) KCF tracking and ORB (Oriented FAST and Rotated BRIEF) feature detection is proposed. Firstly, KCF algorithm is employed to track the registered target position. By considering the strategy of all previous frames, the problem of real-time updating of the weight vector of the target model in KCF tracking is solved, which makes the tracking of the target position real-time and effective. ORB operator with high real-time performance is used to detect the target position. Then the feature points of the detected target position are matched, and the three-dimensional registration matrix is calculated according to the matched feature points, and the three-dimensional registration is carried out. The experimental results show that the improved method achieves better real-time performance, stability and robustness.

Audio fingerprinting has been an active research field typically used for music identification. Robust audio fingerprinting technology is used to successfully perform content-based audio identification regardless of the audio signal being subjected to various types of distortion. These distortions affect the time-frequency correlation relating to pitch and speed changes. In this paper, experiments are done using the computer vision technique ORB (Oriented FAST and Rotated BRIEF) for robust audio identification. Investigations are conducted for ORB, relating to its advantage of robustness against distortions including speed and pitch changes. The ORB prototype compares the features of the spectrogram image query to a database of spectrogram images of the songs. For the initial experiment, a Brute-Force matcher is used to compare the ORB descriptors. Results show that the ORB prototype performs robustly to real-world distortions with fast, reliable performance against distortions such as speed and pitch which justifies the research done.

In this paper, a novel feature-based real-time visual Simultaneous localization and mapping (SLAM) system is proposed. This system generates colored 3-D reconstruction models and 3-D estimated trajectory using a Kinect style camera. Microsoft Kinect, a low priced 3-D camera, is the only sensor we use in our experiment. Kinect style sensors give RGB-D (red-green-blue depth) data which contains 2D image and per-pixel depth information. ORB (Oriented FAST and Rotated BRIEF) is the algorithm used to extract image features for speed up the whole system. Our system can be used to generate 3-D detailed reconstruction models. Furthermore, an estimated 3D trajectory of the sensor is given in this paper. The results of the experiments demonstrate that our system performs robustly and effectively in both getting detailed 3D models and mapping camera trajectory.