A Study on SURF Algorithm and Real-Time Tracking Objects Using Optical Flow

In order to improve the speed of stitching and information extraction of remote sensing images, based on the UAV (Unmanned Aerial Vehicle) image stitching algorithm, this paper proposes a UAV image matching method based on the combination of SURF (Speeded Up Robust Features) and MASC (M-Estimate Sample Consensus) algorithm. the SURF algorithm is fast, and the MASC algorithm can effectively eliminate mismatch points and has high stability. In this paper, the SIFT (Scale-invariant feature transform) algorithm, SURF algorithm, SURF and RANSAC (Random Sample Consensus) algorithm as well as SURF and MASC algorithm are used to stitch the same two images and compare them respectively. The experimental results show that the lowest image matching error rate is only 2.13% and the matching time is 4.772670s when using the combination of SURF and MASC algorithm for image stitching, which has certain advantages.

This study proposes an improved feature-point matching and 3D registration method for augmented reality tourism applications. This method addresses issues such as poor alignment, low stability in complex environments, and low feature-matching accuracy. An improved feature point matching method for tourism images is introduced, which combines the Speeded Up Robust Features (SURF) algorithm with an enhanced Oriented FAST and Rotated BRIEF algorithm. In this method, feature points are initially detected using the SURF algorithm, and their orientation is determined via wavelet response analysis. The Lucas-Kanade optical flow method is employed for feature point tracking. The random sample consensus algorithm is then used to eliminate mistracked points. Furthermore, an augmented reality tourism 3D registration technique based on an improved homography matrix is proposed to overcome the limitations of traditional homography matrices, such as low matching accuracy and registration efficiency. The performance of the proposed method was analyzed through comparative experiments against the SIFT, SURF, and original ORB algorithms under various image transformations, including scale, blur, illumination, and rotation. The correct matching rate and matching time were used as evaluation metrics. Simulation tests were conducted for 3D registration using different 3D models. Registration accuracy and successful registration counts were evaluated under rotational changes. The outcomes indicated that the average correct matching rate of the proposed algorithm is increased by 44.08%, 36.51%, and 16.09% under scale variation than the scale invariant feature transformation algorithm, speeded up robust features algorithm, and unimproved algorithm, respectively. The correct matching rate under fuzzy transformation increased by 33.46%, 19.65%, and 9.35%, respectively. The average registration accuracy of the proposed 3D registration technique was 98.74% under rotational transformation. The outcomes reveal that the study's suggested approach can successfully improve the scene's virtual and real-world fusion effect and offers a fresh approach to the use of augmented reality technology in the travel industry.

Speeded Up Robust Features (SURF) algorithm is a fast robust local feature matching method. Because of the advantage of invariance of scale and rotation, the SURF algorithm has been widely used in the image registration. However, due to the acquired angle, imaging mode, different resolution, SURF algorithm becomes quite difficult in the image registration. This paper presents a normalized SURF algorithm which can reduce the impact of hue difference between remote sensing images. Combined with the idea of two-way matching, the normalized SURF is able to improve the accuracy of the image registration. The experimental results show the normalized SURF algorithm can keep greater accuracy and relatively higher speed of matching than the original SURF algorithm.

Abstract. Remote sensing system fitted on UAV (Unmanned Aerial Vehicle) can obtain clear images and high-resolution aerial photographs. It has advantages of flexibility, convenience and ability to work full-time. However, there are some problems of UAV image such as small coverage area, large number, irregular overlap, etc. How to obtain a large regional map quickly becomes a major obstacle to UAV remote sensing application. In this paper, a new method of fast registration of UAV remote sensing images was proposed to meet the needs of practical application. This paper used Progressive Sample Consensus (PROSAC) algorithm to improve the matching accuracy by removed a large number of mismatching point pairs of remote sensing image registration based-on SURF (Speed Up Robust Feature) algorithm, and GPU (Graphic Processing Unit) was also used to accelerate the speed of improved SURF algorithm. Finally, geometric verification was used to achieve mosaic accuracy in survey area. The number of feature points obtained by using improved SURF based-on PROSAC algorithm was only 9.5% than that of SURF algorithm. Moreover, the accuracy rate of improved method was about 99.7%, while the accuracy rate of improved SURF algorithm was increased by 8% than SURF algorithm. Moreover, the improved running time of SURFGPU algorithm for UAV remote sensing image registration was a speed of around 16 times than SURF algorithm, and the image matching time had reached millisecond level. Thus, improved SURF algorithm had better matching accuracy and executing speed to meet the requirements of real-time and robustness in UAV remote sensing image registration.

Purpose – To build a persistent map with visual landmarks is one of the most important steps for implementing the visual simultaneous localization and mapping (SLAM). The corner detector is a common method utilized to detect visual landmarks for constructing a map of the environment. However, due to the scale-variant characteristic of corner detection, extensive computational cost is needed to recover the scale and orientation of corner features in SLAM tasks. The purpose of this paper is to build the map using a local invariant feature detector, namely speeded-up robust features (SURF), to detect scale- and orientation-invariant features as well as provide a robust representation of visual landmarks for SLAM. Design/methodology/approach – SURF are scale- and orientation-invariant features which have higher repeatability than that obtained by other detection methods. Furthermore, SURF algorithms have better processing speed than other scale-invariant detection method. The procedures of detection, description and matching of regular SURF algorithms are modified in this paper in order to provide a robust representation of visual landmarks in SLAM. The sparse representation is also used to describe the environmental map and to reduce the computational complexity in state estimation using extended Kalman filter (EKF). Furthermore, the effective procedures of data association and map management for SURF features in SLAM are also designed to improve the accuracy of robot state estimation. Findings – Experimental works were carried out on an actual system with binocular vision sensors to prove the feasibility and effectiveness of the proposed algorithms. EKF SLAM with the modified SURF algorithms was applied in the experiments including the evaluation of accurate state estimation as well as the implementation of large-area SLAM. The performance of the modified SURF algorithms was compared with those obtained by regular SURF algorithms. The results show that the SURF with less-dimensional descriptors is the most suitable representation of visual landmarks. Meanwhile, the integrated system is successfully validated to fulfill the capabilities of visual SLAM system. Originality/value – The contribution of this paper is the novel approach to overcome the problem of recovering the scale and orientation of visual landmarks in SLAM tasks. This research also extends the usability of local invariant feature detectors in SLAM tasks by utilizing its robust representation of visual landmarks. Furthermore, data association and map management designed for SURF-based mapping in this paper also give another perspective for improving the robustness of SLAM systems.

This paper presents an efficient feature detection algorithm based on the classical SURF (Speeded Up Robust Feature) detector. The image features are represented and scored with respect to its local symmetry property. The local symmetry has natural properties of scale and transformation invariants, and also insensitive to illumination change and local noise. By the proposed feature descriptor, the calculation of 64-dimensional vectors in SURF algorithm can be reduced to 16-dimensional vector respectively. The local symmetry score is defined as the sum of minimum distance between each feature point and its neighboring points in an image based on the image intensities. The algorithm is experimented with some real images and the results are compared with the original SURF algorithm to show its improvement.

위성 SAR 영상의 활용이 증가하면서 영상의 해석 및 융합을 위한 정밀 기하보정에 대한 필요성이 높아지고 있다. 특히 광역감시 목적으로 활용되기 위해 서로 다른 해상도를 갖는 SAR 영상간 정보융합도 활발해지고 있다. 일반적으로 SAR 영상의 기하보정은 위성의 궤도 및 자세정보를 활용하여 수행할 수 있지만 SAR 센서의 궤도 및 시스템 오차, 대상지형 특성에 의한 왜곡으로 인해 추가적인 보정이 필요하게 된다. SAR 영상을 통한 변화탐지나 타 영상과의 융합에 적용하기 위해서는 기하 오차 보정이 반드시 선행되어야 한다. 이를 위해 다수의 지상 기준점을 선정하고 이를 포함하는 기준 영상과 비교하여 원본 영상에서 대응점을 찾는 방식으로 정밀 기하보정을 수행할 수 있다. Speeded Up Robust Feature (SURF) 기법은 쉽고 빠르게 영상의 기준점을 찾을 수 있지만 상대적으로 해상도가 낮고 스펙클 잡음에 영향을 받는 SAR 영상에서는 활용하기가 어렵다고 알려져 있다. 본 논문에서는 SURF 기법을 위성 SAR 영상에 적용할 때 발생할 수 있는 오차를 추출하고 영상 특성에 따른 성능 변화를 분석하였다. SURF 알고리즘의 적용이 가능한 입력 변수의 적정 범위를 제시하고 그에 따른 영상 정합의 오차를 분석하여 중저해상도의 위성 SAR 영상에 대해서도 SURF 기법을 통한 기하 보정 및 영상 정합이 적용될 수 있음을 검증하였다. As applications of spaceborne SAR imagery are extended, there are increased demands for accurate registrations for better understanding and fusion of radar images. It becomes common to adopt multi-resolution SAR images to apply for wide area reconnaissance. Geometric correction of the SAR images can be performed by using satellite orbit and attitude information. However, the inherent errors of the SAR sensor's attitude and ground geographical data tend to cause geometric errors in the produced SAR image. These errors should be corrected when the SAR images are applied for multi-temporal analysis, change detection applications and image fusion with other sensor images. The undesirable ground registration errors can be corrected with respect to the true ground control points in order to produce complete SAR products. Speeded Up Robust Feature (SURF) technique is an efficient algorithm to extract ground control points from images but is considered to be inappropriate to apply to SAR images due to high speckle noises. In this paper, an attempt is made to apply SURF algorithm to SAR images for image registration and fusion. Matched points are extracted with respect to the varying parameters of Hessian and SURF matching thresholds, and the performance is analyzed by measuring the imaging matching accuracies. A number of performance measures concerning image registration are suggested to validate the use of SURF for spaceborne SAR images. Various simulations methodologies are suggested the validate the use of SURF for the geometric correction and image registrations and it is shown that a good choice of input parameters to the SURF algorithm should be made to apply for the spaceborne SAR images of moderate resolutions.

How to construct compact and powerful feature descriptors is an important research subject in the fields of machine vision and pattern recognition. After the research on SURF, the issue that the Haar descriptor of Speeded Up Robust Features(SURF) algorithm can not make full use of the information around the neighborhood of the feature points is found. To resolve the issue, this paper proposes a improved SURF based on the original SURF algorithm. The algorithm makes full use of the information around the neighborhood of the sub-block by masking with 3*3 window templates and then constructs the descriptor which can obtain better discriminative power. In addition, aimed at the primary direction of the SURF depends too much on the local region, the paper improves the confirmation of the primary direction. At last, this paper combines the nearest distance and the ratio between the nearest distance and the second nearest distance to match the interest points. The experiment shows that the algorithm proposed in this paper has better discriminative power, and the face recognition rate is enhanced.

This paper proposes a VLSI architecture of the optimized Speeded-Up Robust Feature (SURF) algorithm. The SURF algorithm which is widely used in computer vision applications, locates interest points (IPoints) and extracts feature descriptors based on the surrounding gradients. In the proposed approach, the SURF algorithm is modified to make it more suitable for hardware implementation with little accuracy lost compared to OpenSURF, an open source software implementation based on OpenCV. The resource cost and throughput gain are balanced. Word Length Reduction (WLR) is adopted to compress the integral image and reduce the occupied memory resources. The orientations and the feature descriptors of the IPoints are calculated in a more efficient way, which significantly reduces memory accesses. Moreover, the operations are pipelined both in and among the proposed hardware modules. This VLSI architecture has been validated on FPGA (Xilinx Virtex-4 XC4VLX80), which is able to detect IPoints and extract SURF feature descriptors in a VGA (640 × 480) 64 fps video input with a 96 MHz working frequency while dissipating 1.278 W. This throughput is more than double of the ones reported in the latest literatures.

Farsi font detection is considered as the first stage in the Farsi optical character recognition (FOCR) of scanned printed texts. To this aim, this paper proposes an improved version of the speeded-up robust features (SURF) algorithm, as the feature detector in the font recognition process. The SURF algorithm suffers from creation of several redundant features during the detection phase. Thus, the presented version employs the redundant keypoint elimination method (RKEM) to enhance the matching performance of the SURF by reducing unnecessary keypoints. Although the performance of the RKEM is acceptable in this task, it exploits a fixed experimental threshold value which has a detrimental impact on the results. In this paper, an Adaptive RKEM is proposed for the SURF algorithm which considers image type and distortion, when adjusting the threshold value. Then, this improved version is applied to recognize Farsi fonts in texts. To do this, the proposed Adaptive RKEM-SURF detects the keypoints and then SURF is used as the descriptor for the features. Finally, the matching process is done using the nearest neighbor distance ratio. The proposed approach is compared with recently published algorithms for FOCR to confirm its superiority. This method has the capability to be generalized to other languages such as Arabic and English.

To solve the low accuracy of image feature matching in horticultural robot visual navigation, an innovative and effective image feature matching algorithm was proposed combining the improved Oriented FAST and Rotated BRIEF (ORB) and Lucas–Kanade (LK) optical flow algorithm. First, image feature points were extracted according to the adaptive threshold calculated using the Michelson contrast. Then, the extracted feature points were uniformed by the quadtree structure, which can reduce the calculated amount of feature matching, and the uniform ORB feature points were roughly matched to estimate the position of the feature points in the matched image using the improved LK optical flow. Finally, the Hamming distance between rough matching points was calculated for precise matching. Feature extraction and matching experiments were performed in four typical scenes: normal light, low light, high texture, and low texture. Compared with the traditional algorithm, the uniformity and accuracy of the feature points extracted by the proposed algorithm were enhanced by 0.22 and 50.47%, respectively. Meanwhile, the results revealed that the matching accuracy of the proposed algorithm increased by 14.59%, whereas the matching time and total time decreased by 39.18% and 44.79%, respectively. The proposed algorithm shows great potential for application in the visual simultaneous localization and mapping (V-SLAM) of horticultural robots to achieve higher accuracy of real-time positioning and map construction.

A design of parallel architecture for image feature detection and description is proposed in this article. The major component of this architecture is a 2D cellular network composed of simple reprogrammable processors, enabling the Hessian Blob Detector and Haar Response Calculation, which are the most computing-intensive stage of the Speeded Up Robust Features (SURF) algorithm. Combining this 2D cellular network and dedicated hardware for SURF descriptors, this architecture achieves real-time image feature detection with minimal software in the host processor. A prototype FPGA implementation of the proposed architecture achieves 1318.9 GOPS general pixel processing @ 100 MHz clock and achieves up to 118 fps in VGA (640 × 480) image feature detection. The proposed architecture is stand-alone and scalable so it is easy to be migrated into VLSI implementation.

We propose and evaluate a three-dimensional (3D) face recognition approach that applies the speeded up robust feature (SURF) algorithm to the depth representation of shape index map, under real-world conditions, using only a single gallery sample for each subject. First, the 3D scans are preprocessed, then SURF is applied on the shape index map to find interest points and their descriptors. Each 3D face scan is represented by keypoints descriptors, and a large dictionary is built from all the gallery descriptors. At the recognition step, descriptors of a probe face scan are sparsely represented by the dictionary. A multitask sparse representation classification is used to determine the identity of each probe face. The feasibility of the approach that uses the SURF algorithm on the shape index map for face identification/authentication is checked through an experimental investigation conducted on Bosphorus, University of Milano Bicocca, and CASIA 3D datasets. It achieves an overall rank one recognition rate of 97.75%, 80.85%, and 95.12%, respectively, on these datasets.

The keyframe image character of 3D animation refers to the image state of the character or scene and other objects presented at specific keyframes in the animation timeline. The character texture will be blurred, distorted and discontinuous in the animation sequence due to motion, perspective change or lighting difference. Due to the lack of optical flow information to guide the current 3D animation keyframe image enhancement process, it is difficult to effectively distinguish the foreground character from the background elements, which affects the algorithm’s ability of recognizing the animation character’s movement patterns. To this end, a character texture enhancement method based on the SIFT algorithm and LK (Lucas-Kanade) optical flow is investigated for 3D animation keyframe images. The SIFT algorithm is used to extract the feature points of the character in each frame of 3D animation image as the initial tracking template, the LK optical flow method is used to calculate the optical flow vector of the feature points in the new frame, and the weighted directional histogram is used to estimate the direction of the character’s movement, and then the feature points in the tracking template are optimized through feature matching and weight updating to complete the tracking of the character’s target in each frame of the image. According to the target tracking results, a fractional-order differential mask is constructed in the character target region, and an adaptive function is constructed by combining the modes of the local gradient, information entropy and variance parameters of the image, so as to realize the enhancement of the character texture of the key frame images of the 3D animation by dynamically adjusting the order of the fractional-order differential. The experimental results show that the method can realize the dynamic tracking of 3D animation image character, and the extracted 3D animation image character feature points show a balanced distribution, and the distribution ratio is reasonable. It can realize the texture enhancement of 3D animation keyframe images, and the enhanced animation character clothing, facial details and other information are clearly recognizable, so as to improve the overall quality and visual impact of 3D animation.

The strap-down missile-borne image guidance system can be easily affected by the unwanted jitters of the motion of the camera, and the subsequent recognition and tracking functions are also influenced, thus severely affecting the navigation accuracy of the image guidance system. So, a real-time image stabilization technology is needed to help improve the image quality of the image guidance system. To satisfy the real-time and accuracy requirements of image stabilization in the strap-down missile-borne image guidance system, an image stabilization method based on optical flow and image matching with binary feature descriptors is proposed. The global motion of consecutive frames is estimated by the pyramid Lucas-Kanade (LK) optical flow algorithm, and the interval frames image matching based on fast retina keypoint (FREAK) algorithm is used to reduce the cumulative trajectory error. A Kalman filter is designed to smooth the trajectory, which is conducive to fitting to the main motion of the guidance system. Simulations have been carried out, and the results show that the proposed algorithm improves the accuracy and real-time performance simultaneously compared to the state-of-art algorithms.