Improved SIFT matching algorithm for 3D reconstruction from endoscopic images

SIFT (scale invariant feature transform) is one of the most robust and the widely used image matching algorithms based on local features. It has been used in the aspect of the cameras target matching. For its high complexity, this paper proposes a target matching method for multiple cameras by combining SIFT and the rotation-invariant LBP (local binary patterns). The object detection is obtained by improved Back-ground subtraction technique, and particle filter is used for region-based object tracking. The descriptor combining SIFT and the rotation-invariant LBP is calculated for every object region. The object matching is obtained by comparing the distance of feature matrix. Experimental results show that the proposed algorithm is more rapid than the standard SIFT algorithm while the performance is favorably compared to the standard SIFT algorithm, and it is robust in various scenes.

There remain many difficult problems in computer vision research such as object recognition, three dimensional reconstruction, object tracking, etc. And the basis of solving these problems relies on image matching. The scale invariant feature transform (SIFT) algorithm has been widely used for image matching application. The SIFT algorithm can successfully extract the most descriptive feature points in given input images taken under different viewpoints. However, the performance of the original SIFT algorithm degrades under the influence of noise. We propose to modify the SIFT algorithm to produce better invariant feature points for image matching under noise. We also propose to employ the Earth mover's distance (EMD) as the measurement of similarity between two descriptors. We present extensive experiment results to demonstrate the performance of the proposed methods in image matching under noise.

With the emerging techniques of wireless communication and cloud computing, large volumes of multimedia data are outsourced from resource constrained users to the cloud with abundant resource for both delegated storage and computation. Unfortunately, there is a risk of users’ image privacy leakage in the process of outsourcing to untrusted cloud. Most of the existing work achieved privacy-preserving image feature extraction and matching by using public key (fully) homomorphic encryption (FHE), but the heavy computational overhead and communication overhead cannot adapt to resource-constrained mobile devices. Other works disabled to realize image denoising in the encrypted domain or only focused on the scale-invariant feature transform (SIFT) descriptor that is inappropriate for position-sensitive feature extraction. To address these issues, in this paper, a privacy-preserving shape context based image denoising and matching protocol PPOIM with efficient outsourcing is proposed. Firstly, to improve the accuracy of image matching, a privacy-preserving image denoising scheme PPID is proposed without exploiting public key FHE. Then, based on PPID, a privacy-preserving image matching protocol PPOIM adopting shape context descriptor is devised, where two secure and efficient comparison and counting protocols in the encrypted domain are presented. All the original image privacy, query image privacy and image matching result privacy are well protected. Finally, formal security proof and extensive simulations on real-world data sets demonstrate the efficiency and practicability of our proposed PPOIM.

Abstract. Historical aerial photographs directly provide good evidences of past times. The Research Center for Humanities and Social Sciences (RCHSS) of Taiwan Academia Sinica has collected and scanned numerous historical maps and aerial images of Taiwan and China. Some maps or images have been geo-referenced manually, but most of historical aerial images have not been registered since there are no GPS or IMU data for orientation assisting in the past. In our research, we developed an automatic process of matching historical aerial images by SIFT (Scale Invariant Feature Transform) for handling the great quantity of images by computer vision. SIFT is one of the most popular method of image feature extracting and matching. This algorithm extracts extreme values in scale space into invariant image features, which are robust to changing in rotation scale, noise, and illumination. We also use RANSAC (Random sample consensus) to remove outliers, and obtain good conjugated points between photographs. Finally, we manually add control points for registration through least square adjustment based on collinear equation. In the future, we can use image feature points of more photographs to build control image database. Every new image will be treated as query image. If feature points of query image match the features in database, it means that the query image probably is overlapped with control images.With the updating of database, more and more query image can be matched and aligned automatically. Other research about multi-time period environmental changes can be investigated with those geo-referenced temporal spatial data.

Efficient detection and reliable matching of image features constitute a fundamental task in computer vision. When real-time operation is required, the solution to this problem becomes a real challenge, because of increased processing requirements. Scale Invariant Feature Transform (SIFT) is considered as a stable and robust algorithm for the extraction of invariant features, however special hardware is required for high frame-rate applications, in order to overcome its computational complexity in real-time. In this paper, a complete processor for SIFT feature matching in video sequences is presented. It performs SIFT feature extraction and matching, as well as rejection of false correspondences using the random sample consensus (RANSAC) algorithm. The processor was evaluated using metrics like response, repeatability and recall. It was found that the proposed fixed-point hardware implementation has a comparable performance with software implementations of SIFT. The processor architecture can process more than 36 fps with resolution 640×480, when it is implemented in Cyclone IV FPGA technology.

In computer vision, Local Binary Pattern (LBP) and Scale Invariant Feature Transform (SIFT) are two widely used local descriptors. In this paper, we propose to combine them effectively for scene categorization. First, LBP and SIFT features are regularly extracted from training images for constructing a LBP feature codebook and a SIFT feature codebook. Then, a two-dimensional table is created by combining the obtained codebooks. For creating a representation for an image, LBP and SIFT features extracted from the same positions of the image are encoded together based on sparse coding by using the two-dimensional table. After processing all features in the input image, we adopt spatial max pooling to determine its representation. Obtained image representations are forwarded to a Support Vector Machine classifier for categorization. In addition, in order to improve the scene categorization performance further, we propose a method to select correlated visual words from large codebooks for constructing the two-dimensional table. Finally, for evaluating the proposed method, extensive experiments are implemented on datasets Scene Categories 8, Scene Categories 15 and MIT 67 Indoor Scene. It is demonstrated that the proposed method is effective for scene categorization.

In this paper we propose a novel feature based on SIFT (Scale Invariant Feature Transform) algorithm1 for the robust representation of local visual contents. SIFT features have raised much interest for their power of description of visual content characterizing punctual information against variation of luminance and change of viewpoint and they are very useful to capture local information. For a single image hundreds of keypoints are found and they are particularly suitable for tasks dealing with image registration or image matching. In this work we stretched the spatial coverage of descriptors creating a novel feature as composition of keypoints present in an image region while maintaining the invariance properties of SIFT descriptors. The number of descriptors is reduced, limiting the computational weight, and at the same time a more abstract descriptor is achieved. The new feature is therefore suitable at describing objects and characteristic image regions. We tested the retrieval performance with a dataset used to test PCA SIFT2 and image matching capability among images processed with affine transformations. Experimental results are reported.

Multiplicative speckle noise often significantly affects the accuracy and adaptability of the scale-invariant feature transform (SIFT) matching method for synthetic aperture radar (SAR) images. To address this problem, this study proposes a union matching method based on the SIFT and edge strength of the SAR image. First, the rotation constraint iteratively refines the initial SIFT match set based on the parameter decomposition of the common geometry transformation model. Using this model, square summation strength (SSS) similarity is then determined. To get the optimal SIFT matches in the searching space, SSS matching based on pixel migration is applied. Furthermore, we outlined the optimization procedure of a union matching strategy. In the iterative process of global searching, the correctly matched tie-points are added individually. Finally, the accuracy, adaptability, and precision of the proposed method are validated through matching experiments on SAR images. Results showed that the proposed method is accurate and robust with respect to the automatic matching of SAR images.

A novel image registration method is proposed based on the scale-invariant feature transform (SIFT). Our approach combines the image frequency spectrum with a regular grid. The method exploits features of SIFT, the greater number of feature points in high-frequency areas of an image and regular grid traits with a fine spatial distribution to extract evenly distributed feature points. Global correspondence error checking is used to remove false matching pairs. The registration of eight groups of remote sensing images shows that our method has well-distributed feature points in the spatial and scale domains, obtains an average of 15.78% matching pairs more than uniform robust (UR)-SIFT, and provides better registration accuracy than SIFT and UR-SIFT.

Road visual navigation relies on accurate road models. This study was aimed at proposing an improved scale-invariant feature transform (SIFT) algorithm for recovering depth information from farmland road images, which would provide a reliable path for visual navigation. The mean image of pixel value in five channels (R, G, B, S and V) were treated as the inspected image and the feature points of the inspected image were extracted by the Canny algorithm, for achieving precise location of the feature points and ensuring the uniformity and density of the feature points. The mean value of the pixels in 5×5 neighborhood around the feature point at an interval of 45o in eight directions was then treated as the feature vector, and the differences of the feature vectors were calculated for preliminary matching of the left and right image feature points. In order to achieve the depth information of farmland road images, the energy method of feature points was used for eliminating the mismatched points. Experiments with a binocular stereo vision system were conducted and the results showed that the matching accuracy and time consuming for depth recovery when using the improved SIFT algorithm were 96.48% and 5.6 s, respectively, with the accuracy for depth recovery of –7.17%-2.97% in a certain sight distance. The mean uniformity, time consuming and matching accuracy for all the 60 images under various climates and road conditions were 50%-70%, 5.0-6.5 s, and higher than 88%, respectively, indicating that performance for achieving the feature points (e.g., uniformity, matching accuracy, and algorithm real-time) of the improved SIFT algorithm were superior to that of conventional SIFT algorithm. This study provides an important reference for navigation technology of agricultural equipment based on machine vision. Keywords: scale-invariant feature transform (sift), feature matching, canny operator, energy method of feature point, farmland road, depth recovery, visual navigation DOI: 10.25165/j.ijabe.20191204.4821 Citation: Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric & Biol Eng, 2019; 12(4): 141–147.

 Untitled Page In this thesis we study the running of the Scale Invariant Feature Transform (SIFT) algorithm on a pipelined computational platform. The SIFT algorithm is one of the most widely used methods for image feature extraction. We develop a tile based template for running SIFT that facilitates the analysis while abstracting away lower-level details. We formalize the computational pipeline and the time to execute any algorithm on it based on the relative times taken by the pipeline stages. In the context of the SIFT algorithm, this reduces the time to that of running the entire image through a bottlenecked stage and the time to run either the first or last tile through the remaining stages. Through an experimental study of the SIFT algorithm on a broad collection of test images, we determined image feature fraction values, that relate the sizes of the image extracts as it the computation proceeds through the stages of the SIFT algorithm. We show that for a single chip uniprocessor pipeline, the computational stage is the bottleneck. Specifically we show that for an N x N image with n x n tiles the overall time complexity is θ ( (n+x)2 pi Г0+αβN2x2 Г1+ (αβ+γ)n2logx P0 Г2 ) ; here x is the neigborhood of the tile, pi , po are the number of input, output pins of the chip, α,β,γ are the feature fractions, and Г0, Г1, Г2 are the input, compute, output clocks. The three terms in the expression represents the time complexities of input, compute and output stages. The input and output stages can be slowed down substantially without appreciate degradation of the overall performance. This slowdown can be traded off for lower power and higher signal quantity. For multicore chips, we show that for an N x N image on a P-core chip, the overall time complexity to process the image is θ ( N2 pi Г0+ (n2w2 + αβn2x2) P Г1+ (αβ+γ)n2logx P0 Г2 ) ; in addition to the quantities described earlier w is the window size used for the Gaussian blurring. Overall we establish that without improvements in the input bandwidth, the power of multicore processing cannot be used efficiently for SIFT.

This paper presents a multimodal face recognition system for profile views in such scenarios as frontal face images are not available. According to the special physiological location relationship and their supplement between face and ear, a framework of multimodal recognition system fusing profile-view face classifier and ear classifier is proposed. In profile-view face classifier, a new feature extraction and recognition algorithm based on Scale Invariant Feature Transform(SIFT)and local binary patterns (LBP) is applied. Firstly, a set of keypoints are extracted from the face image for matching by applying the SIFT algorithm; Secondly, each keypoint is described by the rotation-invariant LBP patterns; Finally,the matching pairs between the two sets of keypoints are determined by using the nearest neighbor distance ratio based matching strategy. Ear classifier is set up based on LDA. Then decision fusion of multimodal face recognition is carried out using the combination methods of Product, Sum, Median and Vote rules according to the Bayesian theory. The results of experiment show that our method improves the recognition performance and provides a new approach of non-intrusive recognition.KeywordsMultimodal recognitionSIFTLBPDecision fusion

Aiming at the problems of large calculating scale and high complexity in Scale Invariant Feature Transform (SIFT) feature matching algorithm, this paper presents an improved SIFT feature matching algorithm based on quasi Euclidean distance.The traditional Euclidean distance can only calculate out the variance of the two images to the corresponding pixel, so when a slight shift or distortion occurs in the image, it may produce a large deviation.The quasi Euclidean distance instead of Euclidean distance is as the similarity measure of feature descriptors to improve the SIFT feature matching.It reduces the dimensions of SIFT feature vector to improve the efficiency of feature matching.Experimental results show that under the condition of keeping the image matching rate and algorithm robust, the method can not only improve the matching accuracy but also shorten the matching time.The new algorithm has better performance than traditional algorithm, it is possible and valid, which are useful for the fields of image recognition, image reconstruction, etc.

Scale Invariant Feature Transform (SIFT) is used for local features description of images. The proposed technique employs SIFT in order to match binary images. While the employment of SIFT descriptors on binary image databases could be possible, its power is rather limited. The novelty of the proposed algorithm is make use of SIFT for binary image matching by taking the power of Hough Transform (HT) in line detection. HT can be used on any line orientation. Thus, HT investigates lines as criteria for binary image similarity beside SIFT features for local and corner descriptions. The evaluation is achieved and experimental highlights the superiority of this approach for binary images which contains straight lines.

Automatic image classification is the first step toward semantic understanding of an object in the computer vision area. The key challenge of problem for accurate object recognition is the ability to extract the robust features from various viewpoint images and rapidly calculate similarity between features in the image database or video stream. In order to solve these problems, an effective and rapid image classification method was presented for the object recognition based on the video learning technique. The optical-flow and RANSAC algorithm were used to acquire scene images from each video sequence. After the selection of scene images, the local maximum points on corner of object around local area were found using the Harris corner detection algorithm and the several attributes from local block around each feature point were calculated by using scale invariant feature transform (SIFT) for extracting local descriptor. Finally, the extracted local descriptor was learned to the three-dimensional pyramid match kernel. Experimental results show that our method can extract features in various multi-viewpoint images from query video and calculate a similarity between a query image and images in the database.