Fast algorithm for nearest neighbor search based on a lower bound tree

This paper presents a fast Nearest Neighbor Search (NNS) algorithm for geometric data sets, based on a modified k-d tree using the cache technique. First, the tree-based search process of our algorithm starts from an appropriate (cached) leaf node, not from the root node. Initial search space is restricted within small limit around the leaf node and the recursive depth-first tentative search is excluded. Therefore, the length of the node traversal path can be shortened. Second, we developed several techniques of what and how information is cached and reused. The indexing sequence of data can be good information, as well as the data itself. Generally speaking, data are stored in consecutive order. So the indexing data can be similar to (i) the previous indexing data , (ii) the same indexing data in another data set, and (iii) even the NNS result of the previous iteration in the Iterative Closest Point algorithm. Furthermore, we introduce a new method to apply the proposed search algorithm to even randomly sequenced data. We evaluated the algorithm with three kinds of data set (LIDAR, Kinect, and randomly generated data), and the results show that the proposed algorithm is about five times faster than the conventional approximated NNS algorithm using k-d tree.

This paper presents an algorithm, called the winner-update algorithm, for accelerating the nearest neighbor search. By constructing a hierarchical structure for each feature point in the l/sub p/ metric space, this algorithm can save a large amount of computation at the expense of moderate preprocessing and twice the memory storage. Given a query point, the cost for computing the distances from this point to all the sample points can be reduced by using a lower bound list of the distance established from Minkowski's inequality. Our experiments have shown that the proposed algorithm can save a large amount of computation, especially when the distance between the query point and its nearest neighbor is relatively small. With slight modification, the winner-update algorithm can also speed up the search for k nearest neighbors, neighbors within a specified distance threshold, and neighbors close to the nearest neighbor.

We suggest a simple modification to the Kd-tree search algorithm for nearest neighbor search resulting in an improved performance. The Kd-tree data structure seems to work well in finding nearest neighbors in low dimensions but its performance degrades even if the number of dimensions increases to more than two. Since the exact nearest neighbor search problem suffers from the curse of dimensionality we focus on approximate solutions; a c-approximate nearest neighbor is any neighbor within distance at most c times the distance to the nearest neighbor. We show that for a randomly constructed database of points if the query point is chosen close to one of the points in the data base, the traditional Kd-tree search algorithm has a very low probability of finding an approximate nearest neighbor; the probability of success drops exponentially in the number of dimensions d as e − Ω(d/c). However, a simple change to the search algorithm results in a much higher chance of success. Instead of searching for the query point in the Kd-tree we search for a random set of points in the neighborhood of the query point. It turns out that searching for e Ω(d/c) such points can find the c-approximate nearest neighbor with a much higher chance of success.KeywordsSuccess ProbabilityRandom PointNeighbor SearchComputational GeometryQuery PointThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We propose a new algorithm for fast approximate nearest neighbor search based on the properties of ordered vectors. Data vectors are classified based on the index and sign of their largest components, thereby partitioning the space in a number of cones centered in the origin. The query is itself classified, and the search starts from the selected cone and proceeds to neighboring ones. Overall, the proposed algorithm corresponds to locality sensitive hashing in the space of directions, with hashing based on the order of components. Thanks to the statistical features emerging through ordering, it deals very well with the challenging case of unstructured data, and is a valuable building block for more complex techniques dealing with structured data. Experiments on both simulated and real-world data prove the proposed algorithm to provide a state-of-the-art performance.

Nearest neighbor search algorithms have been successful in finding practically useful solutions to computationally difficult problems. In the nearest neighbor search problem, the brute force approach is often more efficient than other algorithms for high-dimensional spaces. A special case exists for objects represented as sparse vectors, where algorithms take advantage of the fact that an object has a zero value for most features. In general, since exact nearest neighbor search methods suffer from the “curse of dimensionality,” many practitioners use approximate nearest neighbor search algorithms when faced with high dimensionality or large datasets. To a reasonable degree, it is known that relying on approximate nearest neighbors leads to some error in the solutions to the underlying data mining problems the neighbors are used to solve. However, no one has attempted to quantify this error or provide practitioners with guidance in choosing appropriate search methods for their task. In this thesis, we conduct several experiments on recommender systems with a goal to find the degree to which approximate nearest neighbor algorithms are subject to these types of error propagation problems. Additionally, we provide persuasive evidence on the trade-off between search performance and analytics effectiveness. Our experimental evaluation demonstrates that a state-of-the-art approximate nearest neighbor search method (L2KNNGApprox) is not an effective solution in most cases. When tuned to achieve high search recall (80% or higher), it provides a fairly competitive recommendation performance compared to an efficient exact search method but offers no advantage in terms of efficiency (0.1x—1.5x speedup). Low search recall (<60%) leads to poor recommendation performance. Finally, medium recall values (60%—80%) lead to reasonable recommendation performance but are hard to achieve and offer only a modest gain in efficiency (1.5x—2.3x).

Modern datasets of importance such as images, videos, protein sequences or text, usually contain very high dimensional information from the search point of view. Nearest neighbor search is one of the most fundamental building blocks in dealing with large amounts of data. It is the problem of finding points in a database that are most similar to a query data point by some distance metric. There is a large body of work in algorithms for nearest-neighbor search on large highdimensional datasets. Since these algorithms invariably involve random access to data, most existing implementations ensure that the data is stored in DRAM, and does not spill into secondary storage such as hard disks. However, the immense size of modern datasets often requires hundreds of computers to accommodate the dataset in DRAM. An alternative to such a system is a much smaller cluster that stores the dataset in flash memory (instead of DRAM) and has in-store computing capability. In this paper, we build and demonstrate the performance of highdimensional nearest-neighbor search on a flash-based system with FPGA acceleration and show that it sometimes exceeds the performance of a DRAM-based solution. We chose two example applications, images and documents, for this demonstration. Since flash storage, in comparison to DRAM, is an order of magnitude cheaper and consumes an order of magnitude less power, a flashbased solution for nearest-neighbor searches offers a viable and attractive alternative.

Nearest neighbor search in high dimensional spaces is an interesting and important problem which is relevant for a wide variety of applications, including multimedia information retrieval, data mining, and pattern recognition. For such applications, the curse of high dimensionality tends to be a major obstacle in the development of efficient search methods. This paper addresses the problem of designing an efficient algorithm for high dimensional nearest neighbor search using a priority queue. The proposed algorithm is based on a simple linear search algorithm and eliminates unnecessary arithmetic operations from distance computations between multidimensional vectors. Moreover, we propose two techniques, a dimensional sorting method and a PCA-based method, to accelerate multidimensional search. Experimental results indicate that our scheme scales well even for a very large number of dimensions.

Speaker identification is one of the main tasks in speech processing. In addition to identification accuracy, large-scale applications of speaker identification give rise to another challenge: fast search in the database of speakers. In this paper, we propose a system based on i-vectors, a current approach for speaker identification, and locality sensitive hashing, an algorithm for fast nearest neighbor search in high dimensions. The connection between the two techniques is the cosine distance: on the one hand, we use the cosine distance to compare i-vectors, on the other hand, locality sensitive hashing allows us to quickly approximate the cosine distance in our retrieval procedure. We evaluate our approach on a realistic data set from YouTube with about 1,000 speakers. The results show that our algorithm is approximately one to two orders of magnitude faster than a linear search while maintaining the identification accuracy of an i-vector-based system.

K-d tree (k-dimensional tree) is a space partitioning data structure for organizing points in a k-dimensional space. K-d tree, or Multidimensional Binary Search Tree is a useful data structure for several applications such as searches involving a multidimensional search key (e.g., Range Search and Nearest Neighbour Search). K-d trees are a special case of binary space partitioning trees.KNN Search is a searching algorithm with complexity O(N log N) {N= no. of data points}. This search algorithm is relatively better than brute force search {Complexity= O(n*k); where k=No. of neighbours searched, N=No. of Data Points in Kd tree} for dimensions N&gt;&gt;2D {N=No. of Points, D=Dimensionality of Tree}.Furthermore, Parallel KNN Search is much more efficient and performs better than KNN Search, as it harnesses parallel processing capabilities of computers and thus, results in better search time.This paper tests the time performance of KNN Search and Parallel KNN Search and compares them by plotting it on a 3D graph. A more comprehensive comparison is done by use of 2D graphs for each dimension(from 2 to 20).

Dimensionality Reduction (DR) is a crucial tool to facilitate high-dimensional data analysis. As the volume and the variety of features used to describe a phenomenon keeps increasing, DR has become not only desirable but paramount. However, DR can result in unreliable depictions of data. The uncertainties involved in DR may stem from the selection of methods, parameter configurations, and the constraints imposed by the user. To address these uncertainties, various means of DR quality assessment have been proposed in the literature. Nevertheless, how to optimize the trade-off between the quantification efficiency and accuracy is yet to be further studied. The purpose of this paper is to present a general technique, in the context of visual analytics, to support efficient uncertainty-aware high-dimensional data exploration. We model the uncertainty based on how well neighborhood geometries are preserved during DR. We employ approximated nearest neighbor (ANN) search algorithms to speed up the quantification process with marginal decrease in accuracy. We then visualize the quantified uncertainties in the form of augmented scatter plot. We test our technique with three real world datasets against several well-known DR techniques, and discuss possible underlying causes that lead to certain embedding patterns. Our results show that our approach is effective and beneficial for both DR assessment and user-centered data exploration.

Nearest neighbor (NN) queries have been extended from Euclidean spaces to road networks. Existing approaches are either based on Dijkstra-like network expansion or NN/distance precomputation. The former may cause an explosive number of node accesses for sparse datasets because all nodes closer than the NN to the query must be visited. The latter, e.g., the Voronoi Network Nearest Neighbor (VN 3) approach, can handle sparse datasets but is inappropriate for medium and dense datasets due to its high precomputation and storage overhead. In this paper, we propose a new approach that indexes the network topology based on a novel network reduction technique. It simplifies the network by replacing the graph topology with a set of interconnected tree-based structures called SPIE’s. An nd index is developed for each SPIE and our new (k)NN search algorithms on an SPIE follow a predetermined tree path to avoid costly network expansion. By mathematical analysis and experimental results, our new approach is shown to be efficient and robust for various network topologies and data distributions.KeywordsRoad NetworkNear NeighborNode AccessShort Path TreeQuery NodeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Snapshot-based visual localization is an important problem in several computer vision and robotics applications such as Simultaneous Localization And Mapping (SLAM). To achieve real-time performance in very large-scale environments with massive amounts of training and map data, techniques such as approximate nearest neighbor search (ANN) algorithms are used. While several state-of-the-art variants of quantization and indexing techniques have demonstrated to be efficient in practice, their theoretical memory cost still scales at least linearly with the training data (i.e., O(n) where n is the number of instances in the database), since each data point must be associated with at least one code vector. To address these limitations, in this paper we present a totally new hierarchical encoding approach that enables a sub-linear storage scale. The algorithm exploits the widespread sequential nature of sensor information streams in robotics and autonomous vehicle applications and achieves, both theoretically and experimentally, sub-linear scalability in storage required for a given environment size. Furthermore, the associated query time of our algorithm is also of sub-linear complexity. We benchmark the performance of the proposed algorithm on several real-world benchmark datasets and experimentally validate the theoretical sub-linearity of our approach, while also showing that our approach yields competitive absolute storage performance as well.

At present, there are many methods to realize the matching of specified images with features, and the basic components include image feature point detection, feature description, and image matching. Based on this background, this article has done different research and exploration around these three aspects. The image feature point detection method is firstly studied, which commonly include image edge information-based feature detection method, corner information-based detection method, and various interest operators. However, all of the traditional detection methods are involved in problems of large computation burden and time consumption. In order to solve this problem, a feature detection method based on image grayscale information-FAST operator is used in this paper, which is combined with decision tree theory to effectively improve the speed of extracting image feature points. Then, the feature point description method BRIEF operator is studied, which is a local expression of detected image feature points based on descriptors. Since the descriptor does not have rotation invariance, the detection operator is endowed by a direction that is proposed in this paper, and then the local feature description is conducted on the feature descriptor to generate a binary string array containing direction information. Finally, the feature matching machine learning method is analyzed, and the nearest search method is used to find the nearest feature point pair in Euclidean distance, of which the calculation burden is small. The simulation results show that the proposed nearest neighbor search and matching machine learning algorithm has higher matching accuracy and faster calculation speed compared with the classical feature matching algorithm, which has great advantages in processing a large number of array images captured by the light field camera.

A simple, but efficient, nearest neighbor search algorithm is proposed and simulation results demonstrating its effectiveness in the case of vector quantization for a given source are presented. The simulation results indicate that use of this approach reduces the number of multiplications and additions to as low as 9 percent of those required for the conventional full search method. The reduction in the number of subtractions is also considerable. The increase in the number of comparisons is moderate, and therefore, the total number of operations can be as low as 28 percent of those required by the full search method. An additional advantage of the described algorithm is the fact that it requires no precomputations and/or extra memory.

Robotic end-effectors in automated production lines are specially designed and built for a unique task and for a particular part. An end-effector capable of grasping different parts with different geometries will expand its benefit and reduce costs. This work focuses on the grasp of sheet metal parts for the automotive industry. In order to maximize the use of a single end-effector, this paper proposes a search algorithm for a simple common grasp configuration. Such configuration imply for an end-effector design capable of grasping a set of sheet metal parts. The algorithm maps possible grasps which are candidate to be common. We define a novel quality measure estimating the distribution of the contacts across the sheet metal part. Candidate grasps which have a sufficient quality are mapped into high-dimensional feature vectors. These feature vectors parameterize the geometry of the polyhedron formed by the contacts locations and the direction of the surface normals relative to the polyhedron. Thereby, they describe the design of the end-effector compatible to the grasp. A database is generated for all possible grasps for each part in the feature vector space. A similarity join based on nearest-neighbor search and classification algorithm are used for intersecting all possible feature vectors over all sheet metal parts and finding common ones. Simulations of a 3-finger grasp on four meshed sheet metal parts resulted in a common grasp. Results of the simulations validate the feasibility of the proposed algorithm.