Reliable, efficient and light distance computation on high-dimensional vectors

Nearest neighbor search is a fundamental problem in computer vision and machine learning. The straightforward solution, linear scan, is both computationally and memory intensive in large scale high-dimensional cases, hence is not preferable in practice. Therefore, there have been a lot of interests in algorithms that perform approximate nearest neighbor (ANN) search. In this paper, we propose a novel addition-based vector quantization algorithm, Asymmetric Mapping Quantization (AMQ), to efficiently conduct ANN search. Unlike existing addition-based quantization methods that suffer from handling the problem caused by the norm of database vector, we map the query vector and database vector using different mapping functions to transform the computation of L-2 distance to inner product similarity, thus do not need to evaluate the norm of database vector. Moreover, we further propose Distributed Asymmetric Mapping Quantization (DAMQ) to enable AMQ to work on very large dataset by distributed learning. Extensive experiments on approximate nearest neighbor search and image retrieval validate the merits of the proposed AMQ and DAMQ.

Nearest neighbor search algorithms have been successful in finding practically useful solutions to computationally difficult problems. In the nearest neighbor search problem, the brute force approach is often more efficient than other algorithms for high-dimensional spaces. A special case exists for objects represented as sparse vectors, where algorithms take advantage of the fact that an object has a zero value for most features. In general, since exact nearest neighbor search methods suffer from the “curse of dimensionality,” many practitioners use approximate nearest neighbor search algorithms when faced with high dimensionality or large datasets. To a reasonable degree, it is known that relying on approximate nearest neighbors leads to some error in the solutions to the underlying data mining problems the neighbors are used to solve. However, no one has attempted to quantify this error or provide practitioners with guidance in choosing appropriate search methods for their task. In this thesis, we conduct several experiments on recommender systems with a goal to find the degree to which approximate nearest neighbor algorithms are subject to these types of error propagation problems. Additionally, we provide persuasive evidence on the trade-off between search performance and analytics effectiveness. Our experimental evaluation demonstrates that a state-of-the-art approximate nearest neighbor search method (L2KNNGApprox) is not an effective solution in most cases. When tuned to achieve high search recall (80% or higher), it provides a fairly competitive recommendation performance compared to an efficient exact search method but offers no advantage in terms of efficiency (0.1x—1.5x speedup). Low search recall (<60%) leads to poor recommendation performance. Finally, medium recall values (60%—80%) lead to reasonable recommendation performance but are hard to achieve and offer only a modest gain in efficiency (1.5x—2.3x).

Nearest Neighbor search is one of the simplest and most intuitive ideas in data mining. Due to it's simplicity and diverse utility, Nearest Neighbor search is often found to be the workhorse of a variety of data mining, machine learning, and computer vision algorithms. For very high dimensional data, the naive linear search tends to be optimal. This is due to the so called curse of dimensionality in database search. To accelerate the search time, may researchers have turned to the power and efficiency of GPGPU computing; mainly for computing the data intensive distance metrics. Though promising speedups are achieved through this method, it fails to capitalize on many of the recent advances in algorithms for bounded error approximate nearest neighbor searches. Approximate nearest neighbor methods give satisfactory results, especially in very large and possibly redundant datasets, while boasting sub-linear search complexities. In this paper we present a c-approximate r-nearest neighbor search algorithm for CUDA using Locality Sensitive Hash with Nearest Neighbor search (LSH-NN). We implement this system in CUDA and test it on real world image SIFT vector data. Our tests show that we are able to achieve significant speedup over the serial version, with good approximations on scalability, while achieving the near-optimal search complexity of LSH-NN.

Nearest neighbor search, a classic way of identifying similar data, can be applied to various areas, including database, machine learning, natural language processing, software engineering, etc. Secure nearest neighbor search aims to find nearest neighbors to a given query point over encrypted data without accessing data in plaintext. It provides privacy protection to datasets when nearest neighbor queries need to be operated by an untrusted party (e.g., a public server). While different solutions have been proposed to support nearest neighbor queries on encrypted data, these existing solutions still encounter critical drawbacks either in efficiency or privacy. In light of the limitations in the current literature, we propose a novel approximate nearest neighbor search solution, referred to as Sparser, by leveraging a combination of space-filling curves, perturbation, and Order-Preserving Encryption. The advantages of Sparser are twofold, strengthening privacy and improving efficiency. Specifically, Sparser pre-processes plaintext data with space-filling curves and perturbation, such that data is sparse, which mitigates leakage abuse attacks and renders stronger privacy. In addition to privacy enhancement, Sparser can efficiently find approximate nearest neighbors over encrypted data with logarithmic time. Through extensive experiments over real-world datasets, we demonstrate that Sparser can achieve strong privacy protection under leakage abuse attacks and minimize search time.

Developed from Nearest Neighbor (NN) queries, Aggregate Nearest Neighbor (ANN) queries return the object that minimizes an aggregate distance function with respect to a set of query points. Because of the multiple query points, ANN queries are much more complex than NN queries. For optimizing the query processing and improving the query efficiency, many ANN queries algorithms utilizes pruning strategies, with or without an index structure. Obviously, the pruning effect highly depends on the tightness of the bound estimation. In this paper, we figure out a property in vector space and develop some efficient bound estimations for two most popular types of ANN queries. Based on these bounds, we design the indexed and non-index ANN algorithms, and conduct experimental studies. Our algorithms show good performance, especially for high dimensional queries, for both real dataset and synthetic datasets.

In this paper, we study the problem of continuous monitoring of reverse k nearest neighbors queries in Euclidean space as well as in spatial networks. Existing techniques are sensitive toward objects and queries movement. For example, the results of a query are to be recomputed whenever the query changes its location. We present a framework for continuous reverse k nearest neighbor (RkNN) queries by assigning each object and query with a safe region such that the expensive recomputation is not required as long as the query and objects remain in their respective safe regions. This significantly improves the computation cost. As a byproduct, our framework also reduces the communication cost in client---server architectures because an object does not report its location to the server unless it leaves its safe region or the server sends a location update request. We also conduct a rigid cost analysis for our Euclidean space RkNN algorithm. We show that our techniques can also be applied to answer bichromatic RkNN queries in Euclidean space as well as in spatial networks. Furthermore, we show that our techniques can be extended for the spatial networks that are represented by directed graphs. The extensive experiments demonstrate that our techniques outperform the existing techniques by an order of magnitude in terms of computation cost and communication cost.

Many large multimedia applications require efficient processing of nearest neighbor queries. Often, multimedia data are represented as a collection of important high-dimensional feature vectors. Existing Locality Sensitive Hashing (LSH) techniques require users to find top-k similar feature vectors for each of the feature vectors that represent the query object. This leads to wasted and redundant work due to two main reasons: 1) not all feature vectors may contribute equally in finding the top-k similar multimedia objects, and 2) feature vectors are treated independently during query processing. Additionally, there is no theoretical guarantee on the returned multimedia results. In this work, we propose a practical and efficient indexing approach for finding top-k approximate nearest neighbors for multimedia data using LSH called mmLSH, which can provide theoretical guarantees on the returned multimedia results. Additionally, we present a buffer-conscious strategy to speed up the query processing. Experimental evaluation shows significant gains in performance time and accuracy for different real multimedia datasets when compared against state-of-the-art LSH techniques.

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2020 ACM-SIAM Symposium on Discrete Algorithms (SODA)Faster Deterministic and Las Vegas Algorithms for Offline Approximate Nearest Neighbors in High DimensionsJosh Alman, Timothy M. Chan, and Ryan WilliamsJosh Alman, Timothy M. Chan, and Ryan Williamspp.637 - 649Chapter DOI:https://doi.org/10.1137/1.9781611975994.39PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract We present a deterministic, truly subquadratic algorithm for offline (1 + ε)-approximate nearest or farthest neighbor search (in particular, the closest pair or diameter problem) in Hamming space in any dimension d ≤ nδ, for a sufficiently small constant δ > 0. The running time of the algorithm is roughly for nearest neighbors, or for farthest. The algorithm follows from a simple combination of expander walks, Chebyshev polynomials, and rectangular matrix multiplication. We also show how to eliminate errors in the previous Monte Carlo randomized algorithm of Alman, Chan, and Williams [FOCS’16] for offline approximate nearest or farthest neighbors, and obtain a Las Vegas randomized algorithm with expected running time . Finally, we note a simplification of Alman, Chan, and Williams' method and obtain a slightly improved Monte Carlo randomized algorithm with running time . As one application, we obtain improved deterministic and randomized (1 + ε)-approximation algorithms for MAX-SAT. Previous chapter Next chapter RelatedDetails Published:2020eISBN:978-1-61197-599-4 https://doi.org/10.1137/1.9781611975994Book Series Name:ProceedingsBook Code:PRDA20Book Pages:xxii + 3011

Case Based Reasoning (CBR) is a method that aims to resolve a new case by adapting the solutions contained in previous cases that are similar to the new case. The system built in this study is the CBR system to make recommendations on the topic of student thesis concentration. This study used data from undergraduate students of Informatics Engineering IST AKPRIND Yogyakarta with a total of 115 data consisting of 80 training data and 35 test data. This study aims to design and build a Case Based Reasoning system using the Nearest Neighbor and Manhattan Distance Similarity Methods, and to compare the results of the accuracy value using the Nearest Neighbor Similarity and Manhattan Distance Similarity methods. The recommendation process is carried out by calculating the value of closeness or similarity between new cases and old cases stored on a case basis using the Nearest Neighbor Method and Manhattan Distance. The features used in this study consisted of GPA and course grades. The case taken is the case with the highest similarity value. If a case doesnt get a topic recommendation or is less than the trashold value of 0.8, a case revision will be carried out by an expert. Successfully revised cases are stored in the system to be made new knowledge. The test results using the Nearest Neighbor Method get an accuracy value of 97.14% and Manhattan Distance Method 94.29%.

Uncertain data pervades many fields, including environmental monitoring, the monitoring of animal migrations, and urban warfare. Such uncertain data collected by field devices, such as Internet of Things (IoT) and Internet of Battlefield Things (IoBT) devices, may also be encrypted and outsourced to an untrustworthy third party for storage and data sharing such as a cloud server. However, the properties of uncertain data and the complication of operating over encrypted data make the searching schemes more ineffective. In this article, we design an efficient and safe K nearest neighbor (KNN) query scheme for uncertain data stored in semi-trusted cloud servers. We apply the modified homomorphic encryption, which requires two servers to interact and encrypt the uncertain data, and we use the authorized rank method to compute KNN. We protect the security of the data while simultaneously improving the query efficiency. Our detailed security analysis show that our scheme can realize the goal of concealing both the access and the search patterns. Comprehensive experiments are conducted to demonstrate the scheme's performance.

The k nearest neighbor (KNN) query, an essential query for information processing in sensor networks, has not received sufficient attention in the research community of sensor networks. In this paper, we examine in-network processing of KNN queries by proposing two alternative algorithms, namely the GeoRouting Tree (GRT) and the KNN Boundary Tree (KBT). The former is based on a distributed spatial index structure and prunes off the irrelevant nodes during query propagation. The latter is based upon ad-hoc geographic routing and first obtains a region within which at least k nearest sensor nodes are enclosed and then decides the k nearest nodes to the query point. We provide an extensive performance evaluation to study the impact of various system factors and protocol parameters. Our results show that GRT yields a good tradeoff between energy consumption and query accuracy in static scenarios. On the other hand, KBT achieves better energy efficiency while being more tolerant to network dynamics.

Aggregate k nearest neighbor (AkNN) queries are useful in many areas, such as multimedia retrieval and resource allocation, to name but a few. Most of existing works on AkNN query only focus on Euclidean space or specific metric space, which employ properties of particular data to accelerate the query. However, due to the complex data types involved and the needs for flexible similarity criteria seen in real applications, properties of particular data cannot be used for general case. Hence, in this paper, we investigate AkNN search in metric spaces, termed as metric AkNN (MAkNN) search, as metric spaces can support any type of data and flexible similarity criteria as long as satisfying triangle inequality. To efficiently answer MAkNN queries, we develop several pruning techniques and corresponding algorithms based on SPB-tree. Extensive experiments using three real data sets verify the efficiency of our MAkNN algorithms.

Due to the explosive increase of data in both the aspects of dimensionality and volume, performing $k$ nearest neighbors search over cloud environments has been progressively receiving more attention among researchers in the field of database cloud computing. However, the key challenge for switching $k$ nearest neighbors search from the local server (i.e., traditional way) to the third-party cloud is, that the database which always contains series of sensitive information has to be kept secret against the cloud. In this work, we present a pair of solutions towards Secure $k$ Nearest Neighbors (S $k$ NN) query in outsourced environments. By skillfully utilizing coarse quantization and the cryptography techniques Advanced Encryption Standard (AES) and Paillier homomorphic encryption, we construct a secure Inverted File (IVF) and compute encrypted approximate distances directly to search for high-dimensional data in the third-party cloud provider, and finally find the better tradeoff between the search quality and security. Empirical study over real datasets and practical environments validate our solutions’ feasibility, completeness, and practicality. Compared to the state-of-the-art, the proposed solutions resolve the S $k$ NN of high-dimensional data novelly, have very limited response time and provide high privacy protection on the side of both the User and the cloud provider.

Broadcasting data with an index is an effective way to disseminate public information to a large clients. For a server, using multiple channels to provide services (e.g., location-based services) makes the broadcast cycle shorter than using one channel. Among location-based services, the k nearest neighbors (k-NN) search is an important one and finds the fc closest objects to a query point in a multi-dimensional space. This paper considers k nearest neighbors search on a broadcast R-tree in a multi-channel environment. We assume that a mobile client can only tune into a specified channel at one time instance. We study how a server generates the broadcast schedules on multiple channels and explore how a client executes the k-NN search on the broadcast. Different broadcast schedules with the client k-NN search processing makes different k-NN search protocols. The objectives of the protocols is to minimize the latency (i.e., the time elapsed between issuing and termination of the query), tuning time (i.e., the amount of time spent on listening to the channel), and the memory usage for k-NN search processing. Last, we present our experiments and the experiment results validate that our mechanisms achieve the objectives.

Nowadays, many applications return to the user a set of results that take the query as their nearest neighbor, which are commonly expressed through reverse nearest neighbor (RNN) queries. When considering moving objects, users would like to find objects that appear in the RNN result set for a period of time in some real-world applications such as collaboration recommendation and anti-tracking. In this work, we formally define the problem of interval reverse nearest neighbor (IRNN) queries over moving objects, which return the objects that maintain nearest neighboring relations to the moving query objects for the longest time in the given interval. Location uncertainty of moving data objects and moving query objects is inherent in various domains, and we investigate objects that exhibit Markov correlations, that is, each object's location is only correlated with its own location at previous timestamp while being independent of other objects. There exists the efficiency challenge for answering IRNN queries on uncertain moving objects with Markov correlations since we have to retrieve not only all the possible locations of each object at current time but also its historically possible locations. To speed up the query processing, we present a general framework for answering IRNN queries on uncertain moving objects with Markov correlations in two phases. In the first phase, we apply space pruning and probability pruning techniques, which reduce the search space significantly. In the second phase, we verify whether each unpruned object is an IRNN of the query object. During this phase, we propose an approach termed Probability Decomposition Verification (PDV) algorithm which avoid computing the probability of any object being an RNN of the query object exactly and thus improve the efficiency of verification. The performance of the proposed algorithm is demonstrated by extensive experiments on synthetic and real datasets, and the experimental results show that our algorithm is more efficient than the Monte-Carlo based approximate algorithm.