Shared Nearest Neighbor Clustering in a Locality Sensitive Hashing Framework.

Several clustering algorithms have been extensively used to analyze vast amounts of spatial data. One of these algorithms is the SNN (Shared Nearest Neighbor), a density-based algorithm, which has several advantages when analyzing this type of data due to its ability of identifying clusters of different shapes, sizes and densities, as well as the capability to deal with noise. Having into account that data are usually progressively collected as time passes, incremental clustering approaches are required when there is the need to update the clustering results as new data become available. This paper proposes SNN <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">++</sup> , an incremental clustering algorithm based on the SNN. Its performance and the quality of the resulting clusters are compared with the SNN and the results show that the SNN <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">++</sup> yields the same result as the SNN and show that the incremental feature was added to the SNN without any computational penalty. Moreover, the experimental results also show that processing huge amounts of data using increments considerably decreases the number of distances that need to be computed to identify the points' nearest neighbors.

High dimensional data clustering is a difficult task due to the curse of dimensionality. Traditional clustering methods usually fail to produce meaningful results for high dimensional data. Hypergraph partition is believed to be a promising method for dealing with this challenge. In this work, a new high dimensional clustering method called Merging Dense SubGraphs (MDSG) is proposed. A graph G is first constructed from the data by defining an adjacency relationship between the data points using Shared k Nearest Neighbors (SNN). Then a hypergraph is created from the graph G by defining the hyperedges to be all the maximal cliques in the graph. After the hypergraph is produced, an improved hypergraph partitioning method is used to produce the final clustering results. The proposed MDSG method is evaluated on several real high dimensional handwritten datasets, and the experimental results show that the proposed method is superior to the traditional clustering method and other hypergraph partition methods for high dimensional handwritten data clustering.

This paper proposes an outlier robust geodesic K-mean algorithm for high dimensional data. The proposed algorithm features three novel contributions. First, it employs a shared nearest neighbour (SNN) based distance metric to construct the nearest neighbour data model. Second, it combines the notion of geodesic distance to the well-known local outlier factor (LOF) model to distinguish outliers from inlier data. Third, it introduces a new ad-hoc strategy to integrate outlier scores into geodesic distances. Numerical experiments with synthetic and real world remote sensing spectral data show the efficiency of the proposed algorithm in clustering of high-dimensional data in terms of the overall clustering accuracy and the average precision.

In recent year, stock investor in Indonesia increased rapidly, so it is required to do analysis about the stock that helps the investor in their investment plan. Clustering is beneficial to select the appropriate stock for investors. Unfortunately, stock prices keep varying from time to time. Consequently, it is not an easy work to select the stock for investment. In addition, stock price time series data are high dimensional data that influenced by many factors. In this study, high dimensional data are obtained by the time frame of each factor. Therefore, it is important to use a suitable technique to cluster high dimensional data. This paper presents High Dimensional Data Clustering (HDDC), a model-based clustering based on Gaussian Mixture Model, using the Expectation-Maximization (EM) algorithm. HDDC via EM algorithm gives a more robust result, and it possible to make an additional assumption. Moreover, this paper combines a high-dimensional clustering technique HDDC via EM algorithm and the most popular feature extraction technique Principal Component Analysis (PCA). This paper comparing methods of clustering technique HDDC and the combination between HDDC and PCA to know the most effective method which gives better result in clustering high-dimensional time series data. The 155 data features are reduced to 7 principal components using PCA analysis. Despite PCA has increased the time efficiency of building the model, clustering technique HDDC via EM algorithm enables to handle the high-dimensional data better than the combination with PCA.

In this paper, we propose a new incremental learning algorithm of radial basis function (RBF) Network to accelerate the learning for large-scale data sequence. Along with the development of the internet and sensor technologies, a time series of large data chunk are continuously generated in our daily life. Thus it is usually difficult to learn all the data within a short period. A remedy for this is to select only essential data from a given data chunk and provide them to a classifier model to learn. In the proposed method, only data in untrained regions, which correspond to a region with a low output margin, are selected. The regions are formed by grouping the data based on their near neighbor using locality sensitive hashing (LSH), in which LSH has been developed to search neighbors quickly in an approximated way. As the proposed method does not use all training data to calculate the output margins, the time of the data selection is expected to be shortened. In the incremental learning phase, in order to suppress catastrophic forgetting, we also exploit LSH to select neighbor RBF units quickly. In addition, we propose a method to update the hash table in LSH so that the data selection can be adaptive during the learning. From the performance of nine datasets, we confirm that the proposed method can learn large-scale data sequences fast without sacrificing the classification accuracies. This fact implies that the data selection and the incremental learning work effectively in the proposed method.

The recognition and extraction of biomedical names is an essential task for the biomedical information extraction. However, the preparation of large annotated corpora hinders the training of the Named Entity Recognition (NER) systems. Active learning is reducing the needed manual annotation work in supervised learning task. In this work, we propose a novel clustering based active learning method for the biomedical NER task. We show that the underlying NER system using the proposed method outperforms those with other state of the art active learning methods, including density, Gibbs error and entropy based approaches, as well as the random selection. We compare variations of our proposed method and find the optimal design of the active learning method, which is to use the vector representation of named entities, and to select documents that are ‘representative’ and ‘informative’, as well as to use the Shared Nearest Neighbor (SNN) clustering approach. In particular, the optimal variant of the proposed method achieves a deficiency gain of 36.3% over the random selection.

High-dimensional data arise naturally in many domains, and have regularly presented a great challenge for traditional data mining techniques, both in terms of effectiveness and efficiency. Clustering becomes difficult due to the increasing sparsity of such data, as well as the increasing difficulty in distinguishing distances between data points. In this paper, we take a novel perspective on the problem of clustering high-dimensional data. Instead of attempting to avoid the curse of dimensionality by observing a lower dimensional feature subspace, we embrace dimensionality by taking advantage of inherently high-dimensional phenomena. More specifically, we show that hubness, i.e., the tendency of high-dimensional data to contain points (hubs) that frequently occur in k-nearest-neighbor lists of other points, can be successfully exploited in clustering. We validate our hypothesis by demonstrating that hubness is a good measure of point centrality within a high-dimensional data cluster, and by proposing several hubness-based clustering algorithms, showing that major hubs can be used effectively as cluster prototypes or as guides during the search for centroid-based cluster configurations. Experimental results demonstrate good performance of our algorithms in multiple settings, particularly in the presence of large quantities of noise. The proposed methods are tailored mostly for detecting approximately hyperspherical clusters and need to be extended to properly handle clusters of arbitrary shapes.

Spatial-temporal clustering is very useful unsupervised learning technique and can be used to identify interesting distribution patterns from geo-located data. It is one of the most commonly used data mining techniques in many application domains, e.g. geographic information science, health science, and environmental science. In this paper, we propose a density-based spatial-temporal clustering algorithm for geo-located data points, based on an extension of the SNN (Shared Nearest Neighbor) clustering. The proposed algorithm allows the integration of location, time and other semantic attributes in the clustering process. This algorithm can find clusters of different sizes, shapes, and densities in noisy data. We evaluate the effectiveness of our algorithm through a case study involving a New York City taxi cab pickup data and Maryland crime data. The experimental results show that the proposed algorithm can discover interesting patterns and useful information from spatial-temporal data.

The analysis of time-series data has been actively studied in various fields, such as biology and economics. Clustering is a method that summarizes a set of objects into several subsets of objects based on similarity measures. It is necessary to define a suitable similarity between objects. When dealing with time-series data, it is also necessary to consider several invariances, including shift-invariance. $k$ -Shape clustering is one of the representative clustering methods for time-series data. It is known that the $k$ -Shape clustering is an algorithm, which considers several invariances of time-series data. The dissimilarity used in $k$ -Shape clustering is robust to differences in time series data features. In this paper, the controlled-sized $k$ -Shape clustering is proposed to handle imbalanced data. Numerical experiments suggest that the proposed method does not show outstanding performance compared to $k$ -Shape clustering.

Since the label collecting is prohibitive and time-consuming, unsupervised methods are preferred in applications such as fraud detection. Meanwhile, such applications usually require modeling the intrinsic clusters in high-dimensional data, which usually displays heterogeneous statistical patterns as the patterns of different clusters may appear in different dimensions. Existing methods propose to model the data clusters on selected dimensions, yet globally omitting any dimension may damage the pattern of certain clusters. To address the above issues, we propose a novel unsupervised generative framework called FIRD, which utilizes adversarial distributions to fit and disentangle the heterogeneous statistical patterns. When applying to discrete spaces, FIRD effectively distinguishes the synchronized fraudsters from normal users. Besides, FIRD also provides superior performance on anomaly detection datasets compared with SOTA anomaly detection methods (over 5% average AUC improvement). The significant experiment results on various datasets verify that the proposed method can better model the heterogeneous statistical patterns in high-dimensional data and benefit downstream applications.

In this paper, a new method for content-based video copy detection is presented. This method includes fingerprint extracting and matching phases. In the fingerprint extracting phase, a video is represented by a set of Speeded Up Robust Features (SURF), which outperforms other local features. In the fingerprint matching phase, the Locality Sensitive Hashing (LSH) is applied to efficiently detect video copy. The experimental results show that the proposed method has excellent performance in accuracy and speed.

Dimension reduction has become an important tool for dealing with high dimensional data. Locally linear embedding (LLE) is a nonlinear dimension reduction method which can preserve local configurations of nearest neighbors. However, finding the nearest neighbors requires the definition of a distance measure, which is a critical step in LLE. In this paper, the Rank-order distance measure is used to substitute the traditional Euclidean distance measure in order to find better nearest neighbor candidates for preserving local configurations of the manifolds. The Rank-order distance between the data points is calculated using their neighbors’ ranking orders, and is shown to be able to improve the clustering of high dimensional data. The proposed method is called Rank-order based LLE (RLLE). The RLLE method is evaluated by comparing with the original LLE, ISO-LLE and IED-LLE on two handwritten datasets. It is shown that the effectiveness of a distance measure in the LLE method is closely related to whether it can be used to find good nearest neighbors. The experimental results show that the proposed RLLE method can improve the process of dimension reduction effectively, and C-index is another good candidate for evaluating the dimension reduction results.

Image matching is a classic technique in computer vision. However, the traditional local invariant features image matching algorithm has two problems, narrow scale range and long time consuming. Aiming at these problems, we proposed a fast image matching algorithm with the aid of improved local invariant features based on Locality-Sensitive Hashing. Firstly, by building simple Gaussian pyramid and achieving FAST keypoint detection, keypoints are extracted from the reference image and the candidate matching image. Then Fast Retina Keypoint feature descriptor is calculated and weighted. Furthermore, the high-dimensional data is mapped to a low dimensional space and hash indexes are built through the local sensitive hashing algorithm in aiming of finding the approximate nearest neighbor. The experimental results in different datasets indicate that the improved algorithm achieves real-time processing in image matching, and has better robustness and shorter processing time than most classical methods.

Locality Sensitive Hashing (LSH) has been popularly used in content-based search systems. There exist two main categories of LSH methods: one is to index the original data in an effective way to accelerate search process; the other one is to embed the high-dimensional data into hamming space and perform bit-wise operations to search similar objects. In this paper, we propose a new LSH scheme, called Distribution-Aware LSH (DALSH), to address the problem of lacking adaptation to real data, which is the intrinsic limitation of most LSH methods belong to the former category. In DALSH, a given dataset is embedded into a low-dimensional space with projection vectors learned from data, followed by deriving hash functions from the distribution of the dimension-reduced data. We also present a multi-probe strategy to improve the query performance. Experimental comparisons with the state-of-the-art LSH methods on two high-dimensional datasets demonstrate the efficacy of DALSH.

Hashing learning has attracted increasing attention these years with the rapid increase of data. Some high-dimensional data have caused the ‘dimension disaster’ which make traditional methods ineffective. In this paper, we propose a method to find the nearest neighbor quickly from the high-dimensional data, named pseudo-inverse locality preserving iterative hashing(PLIH). We use pseudo-inverse to replace the inverse matrix in order to solve the problem of matrix singularity. We construct adjacency graphs and minimize the distance of the neighbors in the subspace to make the projected matrix maintain the neighborhood relations of high dimension, which solves the problem that the locality sensitive hashing cannot preserve the high-dimensional neighborhood relations effectively. Because different bit with different weight has more discriminating power than the same weight, Loss of the projection matrix in the quantization process is minimized by weighted iterative quantization. Experiments on public datasets Cnn_4096d_Caltech and Gist_512d_Caltech demonstrated that accuracy and recall of the PLIPH are both better than the traditional hashing algorithms.