Pairwised Specific Distance Learning from Physical Linkages

Data mining is a collection of methods used to extract useful information from large data bases. Cluster Analysis refers to the grouping of a set of data points into clusters. Most widely used partitioning methods are K-means and Fuzzy c-means (FCM) algorithms. However, they suffer from the difficulties such as random selection of initial centre values and handling outlier data points. Most of the existing clustering methods use the Euclidean distance metric. The modified fuzzy c-means algorithm (MFCM) is efficient in handling outlier data points. In this paper, a new hybrid algorithm is proposed to solve the limitations of the traditional clustering methods. The hybrid K-MFCM algorithm is tested on four real world bench mark data sets from UCI machine learning repository with various distance metrics including Euclidean, City Block and Chessboard. The cluster centroid values of hybrid algorithm are calculated for various data sets. The experimental results show that the hybrid algorithm gives good results in terms of objective function value and better fuzzy cluster validity results for chessboard distance metric than other distance metrics.

This paper focuses on semi-supervised clustering, where the goal is to cluster a set of data-points given a set of similar/dissimilar examples. These examples provide instance-level equivalence/in-equivalence constraints (e.g., similar pairs belong to the same cluster while dissimilar pairs belong to different clusters), but in order to aid final clustering we must propagate them to feature-space level constraints (i.e., how similar are two regions in the feature space?). An increasingly popular approach to accomplish this is by learning distance metrics over the feature space that are guided by the instance-level constraints. Inspired by the success of recent bag-of-words models, we utilize codewords (or visual-words) as building blocks. Our proposed technique learns non-parametric distance metrics over codewords from these equivalence (and optionally, in-equivalence) constraints, which we are then able to propagate back to compute a dissimilarity measure between any two points in the feature space. There are two significant advances over previous work. First, unlike past efforts on global distance metric learning which try to transform the entire feature space so that similar pairs are close, we transform modes in data distribution or pockets of the feature space. This transformation is non-parametric and thus allows arbitrary non-linear deformations of the feature space. Second, while most Mahalanobis metrics are learnt using Semi-Definite Programming (SDP), our proposed solution is developed as a Linear Program (LP) and in practice, is extremely fast. Finally, we provide quantitative analysis on image datasets (MSRC, Corel) where ground-truth segmentation is available, and show that our learnt metrics can significantly improve clustering accuracy.

Learning a proper distance metric is of vital importance for many distance based applications. Distance metric learning aims to learn a set of latent factors based on which the distances between data points can be effectively measured. The number of latent factors incurs a tradeoff: a small amount of factors are not powerful and expressive enough to measure distances while a large number of factors cause high computational overhead. In this paper, we aim to achieve two seemingly conflicting goals: keeping the number of latent factors to be small for the sake of computational efficiency, meanwhile making them as effective as a large set of factors. The approach we take is to impose a diversity regularizer over the latent factors to encourage them to be uncorrelated, such that each factor can capture some unique information that is hard to be captured by other factors. In this way, a small amount of latent factors can be sufficient to capture a large proportion of information, which retains computational efficiency while preserving the effectiveness in measuring distances. Experiments on retrieval, clustering and classification demonstrate that a small amount of factors learned with diversity regularization can achieve comparable or even better performance compared with a large factor set learned without regularization.

A novel travel-time based similarity measure for hierarchical clustering

This paper presents a comparative evaluation of different distance metrics and local planners within the context of probabilistic roadmap methods for planning the motion of rigid objects in three-dimensional workspaces. The study concentrates on cluttered three-dimensional workspaces typical of, for example, virtual prototyping applications such as maintainability studies in mechanical CAD designs. Our results include recommendations for selecting appropriate combinations of distance metrics and local planners for such applications. Our study of distance metrics shows that the importance of the translational distance increases relative to the rotational distance as the environment becomes more crowded. We find that each local planner makes some connections that none of the others does-indicating that better connected roadmaps will be constructed using multiple local planners. We propose a new local planning method we call rotate-at-s that often outperforms the common straight-line in C-space method in crowded environments.

Polymer-electrolyte aqueous two-phase systems (ATPS) have demonstrated their superior performance in the separation and purification of high-value biomolecules. However, these powerful platforms are still a major academic curiosity, without their acceptance and implementation by industry. One of the major obstacles is the absence of models to predict the partition of biomolecules in ATPS in an easy and predictive way. To address this limitation, modelling studies on the binodal curve behavior of polymer-electrolyte ATPS and the partitioning of biomolecules in these aqueous electrolyte solutions are carried out in this work. First, a comprehensive database targeting the studied systems is established. In total, 11,998 experimental binodal data points covering 276 polymer-electrolyte ATPS at different temperatures (273.15 K-399.15 K) and 626 experimental partition data points involving 22 biomolecules in 42 polymer-electrolyte ATPS at different temperatures (283.15 K-333.15 K) are included. Then, a novel modeling strategy that combines a well-known machine learning algorithm, i.e., artificial neural network (ANN) and group contribution (GC) method is proposed. Based on this modeling strategy, an ANN-GC model (ANN-GC model1) is built to describe the binodal curve behavior of polymer-electrolyte ATPS, while another ANN-GC model (ANN-GC model2) is developed to predict the partition of biomolecules in these biphasic systems. ANN-GC model1 gives a mean absolute error (MAE) of 0.0132 and squared correlation coefficient (R2) of 0.9878 for the 9,598 training data points, and for the 1,200 validation data points they are 0.0141 and 0.9858, respectively. Meanwhile, it also gives a MAE of 0.0143 and R2 of 0.9846 for the 1,200 test data points. On the other hand, ANN-GC model2 gives root-mean-square deviation (RMSD) of 0.0577 for 501 training data points, and for the 62 validation data points and 63 test data points their RMSD are 0.0849 and 0.0885, respectively. Furthermore, the obtained results also indicate that the tie-line length of polymer-electrolyte ATPS calculated from ANN-GC model1 can be directly used in ANN-GC model2 for predicting the partition performance coefficient of biomolecules in these ATPS. The developed models offer the possibility to predict the partition of biomolecules in ATPS without any requirement of experimental data. Based on the developed ANN-GC models, some high-performance ATPS are identified to partition four well-known biomolecules.

Similarity search plays an important role in many applications involving high-dimensional data. Due to the known dimensionality curse, the performance of most existing indexing structures degrades quickly as the feature dimensionality increases. Hashing methods, such as locality sensitive hashing (LSH) and its variants, have been widely used to achieve fast approximate similarity search by trading search quality for efficiency. However, most existing hashing methods make use of randomized algorithms to generate hash codes without considering the specific structural information in the data. In this paper, we propose a novel hashing method, namely, robust hashing with local models (RHLM), which learns a set of robust hash functions to map the high-dimensional data points into binary hash codes by effectively utilizing local structural information. In RHLM, for each individual data point in the training dataset, a local hashing model is learned and used to predict the hash codes of its neighboring data points. The local models from all the data points are globally aligned so that an optimal hash code can be assigned to each data point. After obtaining the hash codes of all the training data points, we design a robust method by employing l2,1 -norm minimization on the loss function to learn effective hash functions, which are then used to map each database point into its hash code. Given a query data point, the search process first maps it into the query hash code by the hash functions and then explores the buckets, which have similar hash codes to the query hash code. Extensive experimental results conducted on real-life datasets show that the proposed RHLM outperforms the state-of-the-art methods in terms of search quality and efficiency.

Nonlinear data projection based on geodesic distances requires the construction of a neighborhood graph that spans all data points so that the geodesic distance between any pair of data points could be estimated by the graph distance between the pair. This paper proposes an approach for constructing a k-edge connected neighborhood graph. The approach works by repeatedly extracting minimum spanning trees from the complete Euclidean graph of all data points. The constructed neighborhood graph has the following properties: (1) it is k-connected; (2) each point connects to its k-nearest neighbors; (3) if the graph is cut into two partitions, the cut edges contain k-shortest edges between the two partitions. Experiments show that the presented approach works well for clustered data and outperforms the nearest neighbor approaches used in Isomap for evenly distributed data.

A self-adaptive local metric learning method for classification

Anomaly Detection Using Adaptive Suppression

Metric Learning, which aims at learning a distance metric for a given data set, plays an important role in measuring the distance or similarity between data objects. Due to its broad usefulness, it has attracted a lot of interest in machine learning and related areas in the past few decades. This paper proposes to learn the distance metric from the side information in the forms of must-links and cannot-links. Given the pairwise constraints, our goal is to learn a Mahalanobis distance that minimizes the ratio of the distances of the data pairs in the must-links to those in the cannot-links. Different from many existing papers that use the traditional squared L2-norm distance, we develop a robust model that is less sensitive to data noise or outliers by using the not-squared L2-norm distance. In our objective, the orthonormal constraint is enforced to avoid degenerate solutions. To solve our objective, we have derived an efficient iterative solution algorithm. We have conducted extensive experiments, which demonstrated the superiority of our method over state-of-the-art.

We study classifiers based on prototypes, which classify data points by the nearest neighbor rule, namely a data point is labeled the class which has its nearest prototype. The prototype-based classifiers are trained by the max-margin principle, which is used by support vector machines. For each training data point, we consider the difference of the distances between the nearest prototype of the class to which the point belongs and the nearest one of the other classes. And then, we define a margin by the minimum value of these differences for all the training data points. We verify the efficiency of the obtained classifier by numerical experiments.

Image processing and machine learning communities have long addressed the problems involved in the analysis of large high-dimensional data sets. To deal with high-dimensional data efficiently, learning core properties of given data set is important. The manifold learning methods such as ISOMap try to identify a low-dimensional manifold from a set of unorganized samples. ISOMap method is an extension of the classical multidimensional scaling method for dimension reduction, which find a linear subspace in which dissimilarity between data points is preserved. In order to measure dissimilarity, ISOMap uses the geodesic distances on the manifold instead of Euclidean distance. In this paper, we propose a modification of ISOMap using class information, which is often given in company with input data in many applications such as pattern classification. Since the conventional ISOMap does not use class information in approximating true geodesic distance between each pair of data points, it is difficult to construct a data structure related to class-membership that may give important information for given task such as data visualization and classification. The proposed method utilizes class-membership for measuring distance of data pair so as to find a low-dimensional manifold preserving the distance between classes as well as the distance between data points. Through computational experiments on artificial data sets and real facial data sets, we confirm that the proposed method gives better performance than the conventional ISOMap.

Multi-manifold Discriminant Isomap for visualization and classification

Many machine learning and pattern recognition algorithms rely heavily on good distance metrics to achieve competitive performance. While distance metrics can be learned, the computational expense of doing so is currently infeasible on large datasets. In this paper, we propose two efficient-and-effective approaches for selecting the training dataset using Locality-Sensitive Hashing (LSH) with discriminative information, and with K-Means clustering inside LSH buckets, for accelerating metric learning. Our methods yield a speedup factor of (N/C)2, where N is training set size and C ≪ N is the user-selected compressed set size, achieving quadratic speedup to metric learning often realized as a 1–2 or more orders of magnitude improvement with little degradation to accuracy. For example, our generic filter approach enables the current overall fastest Large Margin Nearest Neighbor (LMNN) to learn metrics on one million samples in 6.8 minutes down from 5.4hrs—a 48x speedup. LMNN and similar state-of-the-art methods use tree data structures to speed up nearest-neighbor queries—an advantage that degrades at higher dimensions. Our approach does not share this limitation.