CHSMST:A Clustering algorithm based on hyper surface and Minimum Spanning Tree

Gene expression data clustering provides a powerful tool for studying functional relationships of genes in a biological process. Identifying correlated expression patterns of genes represents the basic challenge in this clustering problem. This paper describes a new framework for representing a set of multi-dimensional gene expression data as a Minimum Spanning Tree (MST), a concept from the graph theory. A key property of this representation is that each cluster of the expression data corresponds to one subtree of the MST, which rigorously converts a multi-dimensional clustering problem to a tree partitioning problem. We have demonstrated that though the inter-data relationship is greatly simplified in the MST representation, no essential information is lost for the purpose of clustering. Two key advantages in representing a set of multi-dimensional data as an MST are: (1) the simple structure of a tree facilitates efficient implementations of rigorous clustering algorithms, which otherwise are highly computationally challenging; and (2) as an MST-based clustering does not depend on detailed geometric shape of a cluster, it can overcome many of the problems faced by classical clustering algorithms. Based on the MST representation, we have developed a number of rigorous and efficient clustering algorithms, including two with guaranteed global optimality. We have implemented these algorithms as a computer software EXpression data Clustering Analysis and VisualizATiOn Resource (EXCAVATOR). To demonstrate its effectiveness, we have tested it on three data sets, i.e. expression data from yeast Saccharomyces cerevisiae, expression data in response of human fibroblasts to serum, and Arabidopsis expression data in response to chitin elicitation. The test results are highly encouraging. EXCAVATOR is available on request from the authors.

In view of today's unprecedented diverse and discrete mass text data processing, this paper presents a distributed MST (minimum spanning tree) algorithm based on MapReduce programming model. And with this MST algorithm, a distributed MST text clustering algorithm is designed and implemented. In this paper, this clustering algorithm is analyzed in three aspects: text feature vector extraction, graph construction and MST construction. And some data was used to experimentally compare this algorithm with ordinary MST clustering algorithm and MapReduce-based K-means clustering algorithm.

More analysis has been done to discover the meaningful unusual patterns which may mean fraud or anomaly. In this paper, a novel unsupervised approach for discovering meaningful unusual observations is proposed. We firstly apply an unsupervised version of Hyper Surface Classification (HSC) algorithm to gain the separating hyper surface. It needs no domain knowledge but can not discover the local unusual pattern. To solve this problem, we additionally search the Minimum Spanning Tree (MST). Given the domain knowledge, a process of subdividing is proposed to detect unusual pattern in each Minimum Spanning Tree. Experimental results show that our approach can detect unusual patterns effectively, even some of which are overlooked by using the traditional clustering and outlier detection algorithms.

Although density‐based clustering algorithms can successfully define clusters in arbitrary shapes, they encounter issues if the dataset has varying densities or neck‐typed clusters due to the requirement for precise distance parameters, such as eps parameter of DBSCAN. These approches assume that data density is homogenous, but this is rarely the case in practice. In this study, a new clustering algorithm named ANDClust (Adaptive Neighborhood Distance‐based Clustering Algorithm) is proposed to handle datasets with varying density and/or neck‐typed clusters. The algorithm consists of three parts. The first part uses Multivariate Kernel Density Estimation (MulKDE) to find the dataset's peak points, which are the start points for the Minimum Spanning Tree (MST) to construct clusters in the second part. Lastly, an Adaptive Neighborhood Distance (AND) ratio is used to weigh the distance between the data pairs. This method enables this approach to support inter‐cluster and intra‐cluster density varieties by acting as if the distance parameter differs for each data of the dataset. ANDClust is tested on synthetic and real datasets to reveal its efficiency. The algorithm shows superior clustering quality in a good run‐time compared to its competitors. Moreover, ANDClust could effectively define clusters of arbitrary shapes and process high‐dimensional, imbalanced datasets may have outliers.

Clustering is an important tool to explore the hidden structure of large databases. There are several algorithms based on different approaches (hierarchical, partitional, density-based, model-based, etc.). Most of these algorithms have some discrepancies, e.g. they are not able to detect clusters with convex shapes, the number of the clusters should be a priori known, they suffer from numerical problems, like sensitiveness to the initialization, etc. In this paper we introduce a new clustering algorithm based on the sinergistic combination of the hierarchial and graph theoretic minimal spanning tree based clustering and the partitional Gaussian mixture model-based clustering algorithms. The aim of this hybridization is to increase the robustness and consistency of the clustering results and to decrease the number of the heuristically defined parameters of these algorithms to decrease the influence of the user on the clustering results. As the examples used for the illustration of the operation of the new algorithm will show, the proposed algorithm can detect clusters from data with arbitrary shape and does not suffer from the numerical problems of the Gaussian mixture based clustering algorithms.KeywordsCluster AlgorithmGaussian Mixture ModelMinimal Span TreeFuzzy ClusterBase Cluster AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Discovering clusters remains challenging when dealing with complex data structures, including those with varying densities, arbitrary shapes, weak separability, or the presence of noise. In this article, we propose a novel clustering algorithm called representative point-based clustering with neighborhood information (RPC-NI), which highlights the significance of neighborhood information often overlooked by existing clustering methods. The proposed algorithm first introduces a new local centrality metric that integrates both neighborhood density and topological convergence to identify core representative points, effectively capturing the structural characteristics of the data. Subsequently, a density-adaptive distance is defined to evaluate dissimilarities between these core representative points, and such distance is used to construct a minimum spanning tree (MST) over these points. Finally, an MST-based clustering algorithm is employed to yield the desired clusters. Incorporating neighborhood information enables RPC-NI to comprehensively determine representative points, and having multiple representative points per cluster allows RPC-NI to adapt to clusters of arbitrary shapes, varying densities, and different sizes. Extensive experiments on widely used datasets demonstrate that RPC-NI outperforms baseline algorithms in terms of clustering accuracy and robustness. These results provide further evidence for the importance of incorporating neighborhood information discovering clusters with complex structures.

Rapid Evaluation-Based Feature Diminution algorithm involves finding a reduced set with the essential Feature which produces as the original set of Feature. The Feature diminution is performed by removing the irrelevant and redundant Features. The Rapid Evaluation-Based Feature Diminution algorithm (REFD) removes the irrelevancy by identifying relevant Feature to the target, and removing the rest and finally formed as weight graph. The weighted graph is converted to MST by Boruvka's algorithm. In the resultant set the representative features are selected by reducing MST into forest using Symmetric Uncertainty measures. The efficiency of REFD is improved by using efficient Minimum Spanning Tree (MST) clustering algorithm. The image, microarray and text dataset are taken as input to the REFD algorithm and reduced set given as the input to classifiers called the Naive Bayes and the decision tree and compared with original set of Features. The REFD produces smaller subsets of Features yet increases the performances.

Wireless sensor networks (WSNs) are mainly used for systematic gathering of useful information and for the transmission of the gathered data to the base station for further processing. Despite the innumerable applications of WSNs, the sensor nodes of these networks have limited energy, limited computing power and limited bandwidth, in which energy is major constraint. Clustering has been widely used in WSNs to reduce energy consumption, which decrease the communication load, and thereby prolong the network lifetime by minimizing the redundant information using the data aggregation model on the cluster heads. In this paper, we have proposed an information similarity based clustering algorithm where transmission of the data to the sink is done using the least spanning tree algorithm. This approach firstly uses improved LEACH algorithm for election of cluster heads and then cluster formation under the elected cluster heads is done depending upon the non-cluster head node's maximum information similarity with the cluster head nodes. By utilizing the sleep scheduling & least spanning tree algorithm, this approach effectively reduces the redundant data transmission and whole energy consumed in the network. Simulation results show that network lifetime has increased effectively in comparison to other clustering algorithm such as LEACH.

The minimum spanning tree (MST), a graph constructed from a distribution of points, draws lines between pairs of points so that all points are linked in a single skeletal structure that contains no loops and has minimal total edge length. The MST has been used in a broad range of scientific fields such as particle physics (to distinguish classes of events in collider collisions), in astronomy (to detect mass segregation in star clusters) and cosmology (to search for filaments in the cosmic web). Its success in these fields has been driven by its sensitivity to the spatial distribution of points and the patterns within. MiSTree, a public Python package, allows a user to construct the MST in a variety of coordinates systems, including Celestial coordinates used in astronomy. The package enables the MST to be constructed quickly by initially using a k-nearest neighbour graph (kNN, rather than a matrix of pairwise distances) which is then fed to Kruskal's algorithm to construct the MST. MiSTree enables a user to measure the statistics of the MST and provides classes for binning the MST statistics (into histograms) and plotting the distributions. Applying the MST will enable the inclusion of high-order statistics information from the cosmic web which can provide additional information to improve cosmological parameter constraints. This information has not been fully exploited due to the computational cost of calculating N-point statistics. MiSTree was designed to be used in cosmology but could be used in any field which requires extracting non-Gaussian information from point distributions. The source code for MiSTree is available on GitHub at https://github.com/knaidoo29/mistree

Clustering with a minimum spanning tree of scale-free-like structure

N points are given in the Euclidean plane, and the minimum spanning tree problem seeks for a minimum spanning tree interconnecting the n points so that there is only one path between any two points. One of the classic and frequently-used algorithms for minimum spanning tree problem is Prim's algorithm, but it consumes large time and space complexity for the plane minimum spanning tree problem is of O(n2) numbers of edges. Luckily, it was proved that the plane minimum spanning tree is a sub-graph of Delaunay triangulation for the given points in the plane, and the number of edges in the triangulation is O(n). This motivates us to efficiently compute the Delaunay triangulation of the given points and then find the minimum spanning tree in the triangulation. This paper presents an algorithm based on the divide and conquer for Delaunay triangulation together with the Prim's algorithm to produce an O(nlogn) algorithm for minimum spanning tree problem in the plane, implements the visual graphic interface with various selected algorithms for plane minimum spanning tree and compares their running time.

The addition of new houses in the Kota Mas Cimahi housing complex has triggered an increase in the need for electricity distribution installations. To optimize the electricity distribution network, in this case minimizing the installation of electrical cables, an application is needed that can optimize the installation of electrical cables, namely the Kruskal algorithm which is an algorithm for finding the minimum spanning tree value on a weighted graph. In this study, two methods will be used, namely the manual method and the POM-QM for Windows software to help solve the minimum spanning tree problem. This study's goal is to identify the Kota Mas Cimahi housing complex's energy distribution network model's graph structure and determine a minimum spanning tree solution with POM-QM for Windows software and the Kruskal technique done by hand. In order to gather data for this study, PLN was the primary and secondary source Cimahi UP3. The distribution data for the electricity cable network is arranged in the form of a network image. By representing the electricity distribution network in the form of a graph, the minimum spanning tree is then obtained, and the solution to the MST (Minimum Spanning Tree) problem uses the Kruskal algorithm and TORA software. The results of the study showed that the minimum spanning tree cable length which was originally 1424.61 meters, after using the Kruskal algorithm manually became 1258.05 meters and also by using the POM-QM for Windows software obtained a minimum result of 1258.05 meters, so it can be concluded that there is a cable length efficiency of 11.69%.

The Minimum Spanning Tree (MST) problem is one of the most popular and important problems in the research area of distributed computing and networks. Contrary to the theoretical models where we usually have a global knowledge of all nodes and the corresponding distances for MST construction, in a realistic network (e.g., Internet) a node always has to rely on local knowledge only, that is it neither knows all other nodes nor exact distances between these nodes. In this paper we propose an approach for MST approximation based on local knowledge of a small subset of existing nodes by using the CARMA metric as a distance substitute. According to the evaluation results, our approach achieves a good MST approximation with respect to a communication cost and avoids extraneous communication needed for latency measurements.

Based on a recently proposed q-dependent detrended cross-correlation coefficient, ρ_{q} [J. Kwapień, P. Oświęcimka, and S. Drożdż, Phys. Rev. E 92, 052815 (2015)PLEEE81539-375510.1103/PhysRevE.92.052815], we generalize the concept of the minimum spanning tree (MST) by introducing a family of q-dependent minimum spanning trees (qMSTs) that are selective to cross-correlations between different fluctuation amplitudes and different time scales of multivariate data. They inherit this ability directly from the coefficients ρ_{q}, which are processed here to construct a distance matrix being the input to the MST-constructing Kruskal's algorithm. The conventional MST with detrending corresponds in this context to q=2. In order to illustrate their performance, we apply the qMSTs to sample empirical data from the American stock market and discuss the results. We show that the qMST graphs can complement ρ_{q} in disentangling "hidden" correlations that cannot be observed in the MST graphs based on ρ_{DCCA}, and therefore, they can be useful in many areas where the multivariate cross-correlations are of interest. As an example, we apply this method to empirical data from the stock market and show that by constructing the qMSTs for a spectrum of q values we obtain more information about the correlation structure of the data than by using q=2 only. More specifically, we show that two sets of signals that differ from each other statistically can give comparable trees for q=2, while only by using the trees for q≠2 do we become able to distinguish between these sets. We also show that a family of qMSTs for a range of q expresses the diversity of correlations in a manner resembling the multifractal analysis, where one computes a spectrum of the generalized fractal dimensions, the generalized Hurst exponents, or the multifractal singularity spectra: the more diverse the correlations are, the more variable the tree topology is for different q's. As regards the correlation structure of the stock market, our analysis exhibits that the stocks belonging to the same or similar industrial sectors are correlated via the fluctuations of moderate amplitudes, while the largest fluctuations often happen to synchronize in those stocks that do not necessarily belong to the same industry.

Trees, including minimum spanning trees (MSTs), are commonly used in phylogenetic studies. But, for the research community, it may be unclear that the presented tree is just a hypothesis, chosen from among many possible alternatives. In this scenario, it is important to quantify our confidence in both the trees and the branches/edges included in such trees. In this paper, we address this problem for MSTs by introducing a new edge betweenness metric for undirected and weighted graphs. This spanning edge betweenness metric is defined as the fraction of equivalent MSTs where a given edge is present. The metric provides a per edge statistic that is similar to that of the bootstrap approach frequently used in phylogenetics to support the grouping of taxa. We provide methods for the exact computation of this metric based on the well known Kirchhoff’s matrix tree theorem. Moreover, we implement and make available a module for the PHYLOViZ software and evaluate the proposed metric concerning both effectiveness and computational performance. Analysis of trees generated using multilocus sequence typing data (MLST) and the goeBURST algorithm revealed that the space of possible MSTs in real data sets is extremely large. Selection of the edge to be represented using bootstrap could lead to unreliable results since alternative edges are present in the same fraction of equivalent MSTs. The choice of the MST to be presented, results from criteria implemented in the algorithm that must be based in biologically plausible models.