PFHC: A clustering algorithm based on data partitioning for unevenly distributed datasets

Finding clusters in data is a challenging problem especially when the clusters are being of widely varied shapes, sizes, and densities. Herein a new scalable clustering technique which addresses all these issues is proposed. In data mining, the purpose of data clustering is to identify useful patterns in the underlying dataset. Within the last several years, many clustering algorithms have been proposed in this area of research. Among all these proposed methods, density clustering methods are the most important due to their high ability to detect arbitrary shaped clusters. Moreover these methods often show good noise-handling capabilities, where clusters are defined as regions of typical densities separated by low or no density regions. In this paper, we aim at enhancing the well-known algorithm DBSCAN, to make it scalable and able to discover clusters from uneven datasets in which clusters are regions of homogenous densities. We achieved the scalability of the proposed algorithm by using the k-means algorithm to get initial partition of the dataset, applying the enhanced DBSCAN on each partition, and then using a merging process to get the actual natural number of clusters in the underlying dataset. This means the proposed algorithm consists of three stages. Experimental results using synthetic datasets show that the proposed clustering algorithm is faster and more scalable than the enhanced DBSCAN counterpart.

A number of clustering algorithms can be employed to find clusters in multivariate data. However, the effectiveness and efficiency of the existing algorithms are limited, since the respective data has high dimension, contain large amount of noise and consist of clusters with arbitrary shapes and densities. In this paper, a new kernel density-based clustering algorithm, called Local Triangular Kernel-based Clustering (LTKC), is proposed to deal with these conditions. LTKC is based on combination of k-nearest-neighbor density estimation and triangular kernel density-based clustering. The advantages of our LTKC approach are: (1) it has a firm mathematical basis; (2) it requires only one parameter, number of neighbors; (3) it defines the number of cluster automatically; (4) it allows discovering clusters with arbitrary shapes and densities ;and (5) it is significantly faster than existing algorithms. LTKC is tested using artificial data and applied to some UCI data. A comparison with k-means, KFCM and well known density-based clustering algorithms including ILGC, DBSCAN, and DENCLUE shows the superiority of our proposed LTKC algorithm.

Context. Dark energy was first detected from large distances on gigaparsec scales. If it is vacuum energy (or Einstein’s Λ), it should also exist in very local space. Here we discuss its measurement on megaparsec scales of the Local Group. Aims. We combine the modified Kahn-Woltjer method for the Milky Way-M 31 binary and the HST observations of the expansion flow around the Local Group in order to study in a self-consistent way and simultaneously the local density of dark energy and the dark matter mass contained within the Local Group. Methods. A theoretical model is used that accounts for the dynamical effects of dark energy on a scale of ∼ 1M pc. Results. The local dark energy density is put into the range 0.8−3.7ρv (ρv is the globally measured density), and the Local Group mass lies within 3.1−5.8 × 10 12 M� . The lower limit of the local dark energy density, about 4/5× the global value, is determined by the natural binding condition for the group binary and the maximal zero-gravity radius. The near coincidence of two values measured

Clustering algorithms are attractive for the task of class identification in spatial databases. However, the application to large spatial databases rises the following requirements for clustering algorithms: minimal requirements of domain knowledge to determine the input parameters, discovery of clusters with arbitrary shape and good efficiency on large databases.DBSCAN clustering algorithm relying on a density-based notion of clusters which is designed to discover clusters of arbitrary shape. DBSCAN requires only one input parameter and supports the user in determining an appropriate value for it. DBSCAN cannot find clusters based on difference in densities. We extend the DBSCAN algorithm so that it can also detect clusters that differ in densities and without the need to input the value of Eps because our algorithm can find the appropriate value for each cluster individually by replacing Eps by Local cluster density.KeywordsClusteringArbitrary ShapeDBSCANvariable Densities

The localization accuracy of a fingerprint-based localization system is dependent on several factors, one of which is the accuracy and efficiency at which the fingerprint database is clustered. Most highly efficient and accurate clustering algorithms have high time-dependent computational complexity (CC), which tends to limit their practical applicability. A technique that has yet to be explored is the sequential hybridization of multiple low-time CC clustering algorithms to produce a single moderate-time CC clustering algorithm with high localization accuracy. As a result, this paper proposes a clustering algorithm with a moderate time CC that is based on the sequential hybridization of the closest access point (CAP) and improved k-means clustering algorithms. The performance of the proposed sequential hybrid clustering algorithm is determined and compared to the modified affinity propagation clustering (m-APC), fuzzy c-mean (FCM), and 2-CAP algorithms presented in earlier research works using four experimentally generated and publicly available fingerprint databases. The performance metrics considered for the comparisons are the position root mean square error (RMSE) and clustering time based on big O notation. The simulation results show that the proposed sequential hybrid clustering algorithm has improved localization accuracy with position RMSEs of about 54%, 77%, and 52%, respectively, higher than those of the m-APC, FCM, and 2-CAP algorithms. In terms of clustering time, it is 99% and 79% faster than the m-APC and FCM algorithms, respectively, but 90% slower than the 2-CAP algorithm. The results have shown that it is possible to develop a clustering algorithm that has a moderate clustering time with very high localization accuracy through sequential hybridization of multiple clustering algorithms that have a low clustering time with poor localization accuracy. Doi: 10.28991/ESJ-2024-08-02-02 Full Text: PDF

This paper combines the graph theory and P system to solve the clustering problem. In order to effectively identify clusters with arbitrary shapes and uneven densities, we combine MkNN clustering algorithm and graph theory to propose a mutual k-nearest neighbors graph (MkNNG) clustering algorithm. In order to further improve the efficiency of MkNNG algorithm, based on the non-determinism and great parallelism of P system, a cell-like P system with multi-promoters and multi-inhibitors named mutual k-nearest neighbors graph P system (MkNNG-P) is designed. And then based on MkNNG-P system, a novel clustering algorithm named MkNNG-P clustering algorithm is proposed, which uses the membrane objects and rules to solve the clustering problem. MkNNG-P algorithm first calculates the dissimilarity between any two nodes in n-1 membranes in parallel. After then it uses one membrane to get k-nearest neighbors of n nodes. Finally, one membrane is used to find mutual k-nearest neighbors and construct MkNNG to discover the natural clusters in the data set. Experiments show that MkNNG-P algorithm has the advantages of both MkNNG and P system. It not only can obtain good clustering quality for data of different sizes and shapes without presetting clustering numbers, but also has extremely high computing speed.

To develop a local search clustering algorithm for analyzing the dynamic contrasted enhanced (DCE)-MRI data for determining the vascular permeability. The clustered signal will have better contrast-to-noise ratio (CNR) and thus can improve the analysis. In DCE-MRI, data often suffer from low CNR. The CNR is particularly poor in central nervous system and could lead to spurious results. We propose a local search clustering algorithm that groups proximal voxels with similar uptake curves and T1 values. The algorithm starts with a seed voxel, and then grows outward to recruit new voxels into a cluster. Once a cluster is formed, all members of the current cluster form a 'seed collection' and the seed point of the cluster becomes the seed collection. The expansions and updates continue with the seed point/collection until the stopping criterion is met. To investigate the effectiveness of our clustering algorithm, we used a 64×64 2D Shepp-Logan phantom. Variations on the T1 values and permeability parameters were applied on different regions in the phantom. The time resolution was set to 30sec and 82 post-contrast data were created. After the DCE-MRI data were generated, Gaussian noise was applied to the images to investigate the effect of noise on the clustering algorithm. DCE analysis was performed on the clustered data and the results were compared against the Result obtained from voxel-by-voxel analysis. The addition of noise degraded the results of DCE estimation. The clustering algorithm on average reduced the errors of the permeability parameters by more than 50% compared to the voxel-by-voxel analysis. We developed a local search clustering algorithm to segment the concentration time curves of the DCE-MRI data. The proposed clustering algorithm enhances the apparent CNR within the clustered data and reduces the errors of the permeability parameters.

As data mining having attracted a significant amount of research attention, many clustering algorithms have been proposed in the past decades. However, most of existing clustering methods have high computational time or are not suitable for discovering clusters with non-convex shape. In this paper, an efficient clustering algorithm CHSMST is proposed, which is based on clustering based on hyper surface (CHS) and minimum spanning tree. In the first step, CHSMST applies CHS to obtain initial clusters immediately. Thereafter, minimum spanning tree is intro- duced to handle locally dense data which is hard for CHS to deal with. The experiments show that CHSMST can discover clusters with arbitrary shape. Moreover, CHSMST is insensitive to the order of input samples and the run time of the algorithm increases moderately as the scale of dataset becomes large.

<p>Clustering is an important technique for data analysis and knowledge discovery. In the context of big data, the density-based clustering algorithm faces three challenging problems: unreasonable division of data gridding, poor parameter optimization ability and low efficiency of parallelization. In this study, a density-based clustering algorithm by using improve fruit fly optimization based on MapReduce (MR-DBIFOA) is proposed to tackle these three problems. Firstly, based on KD-Tree, a division strategy (KDG) is proposed to divide the cell of grid adaptively. Secondly, an improve fruit fly optimization algorithm (IFOA) which use the step strategy based on knowledge learn (KLSS) and the clustering criterion function (CFF) is designed. In addition, based on IFOA algorithm, the optimal parameters of local clustering are dynamically selected, which can improve the clustering effect of local clustering. Meanwhile, in order to improve the parallel efficiency, the density-based clustering algorithm using IFOA (MR-QRMEC) are proposed to parallel compute the local clusters of clustering algorithm. Finally, based on QR-Tree and MapReduce, a cluster merging algorithm (MR-QRMEC) is proposed to get the result of clustering algorithm more quickly, which improve the core clusters merging efficiency of density-based clustering algorithm. The experimental results show that the MR-DBIFOA algorithm has better clustering results and performs better parallelization in big data.</p> <p> </p>

As the data size increases, the efficiency of algorithm and the clustering quality draw more attraction. CURD (clustering using references and density) is a fast clustering algorithm based on reference and density, which can discover clusters with arbitrary shape and has the linear times complexity. However, it still has some shortcomings such as: the efficiency to deal with the high-dimensional data is uncertain, the noise processing is not ideal, besides the number of the clustering results may not satisfy the requirement of the users. According to these deficiencies, this paper introduces a new method to propose the high-dimensional data with information entropy technology and quotient space theory. Additionally it disposes the noise date in two stages. Finally, some improvement are given on the step of sorting the reference points by quotient space theory to produce multi-level clustering results so as to meet the different needs of customers. Experiments show that the improved algorithm not only improves the quality of the clustering algorithm but also maintains the high efficiency.

In this paper, we introduce a new clustering algorithm, FCC, for intrusion detection based on the concept of fuzzy connectedness. This concept was introduced by Rosenfeld in 1979 and used with success in image segmentation; here we extend this approach to clustering and demonstrate its effectiveness in intrusion detection. Starting with a single or a few seed points in each cluster, all the data points are dynamically assigned to the cluster that has the highest fuzzy connectedness value (strongest connection). With an efficient heuristic algorithm, the time complexity of the clustering process is O(NlogN), where N is the number of data points. The value of fuzzy connectedness is calculated using both the Euclidean distance and the statistical properties of clusters. This unsupervised learning method allows the discovery of clusters of any shape. Application of the method in intrusion detection demonstrates that it can detect not only known intrusion types, but also their variants. Experimental results on the KDD-99 intrusion detection data set show the efficiency and accuracy of this method. A detection rate above 94% and a false alarm rate below 4% are achieved, outperforming major competitors by at least 5%.

Density‐based clustering algorithms such as DBSCAN have been widely used for spatial knowledge discovery as they offer several key advantages compared with other clustering algorithms. They can discover clusters with arbitrary shapes, are robust to noise, and do not require prior knowledge (or estimation) of the number of clusters. The idea of using a scan circle centered at each point with a search radius Eps to find at least MinPts points as a criterion for deriving local density is easily understandable and sufficient for exploring isotropic spatial point patterns. However, there are many cases that cannot be adequately captured this way, particularly if they involve linear features or shapes with a continuously changing density, such as a spiral. In such cases, DBSCAN tends to either create an increasing number of small clusters or add noise points into large clusters. Therefore, in this article, we propose a novel anisotropic density‐based clustering algorithm (ADCN). To motivate our work, we introduce synthetic and real‐world cases that cannot be handled sufficiently by DBSCAN (or OPTICS). We then present our clustering algorithm and test it with a wide range of cases. We demonstrate that our algorithm can perform equally as well as DBSCAN in cases that do not benefit explicitly from an anisotropic perspective, and that it outperforms DBSCAN in cases that do. Finally, we show that our approach has the same time complexity as DBSCAN and OPTICS, namely O(n log n) when using a spatial index and O(n2) otherwise. We provide an implementation and test the runtime over multiple cases.

This is a study of how a local agropastoral society in Southwest Ethiopia conceptualizes and experiences changing ‘resource use’ and environmental problems in a time of state-imposed agrarian development schemes. Local space and livelihoods of local groups in the Omo River valley are under stress due to a far-reaching process of state-induced political–economic change that diminishes local environmental stability and biodiversity. A familiar process of mono-crop plantation agriculture and resettlement by state-sponsored agents and enterprises is set in motion since the early 2000s. Local group ethnic groups—both agropastoralists and peasants—are under pressure due to shrinking resource availability, influx of tens of thousands on newcomers and stronger state presence, including in military form. Local peoples, among which the Suri agropastoralists will stand central here, seek new opportunities to connect, both to local and national arenas of power, but are so far no successful in this. The ‘political ecology’ approach followed here reveals state economic and political expansion leading to subversion of environmental impoverishment, local livelihoods, and a de-emphasizing of ethnic and civic group rights. (178)

Although density‐based clustering algorithms can successfully define clusters in arbitrary shapes, they encounter issues if the dataset has varying densities or neck‐typed clusters due to the requirement for precise distance parameters, such as eps parameter of DBSCAN. These approches assume that data density is homogenous, but this is rarely the case in practice. In this study, a new clustering algorithm named ANDClust (Adaptive Neighborhood Distance‐based Clustering Algorithm) is proposed to handle datasets with varying density and/or neck‐typed clusters. The algorithm consists of three parts. The first part uses Multivariate Kernel Density Estimation (MulKDE) to find the dataset's peak points, which are the start points for the Minimum Spanning Tree (MST) to construct clusters in the second part. Lastly, an Adaptive Neighborhood Distance (AND) ratio is used to weigh the distance between the data pairs. This method enables this approach to support inter‐cluster and intra‐cluster density varieties by acting as if the distance parameter differs for each data of the dataset. ANDClust is tested on synthetic and real datasets to reveal its efficiency. The algorithm shows superior clustering quality in a good run‐time compared to its competitors. Moreover, ANDClust could effectively define clusters of arbitrary shapes and process high‐dimensional, imbalanced datasets may have outliers.

Clustering algorithm plays important roles in many fields, such as data mining, data visualization, and so on. In a variety of clustering algorithms, the CHAMELEON algorithm has become a commonly used algorithm because of its ability to discover clusters with arbitrary shapes. This paper makes an intensive study of clustering algorithm, especially CHAMELEON algorithm. After the study of the structure and features of the proposed algorithm, this paper introduces an optimized CHAMELEON algorithm based on local features and grid structure aiming at the deficiency of CHAMELEON. This algorithm generates neighbor graph adaptively, partitions the graph and merges sub-clusters based on local features, so it can produce high quality clustering results. The proposed clustering algorithm is experimented with four data sets, and its performance is compared with CHAMELEON algorithm, DBSCAN algorithm and K-means algorithm. The experimental results show that the clustering algorithm proposed can obtain satisfied clustering effect of data sets with complex distribution.