Particle Swarm Optimization for Clustering Ensemble

Data clustering is an important task and has found applications in numerous real-world problems. Since no single clustering algorithm is able to identify all different types of cluster shapes and structures, ensemble clustering was proposed to combine different partitions of the same data generated by multiple clustering algorithms. The key idea of most ensemble clustering algorithms is to find a partition that is consistent with most of the available partitions of the input data. One problem with these algorithms is their inability to handle uncertain data pairs, i.e. data pairs for which about half of the partitions put them into the same cluster and the other half do the opposite. When the number of uncertain data pairs is large, they can mislead the ensemble clustering algorithm in generating the final partition. To overcome this limitation, we propose an ensemble clustering approach based on the technique of matrix completion. The proposed algorithm constructs a partially observed similarity matrix based on the data pairs whose cluster memberships are agreed upon by most of the clustering algorithms in the ensemble. It then deploys the matrix completion algorithm to complete the similarity matrix. The final data partition is computed by applying an efficient spectral clustering algorithm to the completed matrix. Our empirical studies with multiple real-world datasets show that the proposed algorithm performs significantly better than the state-of-the-art algorithms for ensemble clustering.

The localization accuracy of a fingerprint-based localization system is dependent on several factors, one of which is the accuracy and efficiency at which the fingerprint database is clustered. Most highly efficient and accurate clustering algorithms have high time-dependent computational complexity (CC), which tends to limit their practical applicability. A technique that has yet to be explored is the sequential hybridization of multiple low-time CC clustering algorithms to produce a single moderate-time CC clustering algorithm with high localization accuracy. As a result, this paper proposes a clustering algorithm with a moderate time CC that is based on the sequential hybridization of the closest access point (CAP) and improved k-means clustering algorithms. The performance of the proposed sequential hybrid clustering algorithm is determined and compared to the modified affinity propagation clustering (m-APC), fuzzy c-mean (FCM), and 2-CAP algorithms presented in earlier research works using four experimentally generated and publicly available fingerprint databases. The performance metrics considered for the comparisons are the position root mean square error (RMSE) and clustering time based on big O notation. The simulation results show that the proposed sequential hybrid clustering algorithm has improved localization accuracy with position RMSEs of about 54%, 77%, and 52%, respectively, higher than those of the m-APC, FCM, and 2-CAP algorithms. In terms of clustering time, it is 99% and 79% faster than the m-APC and FCM algorithms, respectively, but 90% slower than the 2-CAP algorithm. The results have shown that it is possible to develop a clustering algorithm that has a moderate clustering time with very high localization accuracy through sequential hybridization of multiple clustering algorithms that have a low clustering time with poor localization accuracy. Doi: 10.28991/ESJ-2024-08-02-02 Full Text: PDF

In the context of intelligent customer service in the electric power industry, in order to effectively classify customer data in a reasonable and correct manner, greatly improve the analysis effect of big data, and provide accurate customer service, a multi-category Active Learning algorithm based on multiple Clustering algorithms and multivariate Linear regression (ALCL) algorithm was proposed. Firstly, the category matrix corresponding to each algorithm was obtained through multiple heterogeneous clustering algorithms, the category matrix was labeled and pre-classified by querying common points. Secondly, the key examples used to train the weight coefficient model of the clustering algorithm were selected through the largest search strategy and the most confusing query strategy. Thirdly, the objective solving function was defined, the weight coefficients of each clustering algorithm were obtained by training key examples. Finally, samples with high confidence in the results were classified by the classification calculation combined with the weight coefficient. Six public customer data sets in Yuzhong District, Chongqing were used for experiments. The experimental results show that when the classification accuracy of ALCL is the highest, it is improved by 2.07%~14.01% compared with traditional supervised learning algorithms and other active learning algorithms. The results of hypothesis and significance analysis prove that ALCL has better classification effect in customer data processing.

Groundwater resources do not exist in isolation but are deeply connected with social and ecological systems. As humans continue to modify the land surface, drive climate change, and place greater pressures on global freshwater resources, it is increasingly necessary to assess global groundwater resources through their relationships to these coupled systems. While several global classifications of physical groundwater systems exist, there is no data-driven global typology based on groundwater interactions with connected social and ecological systems. Though physical attributes remain hydrogeologically important, a more expansive systems-oriented classification is needed for policy development, applied research, and to develop the next generation of global hydrological models.We fill this gap by producing a spatially-explicit, moderate resolution (5 arcminute) global map of groundwater system archetypes based on groundwater interactions with social and ecological systems. These include interactions with streamflow, ecosystems, climate, agriculture, the economy, and water governance and management, all underpinned by existing global data. Archetypes, each with a unique set of interaction strengths and combinations, form a finite set of characteristic “fingerprints” that represent the dominant modes of interactions between groundwater and connected social and ecological systems. We find all WHYMAP large aquifer systems of the world are characterized by multiple social-ecological archetypes, suggesting that differentiated, context-appropriate approaches are necessary within large aquifers that are often assumed as uniform in global assessments and initiatives.We derive archetypes using multiple clustering algorithms and assign archetype membership based on majority agreement across clustering methods after cluster reclassification to create comparable maps. This multiple-method approach renders the archetypes more robust and less contingent on a single clustering algorithm while simultaneously enabling greater representation of archetype uncertainty.We additionally provide an outlook on sustainable development opportunities and challenges for each archetype. We summarize data sets that represent notable social-ecological outcomes  related to the UN Sustainable Development Goals (SDGs), including: crop yield gaps (SDG 2), remotely sensed groundwater storage trends (SDG 6), economic inequality (SDG 10), human modification of terrestrial systems (SDG 15), and likelihood for hydropolitical interaction (SDG 16), among others. This work provides a number of useful contributions. First, the combination of archetyping (i.e., system characterization) and archetype-specific SDG outlook analysis provides a robust, data-driven overview of the role of groundwater in the global sustainability discourse. Secondly, the archetypes identify social-ecological system similarities across the globe, which may support interregional cooperation and networking, coordinated investment and interventions. Thirdly, as we harness the rapid growth in global data that document groundwater system interactions as the basis for our analysis, we simultaneously provide a synthesis and snapshot of the pertinent global data space. This snapshot can be used to identify the need for further data collection, especially on socio-economic interactions that remain underrepresented in global data. And finally, the archetypes raise awareness, build capacity, and shift mental models about the emerging perspective that it is necessary to conceptualize groundwater as a socially and ecologically connected resource.

<p>Microarray technology allows the simultaneous monitoring of thousands of genes. Based on the gene expression measurements, microarray technology have proven powerful in gene expression profiling for discovering new types of diseases and for predicting the type of a disease. Gridding, segmentation and intensity extraction are the three important steps in microarray image analysis. Clustering algorithms have been used for microarray image segmentation with an advantage that they are not restricted to a particular shape and size for the spots. Instead of using single feature clustering algorithm, this paper presents multiple feature clustering algorithm with three features for each pixel such as pixel intensity, distance from the center of the spot and median of surrounding pixels. In all the traditional clustering algorithms, number of clusters and initial centroids are randomly selected and often specified by the user. In this paper, a new algorithm based on empirical mode decomposition algorithm for the histogram of the input image will generate the number of clusters and initial centroids required for clustering. It overcomes the shortage of random initialization in traditional clustering and achieves high computational speed by reducing the number of iterations. The experimental results show that multiple feature Fuzzy C-means has segmented the microarray image more accurately than other algorithms.</p>

Clustering ensemble is one of the popular methods in the field of data mining for discovering hidden patterns in unlabeled datasets. Researches have shown that selecting base clustering results with certain differences and high quality to participate in the fusion process can improve the quality of the final result. However, the existing inherent characteristics of uncertainty, ambiguity, and overlap of the base clustering results make the selection of the base clustering members more difficult. The accuracy of the final results is easily disturbed by low-quality base clustering members. From the perspective of granular computing, a novel clustering ensemble model is proposed. The similarity among ensemble members is measured by granularity distance, so the quality of the base clustering results is ensured meanwhile the difference among them is enlarged, which is beneficial to improve the accuracy of the final result. According to the dividing ability of knowledge granularity, the method of elements generation for the co-association matrix is optimized and improved. The results obtained from the improved sample similarity measurement are more consistent with the structure of the real data. Compared with the traditional single clustering algorithm and some popular clustering ensemble methods, experiments show that the proposed model improves the quality of the final clustering result and has good expandability.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Each clustering algorithm usually optimizes a qualification metric during its learning process. The qualification metric in traditional clustering algorithms considers all the features of under-consideration dataset equally; it means each feature participates in the clustering process equivalently. Considering that some features have more information than the others in a dataset (due to their lower information or their higher variances, etc.), we proposes a fuzzy weighted clustering algorithm. We name this new clustering algorithm, Fuzzy Weighted Locally Adaptive Clustering (FWLAC) algorithm. The proposed FWLAC algorithm is capable of handling imbalanced clustering. However, FWLAC algorithm suffers from its sensitivity to the two parameters that should be tuned manually. The performance of FWLAC algorithm is affected by well-tuning of its parameters. So the paper proposes two solutions to well-tuning of its two parameters. In the first solution, we propose a simple clustering ensemble framework to show the sensitivity of the WLAC algorithm to its manual well-tuning. Although it is not a try-and-error procedure, it is like a grid search, where we use different pairs of values for both parameters h 1 and h 2. Per each pair of values for parameters h 1 and h 2, the algorithm produces a partitioning. So after the grid search, we obtain a large number of partitionings. We break any of the partitionings into its clusters, and then they form an ensemble of clusters. Finally the consensus partitioning is extracted from them by a consensus function. The algorithm is not data dependent at all. For all datasets, we use a similar grid search and a similar set of values for the parameters h 1 and h 2. In this way we have proposed an alternative solution to parameter selection. We use a selection phase to select/remove some clusters from our ensemble of clusters to obtain an elite ensemble of clusters. To do this, a stability measure, normalized mutual information (NMI), was used to validate a cluster. The paper shows the effectiveness of the proposed clustering frameworks both theoretically and experimentally.

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Clustering of multi-view relational data based on particle swarm optimization

Clustering ensemble technique has been shown to be effective in improving the accuracy and stability of single clustering algorithms. With the development of information technology, the amount of data, such as image, text and video, has increased rapidly. Efficiently clustering these large-scale datasets is a challenge. Clustering ensembles usually transform clustering results to a co-association matrix, and then to a graph-partition problem. These methods may suffer from information loss when computing the similarity among samples or base clusterings. Rich information between samples and base clusterings is ignored. Moreover, the results are not discrete. They need post-processing steps to obtain the final clustering result, which will deviate greatly from the real clustering result. To address this problem, we propose a co-clustering ensemble based on bilateral k-means (CEBKM) algorithm. Our algorithm can simultaneously cluster samples and base clusterings of a dataset, to fully exploit the potential information between the samples and the base clusterings. In addition, it can directly obtain the final clustering results without using other clustering algorithms. The proposed method, outperformed several state-of-the-art clustering ensemble methods in experiments conducted on real-world and toy datasets.

There are some general procedures to generate the clusterings in clustering ensembles. One can apply different algorithms to create different clusterings of the data. Some clustering algorithms like k-means require initialization of parameters. Different initializations can lead to different clustering results. The parameters of a clustering algorithm, such as the number of clusters, can be altered to create different data clusterings. Different versions of the data can also be used as the input to the clustering algorithm, leading to different partitions. In this paper, we propose a new scheme for constructing multiple independent clusterings using additional artificially generated data. Then, we compare our new method with seven general clustering ensemble constructing methods. The experiments show that our approach can achieve higher or comparable performance than the other methods.

This paper develops a fuzzy clustering method based on the fuzzy t-norm, and uses the data of China securities market to analyze the factors influencing the portfolio selection. Furthermore, this paper estimates an investment portfolio based on fuzzy clustering. The results show that the investor can learn the feature of securities. This method is very important for the portfolio selection. it is very important to further study the modern portfolio theory for the investor. Portfolio selection is the study of how people should invest their wealth. It is a process of trading off risk and expected return to find the best portfolio of assets and liabilities. Cluster analysis has been widely applied in portfolio selection. However, it is an ill-posed problem and no single clustering algorithm is able to obtain satisfactory solutions for all types of data sets in customer segmentation. Cluster ensemble emerges as an alternative approach for improving the investment performance. Using the clustering analysis, investors can understand the basic features of securities, and determine the scope of the portfolio choice. The traditional clustering analysis is that all objects or characteristics need to identify and strictly divided. In this process, the division of category boundaries is very obvious. However, in the real situation, many characteristics of objects are not obviously. In order to solve this situation, this paper develops a cluster segment method based on the fuzzy t-Norm, considers the problem of how to construct a port-folio based on the bid-ask spread, depth, order imbalance, market value, profit margins, asset-liability ratio.

In an effort to achieve improved classifier accuracy, extensive research has been conducted in classifier ensembles. Very recently, cluster ensembles have emerged. It is well known that off-the-shelf clustering methods may discover different structures in a given set of data. This is because each clustering algorithm has its own bias resulting from the optimization of different criteria. Furthermore, there is no ground truth against which the clustering result can be validated. Thus, no cross-validation technique can be carried out to tune input parameters involved in the clustering process. As a consequence, the user is not equipped with any guidelines for choosing the proper clustering method for a given dataset. Cluster ensembles offer a solution to challenges inherent to clustering arising from its ill-posed nature. Cluster ensembles can provide more robust and stable solutions by leveraging the consensus across multiple clustering results, while averaging out emergent spurious structures that arise due to the various biases to which each participating algorithm is tuned. In this chapter, we discuss the problem of combining multiple weighted clusters, discovered by a locally adaptive algorithm (Domeniconi, Papadopoulos, Gunopulos, & Ma, 2004) which detects clusters in different subspaces of the input space. We believe that our approach is the first attempt to design a cluster ensemble for subspace clustering (Al-Razgan & Domeniconi, 2006). Recently, several subspace clustering methods have been proposed (Parsons, Haque, & Liu, 2004). They all attempt to dodge the curse of dimensionality which affects any algorithm in high dimensional spaces. In high dimensional spaces, it is highly likely that, for any given pair of points within the same cluster, there exist at least a few dimensions on which the points are far apart from each other. As a consequence, distance functions that equally use all input features may not be effective. Furthermore, several clusters may exist in different subspaces comprised of different combinations of features. In many real-world problems, some points are correlated with respect to a given set of dimensions, while others are correlated with respect to different dimensions. Each dimension could be relevant to at least one of the clusters. Global dimensionality reduction techniques are unable to capture local correlations of data. Thus, a proper feature selection procedure should operate locally in input space. Local feature selection allows one to embed different distance measures in different regions of the input space; such distance metrics reflect local correlations of data. In (Domeniconi, Papadopoulos, Gunopulos, & Ma, 2004) we proposed a soft feature selection procedure (called LAC) that assigns weights to features according to the local correlations of data along each dimension. Dimensions along which data are loosely correlated receive a small weight, which has the effect of elongating distances along that dimension. Features along which data are strongly correlated receive a large weight, which has the effect of constricting distances along that dimension. Thus the learned weights perform a directional local reshaping of distances which allows a better separation of clusters, and therefore the discovery of different patterns in different subspaces of the original input space.

In remotely sensed data, two commonly held ways are used to obtain a relevant partitioning. The first one consists to apply several clustering algorithms individually, then to choose the best partition. The second commonly held way consists to apply the same clustering algorithm with several initializations and to choose the best partition. It is worth noting that the best partition is chosen according to a given evaluation criterion. Conversely, the Clustering ensembles can provide the best partition with high accuracy and consequently overcome limitations of traditional approaches. Clustering ensembles usually involve two stages. First, multiple partitions are obtained through several runs of initial clustering analysis. Subsequently, the specific consensus function is used in order to find a final consensus partition from multiple input partitions. In this paper, we investigate this technique in the unsupervised classification by using Synthetic data and composite data image. The first stage is assumed by four clustering algorithms, the well-known k-means algorithm, the k-harmonic means algorithm, Fuzzy c-means and the self-organizing map. The best clustering is obtained according to WB index. The relabeling and the voting methods are used in the second stage. Experimental results obtained are satisfactory.