Dual-granularity weighted ensemble clustering

Clustering ensemble technique has been shown to be effective in improving the accuracy and stability of single clustering algorithms. With the development of information technology, the amount of data, such as image, text and video, has increased rapidly. Efficiently clustering these large-scale datasets is a challenge. Clustering ensembles usually transform clustering results to a co-association matrix, and then to a graph-partition problem. These methods may suffer from information loss when computing the similarity among samples or base clusterings. Rich information between samples and base clusterings is ignored. Moreover, the results are not discrete. They need post-processing steps to obtain the final clustering result, which will deviate greatly from the real clustering result. To address this problem, we propose a co-clustering ensemble based on bilateral k-means (CEBKM) algorithm. Our algorithm can simultaneously cluster samples and base clusterings of a dataset, to fully exploit the potential information between the samples and the base clusterings. In addition, it can directly obtain the final clustering results without using other clustering algorithms. The proposed method, outperformed several state-of-the-art clustering ensemble methods in experiments conducted on real-world and toy datasets.

Clustering ensemble is one of the popular methods in the field of data mining for discovering hidden patterns in unlabeled datasets. Researches have shown that selecting base clustering results with certain differences and high quality to participate in the fusion process can improve the quality of the final result. However, the existing inherent characteristics of uncertainty, ambiguity, and overlap of the base clustering results make the selection of the base clustering members more difficult. The accuracy of the final results is easily disturbed by low-quality base clustering members. From the perspective of granular computing, a novel clustering ensemble model is proposed. The similarity among ensemble members is measured by granularity distance, so the quality of the base clustering results is ensured meanwhile the difference among them is enlarged, which is beneficial to improve the accuracy of the final result. According to the dividing ability of knowledge granularity, the method of elements generation for the co-association matrix is optimized and improved. The results obtained from the improved sample similarity measurement are more consistent with the structure of the real data. Compared with the traditional single clustering algorithm and some popular clustering ensemble methods, experiments show that the proposed model improves the quality of the final clustering result and has good expandability.

目的 随着实际应用场景中海量数据采集技术的发展和数据标注成本的不断增加，自监督学习成为海量数据分析的一个重要策略。然而，如何从海量数据中抽取有用的监督信息，并该监督信息下开展有效的学习仍然是制约该方向发展的研究难点。为此，提出了一个基于共识图学习的自监督集成聚类框架。方法 框架主要包括3个功能模块。首先，利用集成学习中多个基学习器构建共识图；其次，利用图神经网络分析共识图，捕获节点优化表示和节点的聚类结构，并从聚类中挑选高置信度的节点子集及对应的类标签生成监督信息；再次，在此标签监督下，联合其他无标注样本更新集成成员基学习器。交替迭代上述功能块，最终提高无监督聚类的性能。结果 为验证该框架的有效性，在标准数据集（包括图像和文本数据）上设计了一系列实验。实验结果表明，所提方法在性能上一致优于现有聚类方法。尤其是在MNIST-Test （modified national institute of standards and technology database）上，本文方法实现了97.78%的准确率，比已有最佳方法高出3.85%。结论 该方法旨在利用图表示学习提升自监督学习中监督信息捕获的能力，监督信息的有效获取进一步强化了集成学习中成员构建的能力，最终提升了无监督海量数据本质结构的挖掘性能。;Objective Clustering is focused on machine learning-related data segmentation for multiple datasets. Its applications are in relevant to such domains like image segmentation and anomaly detection. In addition，to simplify complex tasks optimize its performance，clustering is used in data preprocessing tasks of those are data sub-blocks segmentation， pseudo-labels generation，and abnormal points-removal. Self-supervised learning has become an essential technique for massive data analysis. However，it is challenged to extract effective supervision information and analyze the input data. Method A consensus graph learning based self-supervised ensemble clustering（CGL-SEC）framework is developed. It consists of three main modules：1）to construct the consensus graph based on several ensemble components（i. e. ，the basic clustering methods）. 2）to extract the supervision information by learning the consensus graph representation，and 3）its node clustering results，where the subset of nodes with the high-confidence are selected as labeled samples. To optimize the ensemble components and the corresponding consensus graph，t basic clustering methods are re-trained in related the option of samples-labeled and other samples-unlabeled. The final clustering results can be optimized iteratively until the learning process converges. Result A series of experiments are carried out on benchmarks，including both image and textual datasets. Especially，CGL-SEC is 3. 85% over baseline in terms of clustering evaluation metric on themodified national institute of standards and technology database（MNIST-Test）. First，to optimize data representation and cluster assignment at the same time，deep embedding clustering can be focused on data itself as the supervision information and auto-encoder with the reconstruction loss is pre-trained. The soft cluster assignment of features-embedded is then calculated，and the KL（Kullback-Leibler）divergence is minimized between the soft cluster assignment and the auxiliary target distribution. To improve the performance of the model further，following deep clustering network（DCN）can use hard clustering instead of soft allocation，and local constraints are applied by improved deep embedding clustering（IDEC）. The pseudo-label strategy is implemented as a self-supervised learning method that uses the prediction results of the neural network as the label to simulate the supervision information compared to using data itself as the supervision information. Deepcluster-based K-means clustering is used to generate pseudo-labels to guide the training of convolutional networks. However，the generated pseudo-labels have lower confidence and are prone to trivial solutions in the initial stage of network training. Deep embedding clustering with data augmentation（DEC-DA）and MixMatch-based prediction of data-enhanced samples are used as the supervision information of the original data，which improves the accuracy of the supervision information to a certain extent，but this method is difficult to extend to text and other fields. Deep adaptive clustering-based high-confidence pseudo-label subsets-selected are iteratively trained the network in the prediction results，but lowconfidence samples-involved data distribution information is ignored. Pseudo-semi-supervised clustering votes are used to select a subset of high-confidence pseudo-labels，and all samples are used to train semi-supervised neural network. Although the ensemble strategy can improve the confidence of the pseudo-label，the voting strategy is concerned of category representation only without the feature representation of the sample itself，which can reduce the clustering performance in some cases. The ensemble learning is regarded as a representative machine learning method that reflects the ability of group intelligence，whereas a learning method can improve the overall prediction performance via multiple base learners training and their coordinated prediction results. In pseudo-label-based clustering tasks，it can coordinate multiple base learners to obtain high-confidence pseudo-labels. However，the effectiveness of the supervision information acquisition is still to be resolved. The category information of the sample is considered for current pseudo-label-based ensemble clustering method only when the label is captured and some effective information are ignored like the feature representation of the sample itself and the clustering structure between samples. Conclusion Graph neural network is composed of content information of nodes and the structural information between nodes at the same time. To design a self-supervised ensemble clustering method based on consensus graph representation learning，it is required to make full use of sample features and relationships between samples in ensemble learning. To obtain higher confidence pseudo-labels as supervised information and improve the performance of self-supervised clustering，it is necessary to mine global and local information at the same time. We illustrate a learnable data ensemble representation through graph neural network. The confidence of pseudo-labels is improved，and the entire model is trained in self-supervision iteratively. To be summarized：1）Commonly-used consensus graph learning-integrated clustering framework is developed，which can use multi-level information like clusteringintegrated sample characteristics and category structure. 2）Self-supervision method is proposed，which uses graph neural network to mine the global and local information of the consensus graph，and high-confidence pseudo-labels are obtained as supervised information. 3）Experiments are demonstrated that the consensus graph learning ensemble clustering method has its potentials on image and text datasets.

The classic Fuzzy C-means (FCM) algorithm has limited clustering performance and is prone to misclassification of border points. This study offers a bi-directional FCM clustering ensemble approach that takes local information into account (LI_BIFCM) to overcome these challenges and increase clustering quality. First, various membership matrices are created after running FCM multiple times, based on the randomization of the initial cluster centers, and a vertical ensemble is performed using the maximum membership principle. Second, after each execution of FCM, multiple local membership matrices of the sample points are created using multiple K-nearest neighbors, and a horizontal ensemble is performed. Multiple horizontal ensembles can be created using multiple FCM clustering. Finally, the final clustering results are obtained by combining the vertical and horizontal clustering ensembles. Twelve data sets were chosen for testing from both synthetic and real data sources. The LI_BIFCM clustering performance outperformed four traditional clustering algorithms and three clustering ensemble algorithms in the experiments. Furthermore, the final clustering results has a weak correlation with the bi-directional cluster ensemble parameters, indicating that the suggested technique is robust.

Ensemble clustering has become a widely used technique for improving robustness and accuracy by combining multiple clustering results. However, traditional ensemble clustering methods often fail to provide fair treatment between groups defined by sensitive attributes. Central to many ensemble methods is the symmetric co-association matrix, which captures pairwise similarity between data points based on their co-occurrence across base clusterings. This paper introduces a fair ensemble clustering method based on the symmetric co-association matrix. The proposed method integrates fairness constraints into the objective function of the ensemble process, using the results from base clusterings that lack fairness considerations as input. The optimization is performed iteratively, and the final clustering results are represented directly by a label matrix obtained efficiently using a coordinate descent approach. By integrating fairness into the clustering process, the method avoids the need for post-processing to achieve fair results. Comprehensive experiments on both real-world and synthetic datasets validate the effectiveness and practicality of the proposed method.

A robust adaptive clustering analysis method for automatic identification of clusters

Cluster ensemble is a popular method in data mining to discover hidden structures in unlabeled datasets. Some studies have indicated that choosing base clusters with more significant distinction and higher quality can effectively improve the final clustering results. However, the existing base clustering optimization methods consider the diversity and the quality of base clustering separately. Inspired by the three-way decisions, we introduce a three-way optimization model that considers both quality and diversity (3W_CBQD). With two thresholds α and β where α>β, the model calculates the product of the diversity and quality among base clusters as the quality metric and evaluates the quality metric by the thresholds to make decisions. A base cluster will be retained if the quality metric is greater than or equal to the threshold α, removed if the quality metric is less than or equal to the threshold, and recalculated otherwise. This procedure will be repeated until either no base clusters are removed or the number of repeats reaches a specified threshold. Finally, by comparing with some traditional clustering algorithms and clustering ensemble methods, our model shows an effective improvement in the final clustering results with good scalability.

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Previous multi-view spectral clustering methods are a two-step strategy, which first learns a fixed common representation (or common affinity matrix) of all the views from original data and then conducts k-means clustering on the resulting common affinity matrix. The two-step strategy is not able to output reasonable clustering performance since the goal of the first step (i.e., the common affinity matrix learning) is not designed for achieving the optimal clustering result. Moreover, the two-step strategy learns the common affinity matrix from original data, which often contain noise and redundancy to influence the quality of the common affinity matrix. To address these issues, in this paper, we design a novel One-step Multi-view Spectral Clustering (OMSC) method to output the common affinity matrix as the final clustering result. In the proposed method, the goal of the common affinity matrix learning is designed to achieving optimal clustering result and the common affinity matrix is learned from low-dimensional data where the noise and redundancy of original high-dimensional data have been removed. We further propose an iterative optimization method to fast solve the proposed objective function. Experimental results on both synthetic datasets and public datasets validated the effectiveness of our proposed method, comparing to the state-of-the-art methods for multi-view clustering.

Clustering analysis elicits the natural groupings of a dataset without requiring information about the sample class and has been widely used in various fields. Although numerous clustering algorithms have been proposed and proven to perform reasonably well, no consensus exists about which one performs best in real situations. In this study, we propose a nonparametric clustering method based on recursive binary partitioning that was implemented in a classification and regression tree model. The proposed clustering algorithm has two key advantages: (1) users do not have to specify any parameters before running it; (2) the final clustering result is represented by a set of if---then rules, thereby facilitating analysis of the clustering results. Experiments with the simulations and real datasets demonstrate the effectiveness and usefulness of the proposed algorithm.

This paper introduces a new approach for clustering large datasets based on spectral clustering and topological unsupervised learning. Spectral clustering method needs to construct an adjacency matrix and calculate the eigen-decomposition of the corresponding Laplacian matrix [4] which are computational expensive and is not easy to apply on large-scale data sets. Contrarily, the topological learning i.e. SOM method allows a projection of the dataset in low dimensional spaces that make it easy to use for very large datasets. The prototypes matrix weighted by the neighbourhood function will be used in this work to reduce the computational time of the clustering algorithm and to add the topological information to the final clustering result. We illustrate the power of this method with several real datasets. The results show a good quality of clustering results and a higher speed.

Multiview clustering (MVC) is a popular research topic in the fields of data mining and pattern recognition, which focuses on fully exploring and employing the correlations between views to jointly discover a consistent cluster structure across data. Typically, weighted MVC is a common clustering method aimed at learning the importance or weights of each view and applying them to explore the complementary information between views. However, current weighted MVCs primarily focus on the quality of each view while overlooking the crucial role of pseudo-label-based self-supervision in weight learning. In addition, most weighted MVCs only use a weighting mechanism to utilize complementary features without sufficiently considering the consistency relationship between the clustering results of individual views and the final clustering result. Aiming to solve the above problems, this article proposes a structure-enhanced self-supervised weighted information bottleneck (S2WIB) method for MVC. Specifically, the S2WIB method establishes a view-weight learning mechanism that leverages both the view-contained information and the self-supervised information to learn view weights, and then integrates the weighted information from different views. Meanwhile, based on the information bottleneck (IB) theory, it explores view correlations from two perspectives, namely complementary information and consistent view cluster structure information, thereby fully exploiting the potential information contained in multiview data. Experiments on various multiview text datasets, multifeature image datasets, multiangle video datasets, multimodal text-image datasets, as well as large-scale datasets and biological multiomics datasets, demonstrate that the S2WIB method performs effectively and exhibits superiority across various fields.

Incremental fuzzy cluster ensemble learning based on rough set theory

Ensemble and Consensus Clustering address the problem of unifying multiple clustering results into a single output to best reflect the agreement of input methods. They can be used to obtain more stable and robust clustering results in comparison with a single clustering approach. In this study, we propose a novel subset selection method that looks at controlling the number of clustering inputs and datasets in an efficient way. The authors propose a number of manual selection and heuristic search techniques to perform the selection. Our investigation and experiments demonstrate very promising results. Using these techniques can ensure better selection methods and datasets for Ensemble and Consensus Clustering and thus more efficient clustering results.

Weighted adaptively ensemble clustering method based on fuzzy Co-association matrix