Improving image clustering: An unsupervised feature weight learning framework

Data analytics methods in the clinical domain are challenging to put into practice. Unsupervised learning provides opportunity for giving the level of personalization in evidence based decision-making that can otherwise only be achieved through the use of prediction models, by helping doctors gaining insights from data. In this context, grouping of clinical subjects, in terms of biomedical information of patients, is an important task for patient cohort identification for comparative effectiveness studies and clinical decision-support applications. It allows the decision-making process to leverage not only on data but also on doctors’ domain knowledge. However, one of the issues that needs to be addressed for a focused and realist unsupervised clustering of clinical subjects, is the fact that in the majority of the cases patients datasets are heterogeneous, i.e. their data features belong to several different feature spaces, e.g. nominal, ordinal, interval or rational, with completely different variation ranges and statistical distributions, affecting clustering quality and performance. In order to use these data measurements properly in an unsupervised manner, their corresponding weights need to be modeled. In this paper, we present a method for learning feature weights on clinical data. We show that learning feature weights is necessary in order to generate meaningful separation of data in high dimensional space. The method is based on silhouette score and principal component analysis, demonstrating its performance on a clinical test dataset.

Developing the utilized intelligent systems is increasingly important to learn effective text representations, especially extract the sentence features. Numerous previous studies have been concentrated on the task of sentence representation learning based on deep learning approaches. However, the present approaches are mostly proposed with the single task or replied on the labeled corpus when learning the embedding of the sentences. In this paper, we assess the factors in learning sentence representation and propose an efficient unsupervised learning framework with multi-task learning (USR-MTL), in which various text learning tasks are merged into the unitized framework. With the syntactic and semantic features of sentences, three different factors to some extent are reflected in the task of the sentence representation learning that is the wording, or the ordering of the neighbored sentences of a target sentence in other words. Hence, we integrate the word-order learning task, word prediction task, and the sentence-order learning task into the proposed framework to attain meaningful sentence embeddings. Here, the process of sentence embedding learning is reformulated as a multi-task learning framework of the sentence-level task and the two word-level tasks. Moreover, the proposed framework is motivated by an unsupervised learning algorithm utilizing the unlabeled corpus. Based on the experimental results, our approach achieves the state-of-the-art performances on the downstream natural language processing tasks compared to the popular unsupervised representation learning techniques. The experiments on representation visualization and task analysis demonstrate the effectiveness of the tasks in the proposed framework in creating reasonable sentence representations proving the capacity of the proposed unsupervised multi-task framework for the sentence representation learning.

Joint Laplacian feature weights learning

Computational methods for identification of cell populations from polychromatic flow cytometry data are changing the paradigm of cytometry bioinformatics. Data clustering is the most common computational approach to unsupervised identification of cell populations from multidimensional cytometry data. However, interpretation of the identified data clusters is labor-intensive. Certain types of user-defined cell populations are also difficult to identify by fully automated data clustering analysis. Both are roadblocks before a cytometry lab can adopt the data clustering approach for cell population identification in routine use. We found that combining recursive data filtering and clustering with constraints converted from the user manual gating strategy can effectively address these two issues. We named this new approach DAFi: Directed Automated Filtering and Identification of cell populations. Design of DAFi preserves the data-driven characteristics of unsupervised clustering for identifying novel cell subsets, but also makes the results interpretable to experimental scientists through mapping and merging the multidimensional data clusters into the user-defined two-dimensional gating hierarchy. The recursive data filtering process in DAFi helped identify small data clusters which are otherwise difficult to resolve by a single run of the data clustering method due to the statistical interference of the irrelevant major clusters. Our experiment results showed that the proportions of the cell populations identified by DAFi, while being consistent with those by expert centralized manual gating, have smaller technical variances across samples than those from individual manual gating analysis and the nonrecursive data clustering analysis. Compared with manual gating segregation, DAFi-identified cell populations avoided the abrupt cut-offs on the boundaries. DAFi has been implemented to be used with multiple data clustering methods including K-means, FLOCK, FlowSOM, and the ClusterR package. For cell population identification, DAFi supports multiple options including clustering, bisecting, slope-based gating, and reversed filtering to meet various autogating needs from different scientific use cases. © 2018 International Society for Advancement of Cytometry.

The ancient Yi books have a long history and are one of the most important cultural heritages of humanity. Currently, Yi character image recognition datasets are all constructed by manual handwriting. Nevertheless, there are significant feature differences between handwritten character images and real ancient manuscript character images. This limits their effectiveness in practical ancient manuscript character recognition scenarios. To address these issues, we propose a novel dataset construction method to construct a real-world character dataset Yi_1047. Our method first uses an unsupervised contrastive learning model to perform accurate feature extraction. It applies this to a large number of unlabeled single-character images from real ancient manuscripts. Then, we use image retrieval and clustering methods to construct a real Yi script single-character dataset. It contains 1047 classes of ancient Yi script characters, totaling 265,094 character images. It is currently the only non-manually handwritten ancient Yi single-character dataset. Additionally, for highly similar characters, we specifically construct a comprehensive list of similar characters. Extensive experimental results demonstrate the effectiveness and superiority of our proposed method for constructing the Yi_1047 dataset. This innovative method not only achieves the digital preservation of ancient Yi script, but our experiments also show that it plays a role in constructing single-character recognition datasets for other ancient books.

With recent deep learning based approaches showing promising results in removing noise from images, the best denoising performance has been reported in a supervised learning setup that requires a large set of paired noisy images and ground truth data for training. The strong data requirement can be mitigated by unsupervised learning techniques, however, accurate modelling of images or noise variances is still crucial for high-quality solutions. The learning problem is ill-posed for unknown noise distributions. This paper investigates the tasks of image denoising and noise variance estimation in a single, joint learning framework. To address the ill-posedness of the problem, we present deep variation prior (DVP), which states that the variation of a properly learnt denoiser with respect to the change of noise satisfies some smoothness properties, as a key criterion for good denoisers. Building upon DVP and under the assumption that the noise is zero mean and pixel-wise independent conditioned on the image, an unsupervised deep learning framework, that simultaneously learns a denoiser and estimates noise variances, is developed. Our method does not require any clean training images or an external step of noise estimation, and instead, approximates the minimum mean squared error denoisers using only a set of noisy images. With the two underlying tasks being considered in a single framework, we allow them to be optimised for each other. The experimental results show a denoising quality comparable to that of supervised learning and accurate noise variance estimates.

We consider the problem of image representation and clustering. Traditionally, an n1 x n2 image is represented by a vector in the Euclidean space R n1 x n2. Some learning algorithms are then applied to these vectors in such a high dimensional space for dimensionality reduction, classification, and clustering. However, an image is intrinsically a matrix, or the second order tensor. The vector representation of the images ignores the spatial relationships between the pixels in an image. In this paper, we introduce a tensor framework for image analysis. We represent the images as points in the tensor space Rn1 mathcal Rn2 which is a tensor product of two vector spaces. Based on the tensor representation, we propose a novel image representation and clustering algorithm which explicitly considers the manifold structure of the tensor space. By preserving the local structure of the data manifold, we can obtain a tensor subspace which is optimal for data representation in the sense of local isometry. We call it TensorImage approach. Traditional clustering algorithm such as k-means is then applied in the tensor subspace. Our algorithm shares many of the data representation and clustering properties of other techniques such as Locality Preserving Projections, Laplacian Eigenmaps, and spectral clustering, yet our algorithm is much more computationally efficient. Experimental results show the efficiency and effectiveness of our algorithm.

With the increasing use of CT in clinical practice, limiting CT radiation exposure to reduce potential cancer risks has become one of the important directions of medical imaging research. As the dose decreases, the reconstructed CT image will be severely degraded by projection noise. As an important method of image processing, supervised deep learning has been widely used in the restoration of low-dose CT(LDCT) in recent years. However, the normal-dose CT (NDCT) corresponding to a specific LDCT (it is regareded as the label of the LDCT, which is necessary for supervised learning) are very difficult to obtain. So that the application of supervised learning methods in LDCT reconstruction is limited. It is necessary to construct a unsupervised deep learning framework for LDCT reconstruction that does not depend on paired LDCT-NDCTdatasets. We presented an unsupervised learning framework for the transferring from the identity mapping to the low-dose recontruction task, called Marginal Distribution Adaptation in Multi-scale (MDAM). For NDCTs as source domain data, MDAM is an identity map with two parts: firstly, it establishes a dimensionality reduction mapping, which can obtain the same feature distribution from NDCTs and LDCTs; and then NDCTs is retrieved by reconstructing the image overview and details from the low-dimensional features. For the purpose of the feature transfer between source domain and target domain (LDCTs), we introduce the multi-scale feature extraction in the MDAM, and then eliminate differences in probability distributions of these multi-scale features between NDCTs and LDCTs through wavelet decomposition and domain adaptationlearning. Image quality evaluation metrics and subjective quality scores show that, as an unsupervised method, the performance of the MDAM approaches or even surpasses some state-of-the-art supervised methods. Especially, MDAM has been favorably evaluated in terms of noise suppression, structural preservation, and lesiondetection. We demonstrated that, the MDAM framework can reconstruct corresponding NDCTs from LDCTs with high accuracy, and without relying on any labeles. Moreover, it is more suitable for clinical application compared with supervised learningmethods. This article is protected by copyright. All rights reserved.

Infinity Laplacian on graphs with gradient terms for image and data clustering

An unsupervised methodology is introduced for quantitative zonal rate allocation in wellbores equipped with temporary or permanent fiber optic cables for acoustic sensing. This approach aims to mitigate key limitations of conventional data-driven methods, which rely heavily on labeled training data and prior knowledge and often struggle to accommodate temporal and spatial variability in acoustic signatures originating from perforations or flow control devices distributed across the lower completion. The proposed framework employs an adaptive, unsupervised learning model capable of autonomous zonal flow allocation using distributed acoustic sensing data. At each rate change event, the algorithm leverages differential acoustic responses along the fiber, identifying how zone-specific signatures evolve relative to total flow variations. This enables the development of unique flow-acoustic relationships for individual zones without prior assumptions. Furthermore, the self-learning model continuously updates zonal contributions, adapting to changing downhole environments and completion characteristics. The model is applicable across all completion types—including open hole scenarios—provided acoustic data is available. Validation using synthetic wells demonstrates successful allocation of flow rates across multiple zones, aligning with expected injection and production profiles. This technique offers potential value for reservoir management, production optimization, and well integrity monitoring by enabling continuous, quantitative zonal flow allocation. The novelty of the method lies in its unsupervised learning framework, which leverages acoustic data during rate change events to dynamically calibrate the relationship between flow-induced noise and zonal flow rates—surpassing the limitations of empirical and supervised models. Beyond allocation, the learned acoustic-flow relationships offer diagnostic value, enabling insights into completion performance and zonal behavior over time.

Unsupervised methods have showed promising results on monocular depth estimation. However, the training data must be captured in scenes without moving objects. To push the envelope of accuracy, recent methods tend to increase their model parameters. In this paper, an unsupervised learning framework is proposed to jointly predict monocular depth and complete 3D motion including the motions of moving objects and camera. (1) Recurrent modulation units are used to adaptively and iteratively fuse encoder and decoder features. This improves the single-image depth inference without overspending model parameters. (2) Instead of using a single set of filters for upsampling, multiple sets of filters are devised for the residual upsampling. This facilitates the learning of edge-preserving filters and leads to the improved performance. (3) A warping-based network is used to estimate a motion field of moving objects without using semantic priors. This breaks down the requirement of scene rigidity and allows to use general videos for the unsupervised learning. The motion field is further regularized by an outlier-aware training loss. Despite the depth model just uses a single image in test time and 2.97M parameters, it achieves state-of-the-art results on the KITTI and Cityscapes benchmarks.

Bag of shape descriptor using unsupervised deep learning for non-rigid shape recognition

In many real applications, the collected data is usually high dimensionality of feature space and no labels are given, which will result in the “curse of dimensionality” problem and bring the challenge for the feature selection. Although tremendous efforts have been made in unsupervised feature selection, the existing methods neglect the robustness of jointly feature selection and clustering. In this paper, we present a robust simultaneous feature selection and clustering method. By introducing a robust loss function, the unsupervised linear discriminant analysis methods with two different strategies (the ratio trace and the trace ratio) are incorporated into this loss function, which can guarantee the selected features with robustness. Then two novel simultaneous unsupervised feature selection and clustering objective functions are presented by the proposed loss function equipped with $\ell_{2,1}$-norm. Two alternative iterative optimization algorithms are developed to solve these two objective functions. The comparative experimental results conducted on eight UCI datasets show that the proposed methods are better than other competing methods.

Data clustering aims to group the input data instances into certain clusters according to the high similarity to each other, and it could be regarded as a fundamental and essential immediate or intermediate task that appears in areas of machine learning, pattern recognition, and information retrieval. Clustering algorithms based on graph regularized extensions have accumulated much interest for a couple of decades, and the performance of this category of approaches is largely determined by the data similarity matrix, which is usually calculated by the predefined model with carefully tuned parameters combination. However, they may lack a more flexible ability and not be optimal in practice. In this paper, we consider both discriminative information as well as the data manifold in a matrix factorization point of view, and propose an adaptive local learning regularized nonnegative matrix factorization (ALLRNMF) approach for data clustering, which assumes that similar instance pairs with a smaller distance should have a larger probability to be assigned to the probabilistic neighbors. ALLRNMF simultaneously learns the data similarity matrix under the assumption and performs the nonnegative matrix factorization. The constraint of the similarity matrix encodes both the discriminative information as well as the learned adaptive local structure and benefits the data clustering on manifold. In order to solve the optimization problem of our approach, an effective alternative optimization algorithm is proposed such that our objective function could be decomposed into several subproblems that each has an optimal solution, and its convergence is theoretically guaranteed. Experiments on real-world benchmark datasets demonstrate the superior performance of our approach against the existing clustering approaches.

DBSCAN is arguably the most popular density-based clustering algorithm, and it is capable of recovering non-spherical clusters. One of its main weaknesses is that it treats all features equally. In this paper, we propose a density-based clustering algorithm capable of calculating feature weights representing the degree of relevance of each feature, which takes the density structure of the data into account. First, we improve DBSCAN and introduce a new algorithm called DBSCANR. DBSCANR reduces the number of parameters of DBSCAN to one. Then, a new step is introduced to the clustering process of DBSCANR to iteratively update feature weights based on the current partition of data. The feature weights produced by the weighted version of the new clustering algorithm, W-DBSCANR, measure the relevance of variables in a clustering and can be used in feature selection in data mining applications where large and complex real-world data are often involved. Experimental results on both artificial and real-world data have shown that the new algorithms outperformed various DBSCAN type algorithms in recovering clusters in data.