• Home
  • Search
  • Thresher: determining the number of clusters while removing outliers
  • Cite Icon50
  • https://doi.org/10.1186/s12859-017-1998-9Copy DOI Icon

Thresher: determining the number of clusters while removing outliers

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

BackgroundCluster analysis is the most common unsupervised method for finding hidden groups in data. Clustering presents two main challenges: (1) finding the optimal number of clusters, and (2) removing “outliers” among the objects being clustered. Few clustering algorithms currently deal directly with the outlier problem. Furthermore, existing methods for identifying the number of clusters still have some drawbacks. Thus, there is a need for a better algorithm to tackle both challenges.ResultsWe present a new approach, implemented in an R package called Thresher, to cluster objects in general datasets. Thresher combines ideas from principal component analysis, outlier filtering, and von Mises-Fisher mixture models in order to select the optimal number of clusters. We performed a large Monte Carlo simulation study to compare Thresher with other methods for detecting outliers and determining the number of clusters. We found that Thresher had good sensitivity and specificity for detecting and removing outliers. We also found that Thresher is the best method for estimating the optimal number of clusters when the number of objects being clustered is smaller than the number of variables used for clustering. Finally, we applied Thresher and eleven other methods to 25 sets of breast cancer data downloaded from the Gene Expression Omnibus; only Thresher consistently estimated the number of clusters to lie in the range of 4–7 that is consistent with the literature.ConclusionsThresher is effective at automatically detecting and removing outliers. By thus cleaning the data, it produces better estimates of the optimal number of clusters when there are more variables than objects. When we applied Thresher to a variety of breast cancer datasets, it produced estimates that were both self-consistent and consistent with the literature. We expect Thresher to be useful for studying a wide variety of biological datasets.

Loading PDF

Similar Papers
  • Research Article
  • Citations3

Improved Self-Adaptive ACS Algorithm to Determine the Optimal Number of Clusters

  • Jun 21, 2021
  • International Journal on Advanced Science, Engineering and Information Technology
  • Ayad Mohammed Jabbar +2
  • Conference Article
  • Citations5

An Integrated Approach to Optimize Production in Marcellus Shale Gas Reservoirs

  • Sep 28, 2015
  • Wei Yu +2
  • Conference Article
  • Citations1

Clustering method for fMRI activation detection using optimal number of clusters

  • Apr 01, 2009
  • Ali Taalimi +2
  • Research Article
  • Citations89

Rapid assessment of prevalence of cataract blindness at district level.

  • Oct 01, 1997
  • International Journal of Epidemiology
  • H Limburg
  • Research Article

Performance Improvement of Clustering Affinity Propagation Method using Principal Component Analysis

  • Jun 01, 2020
  • Journal of Physics: Conference Series
  • Jasael Simanullang +2
  • Conference Article
  • Citations10

Clustering Algorithms and Validation Indices for mmWave Radio Multipath Propagation

  • Apr 01, 2019
  • Miead Tehrani Moayyed +2
  • Research Article
  • Citations1

Optimal Number of Clusters for Fast Similarity Search Considering Transformations of Time Varying Data

  • Jan 01, 2015
  • International Journal of Networked and Distributed Computing
  • Toshiichiro Iwashita +2
  • Conference Article
  • Citations2

An efficient and robust combined clustering technique for mining in large spatial databases

  • Nov 01, 2007
  • R S Elhadary +3
  • Research Article
  • Citations33

Particle Swarm Optimization Based Fuzzy Clustering Approach to Identify Optimal Number of Clusters

  • Jan 01, 2014
  • Journal of Artificial Intelligence and Soft Computing Research
  • Min Chen +1
  • Book Chapter
  • Citations3

A Novel Hybrid Structure for Clustering

  • Jan 01, 2008
  • Mahdi Yazdian Dehkordi +2
  • Conference Article
  • Citations4

Multi-PFKCN : A fuzzy possibilistic clustering algorithm based on neural network

  • Jul 01, 2013
  • Balkis Abidi +1
  • Research Article
  • Citations12

Optimal Number of Clusters by Measuring Similarity Among Topographies for Spatio-Temporal ERP Analysis.

  • Jul 18, 2022
  • Brain Topography
  • Reza Mahini +9
  • Research Article
  • Citations30

Improving k-means through distributed scalable metaheuristics

  • Feb 07, 2017
  • Neurocomputing
  • G.V Oliveira +3
  • Research Article
  • Citations1

IMPLEMENTATION OF K-MEANS ALGORITHM ELBOW METHOD AND SILHOUETTE COEFFICIENT FOR RAINFALL CLASSIFICATION

  • Aug 29, 2024
  • Proxies : Jurnal Informatika
  • Daniel Adrian Setiady +1
  • Research Article
  • Citations2

Performances of Clustering Methods Considering Data Transformation and Sample Size: An Evaluation with Fisheries Survey Data

  • May 02, 2020
  • Journal of Ocean University of China
  • Jia Wo +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.