Clustering microarray gene expression data using weighted Chinese restaurant process

With the development of shale gas reservoirs, various fracture propagation models have been developed to predict hydraulic fracture geometry by considering the stress shadow effects. Also, many researchers have been working on development of various production-simulation approaches to simulate production from the complex fracture geometries. The objective of this study is to combine the fracture propagation modeling and reservoir simulation to optimize production in shale gas reservoirs. First, we have developed a semi-analytical model to simulate shale gas production from the complex non-planar fracture geometry with varying fracture width and fracture permeability along fracture length. The important gas transport mechanisms including gas slippage, gas diffusion, and gas desorption are considered. Second, we integrate the semi-analytical model with a three-dimensional fracture propagation model, which fully couples elastic deformation of the rock and fluid flow to simulate complex hydraulic fracture propagation, to optimize fracture treatment design in Marcellus shale gas reservoirs. Specifically, the fracture propagation model is utilized to predict the more-realistic non-planar fracture geometry with varying fracture width and fracture permeability along fracture length. In this study, the effect of varying number of perforation clusters within a given stage ranging from 2 to 5 on the fracture propagation was investigated. Three values of stage spacing were considered including 100 ft, 200 ft, and 300 ft. After predicting the non-planar fracture geometry, the semi-analytical model is employed to simulate production from such fracture geometries. According to the well productivity, the optimal number of clusters per stage for varying stage spacing is discussed and determined. In addition, we compared the well performance with the optimal number of perforation clusters for the stage spacing of 100 ft, 200 ft, and 300 ft under a given horizontal well length of 2,500 ft. Furthermore, we used the integrated approach to analyze a well performance from Marcellus shale gas reservoirs. Through the field case study, the more-realistic non-planar fracture geometry was quantified. Also, the difference between the non-planar fracture geometry and ideal planar fracture geometry with equal fracture width and fracture half-length was compared. Finally, the optimal fracture treatment design for the field well development was recommended. This work can provide new insights into optimization of number of perforation clusters per stage within a given fracture stage and a better understanding of the non-planar fracture geometry in shale gas reservoirs.

This article deals with the development of an improved clustering technique for categorical data that is based on the identification of points having significant membership to multiple classes. Cluster assignments of such points are difficult, and they often affect the actual partitioning of the data. As a consequence, it may be more effective if the points that are associated with maximum confusion regarding their cluster assignments are first identified and excluded from consideration at the first stage of algorithm and these points may be assigned to one of the identified clusters based on an ANN classifier at the second stage of this algorithm. At the first stage of this algorithm we are using our developed genetic algorithm and simulated annealing based fuzzy clustering and well known Fuzzy C-Medoids algorithm when the number of clusters is known a priori. The performance of the proposed clustering algorithms has been compared with the average linkage hierarchical clustering algorithm, in addition to the genetic algorithm based fuzzy clustering, simulated annealing based fuzzy clustering and Fuzzy C-Medoids with ANN for a variety of artificial and real life categorical data sets. Also statistical significance test have been performed to establish the superiority of the proposed algorithm.

To find an optimal cluster size and number of clusters for a reasonable estimate of the prevalence of cataract blindness in people aged > or = 50 years in 19 rural districts of a state in India. Cluster sampling methodology was used in 19 rural districts of Karnataka State, India. In each district, 15 clusters were randomly selected and 90 people aged > or = 50 years were examined in each cluster. As a result the visual acuity and lens status of a total of 22,218 people were assessed. For each district, the design effect for cluster size ranging from 20 to 90 was calculated and the optimal cluster size and the required number of clusters to achieve an accuracy of 1% errors and 80% confidence was assessed. The age and gender adjusted prevalence of cataract blindness varied from 1.58% to 7.24%, which justifies district level surveys. The design effect is nearly 1.5 for clusters of sizes 30 and 40. With an average prevalence of 4.93% with 1% error and 80% confidence level, the optimal number of clusters is 37 and 28 for a cluster size of 30 and 40 respectively and the average sample size for a district around 1100. Rapid assessments for cataract blindness in those aged > or = 50 years can be conducted at district level in India with existing resources and at affordable costs. These provide reliable data, essential for effective monitoring and planning. Other parameters, for instance, surgical coverage can also be assessed. The availability of standardized software for data entry and analysis and strict adherence to survey procedures is essential.

BackgroundCluster analysis is the most common unsupervised method for finding hidden groups in data. Clustering presents two main challenges: (1) finding the optimal number of clusters, and (2) removing “outliers” among the objects being clustered. Few clustering algorithms currently deal directly with the outlier problem. Furthermore, existing methods for identifying the number of clusters still have some drawbacks. Thus, there is a need for a better algorithm to tackle both challenges.ResultsWe present a new approach, implemented in an R package called Thresher, to cluster objects in general datasets. Thresher combines ideas from principal component analysis, outlier filtering, and von Mises-Fisher mixture models in order to select the optimal number of clusters. We performed a large Monte Carlo simulation study to compare Thresher with other methods for detecting outliers and determining the number of clusters. We found that Thresher had good sensitivity and specificity for detecting and removing outliers. We also found that Thresher is the best method for estimating the optimal number of clusters when the number of objects being clustered is smaller than the number of variables used for clustering. Finally, we applied Thresher and eleven other methods to 25 sets of breast cancer data downloaded from the Gene Expression Omnibus; only Thresher consistently estimated the number of clusters to lie in the range of 4–7 that is consistent with the literature.ConclusionsThresher is effective at automatically detecting and removing outliers. By thus cleaning the data, it produces better estimates of the optimal number of clusters when there are more variables than objects. When we applied Thresher to a variety of breast cancer datasets, it produced estimates that were both self-consistent and consistent with the literature. We expect Thresher to be useful for studying a wide variety of biological datasets.

A fundamental problem in data clustering is how to determine the correct number of clusters. The k-adaptive medoid set ant colony optimization (ACO) clustering (METACOC-K) algorithm is superior in solving clustering problems. However, METACOC-K does not guarantee in finding the best number of clusters. It assumed the number of clusters based on an adaptive parameter strategy that lacks feedback learning. This has restrained the algorithm in producing compact clusters and the optimal number of clusters. In this paper, a self-adaptive ACO clustering (S-ACOC) algorithm is proposed to produce the optimal number of clusters by incorporating a self-adaptive parameter strategy. The S-ACOC algorithm is a centroid-based algorithm that automatically adjusts the number of clusters during the algorithm run. The selection of the number of clusters is based on a construction graph that reflects the influence of a pheromone in algorithm learning. Experiments were conducted on real-world datasets to evaluate the performance of the proposed algorithm. The external evaluation metrics (purity, F-measure, and entropy) were used to compare the results of the proposed algorithm with other swarm clustering algorithms, including a genetic algorithm (GA), particle swarm optimization (PSO), and METACOC-K. Results showed that S-ACOC provides higher purity (50%) and lower entropy (40%) than GA, PSO, and METACOC-K. Experiments were also performed on several predefined clusters, and results demonstrate that the S-ACOC algorithm is superior to GA, PSO, and METACOC-K. Based on the superior performance, S-ACOC can be used to solve clustering problems in various application domains.

Purpose:Radiogenomics aims to establish relationships between patient genotypes and imaging phenotypes. An open question remains on how best to integrate information from these distinct datasets. This work investigates if similarities in genetic features across patients correspond to similarities in PET‐imaging features, assessed with various clustering algorithms.Methods:[18F]FDG PET data was obtained for 26 NSCLC patients from a public database (TCIA). Tumors were contoured using an in‐house segmentation algorithm combining gradient and region‐growing techniques; resulting ROIs were used to extract 54 PET‐based features. Corresponding genetic microarray data containing 48,778 elements were also obtained for each tumor. Given mismatch in feature sizes, two dimension reduction techniques were also applied to the genetic data: principle component analysis (PCA) and selective filtering of 25 NSCLC‐associated genes‐ofinterest (GOI). Gene datasets (full, PCA, and GOI) and PET feature datasets were independently clustered using K‐means and hierarchical clustering using variable number of clusters (K). Jaccard Index (JI) was used to score similarity of cluster assignments across different datasets.Results:Patient clusters from imaging data showed poor similarity to clusters from gene datasets, regardless of clustering algorithms or number of clusters (JImean= 0.3429±0.1623). Notably, we found clustering algorithms had different sensitivities to data reduction techniques. Using hierarchical clustering, the PCA dataset showed perfect cluster agreement to the full‐gene set (JI =1) for all values of K, and the agreement between the GOI set and the full‐gene set decreased as number of clusters increased (JI=0.9231 and 0.5769 for K=2 and 5, respectively). K‐means clustering assignments were highly sensitive to data reduction and showed poor stability for different values of K (JIrange: 0.2301–1).Conclusion:Using commonly‐used clustering algorithms, we found poor agreement in patient characterization from genomic versus imaging data. Interpretation of radiogenomic associations was found to be dependent on clustering algorithms and sensitive to data reduction techniques.

Background This paper presents an optimized clustering approach applied to image segmentation. Accurate image segmentation impacts many fields like medical, machine vision, object detection. Applications involve tumor detection, face detection and recognition, and video surveillance. Methods The developed approach is based on obtaining an optimum number of clusters and regions of an image. We combined Region-based and contour-based approaches. Initial rough regions are obtained using edge detection. We have used Gabor wavelets for texture classification and spatial resolutions. Color frequencies are also used to determine the number of clusters of the Fuzzy c-means (FCM) algorithm which gave an optimum number of clusters or regions. Results We have compared our approach with other similar wavelet and clustering techniques. Our algorithm gave better values for segmentation metrics like SNR, PSNR, and MCC. Conclusions Optimizing the number of clusters or regions has a significant effect on the performance of the image segmentation techniques. This will result in better detection and localization of the segmentation-based application.

Identification of coherent generators (CGs) is necessary for the area-based monitoring and protection system of a wide area power system. Synchrophasor has enabled smarter monitoring and control measures to be devised; hence, measurement-based methodologies can be implemented in online applications to identify the CGs. This paper presents a new framework for coherency identification that is based on the dynamic coupling of generators. A distance matrix that contains the dissimilarity indices between any pair of generators is constructed from the pairwise dynamic coupling of generators after the post-disturbance data are obtained by phasor measurement units (PMUs). The dataset is embedded in Euclidean space to produce a new dataset with a metric distance between the points, and then the support vector clustering (SVC) technique is applied to the embedded dataset to identify the final clusters of generators. Unlike other clustering methods that need a priori knowledge about the number of clusters or the parameters of clustering, this information is set in an automatic search procedure that results in the optimal number of clusters. The algorithm is verified by time-domain simulations of defined scenarios in 39 bus and 118 bus test systems. Finally, the clustering result of 39 bus systems is validated by cluster validity measures, and a comparative study investigates the efficacy of the proposed algorithm to cluster the generators with an optimal number of clusters and also its computational efficiency compared with other clustering methods.

This paper proposes a method of determining the optimal number of clusters dividing the multiple transformations for the purpose of the efficient processing of query against the results of applying the transformations to time series. In this paper, the moving average is used as a transformation for simplicity. The model of query time to the number of clusters is constructed for determining the optimal number of clusters. As the query time could be represented with the concave function of the number of clusters, it is shown that the optimal number of clusters for the best query time can be obtained. The verification experiment confirms the validity of the model constructed. It is revealed that the optimal number of clusters could be determined by the times obtained from a single query execution.

In this study, clustering based method for activation detection in functional magnetic resonance imaging (fMRI) is employed. Moreover, some features are obtained by fitting two models namely FIR filter and Gamma function, to hemodynamic response function (HRF). After applying clustering methods (that require number of clusters as an input) to feature space, our simulations show that number of clusters can affect activation detection significantly. Therefore a newly proposed clustering algorithm namely evolving neural gas (ENG) that gives optimal number of clusters is exploited. In addition to ENG, the result of four clustering algorithms namely k-means, fuzzy C-means, neural gas, and clara in different number of clusters are evaluated. The results show that the best activation detection is taken place using obtained optimal number of clusters.

Clustering is a useful exploratory technique for the analysis of gene expression data. Many different heuristic clustering algorithms have been proposed in this context. Clustering algorithms based on probability models offer a principled alternative to heuristic algorithms. In particular, model-based clustering assumes that the data is generated by a finite mixture of underlying probability distributions such as multivariate normal distributions. The issues of selecting a 'good' clustering method and determining the 'correct' number of clusters are reduced to model selection problems in the probability framework. Gaussian mixture models have been shown to be a powerful tool for clustering in many applications. We benchmarked the performance of model-based clustering on several synthetic and real gene expression data sets for which external evaluation criteria were available. The model-based approach has superior performance on our synthetic data sets, consistently selecting the correct model and the number of clusters. On real expression data, the model-based approach produced clusters of quality comparable to a leading heuristic clustering algorithm, but with the key advantage of suggesting the number of clusters and an appropriate model. We also explored the validity of the Gaussian mixture assumption on different transformations of real data. We also assessed the degree to which these real gene expression data sets fit multivariate Gaussian distributions both before and after subjecting them to commonly used data transformations. Suitably chosen transformations seem to result in reasonable fits. MCLUST is available at http://www.stat.washington.edu/fraley/mclust. The software for the diagonal model is under development. kayee@cs.washington.edu. http://www.cs.washington.edu/homes/kayee/model.

Averaging amplitudes over consecutive time samples (i.e., time window) is widely used to calculate the peak amplitude of event-related potentials (ERPs). Cluster analysis of the spatio-temporal ERP data is a promising tool to determine the time window of an ERP of interest. However, determining an appropriate number of clusters to optimally represent ERPs is still challenging. Here, we develop a new method to estimate the optimal number of clusters utilizing consensus clustering. Various polarity dependent clustering methods, namely, k-means, hierarchical clustering, fuzzy c-means, self-organizing map, spectral clustering, and Gaussian mixture model, are used to configure consensus clustering after assessing them individually. When a range of clusters is applied many times, the optimal number of clusters should correspond to the expectation, which is the average of the obtained mean inner-similarities of estimated time windows across all conditions and groups converge in the satisfactory thresholds. In order to assess our method, the proposed method has been applied to simulated data and prospective memory experiment ERP data aimed to qualify N2 and P3, and N300 and prospective positivity components, respectively. The results of determining the optimal number of clusters meet at six cluster maps for both ERP data. In addition, our results revealed that the proposed method could be reliably applied to ERP data to determine the appropriate time window for the ERP of interest when the measurement interval is not accurately defined.

Clustering is a basic data mining operation that groups data points with similar inherent structure. Among clustering techniques, Density Peak Clustering (DPC) is notable for detecting clusters of arbitrary shapes without requiring the number of clusters in advance. However, DPC suffers from parameter-sensitive local density estimation, inadequate handling of noise and boundary points, and rigid binary cluster assignments. To overcome these limitations, fuzzy logic is often embedded in DPC, but the selection of the most appropriate membership function is still an open research question. In this paper, we propose an enhanced DPC variant with three methodological improvements: (i) substituting cutoff distance-based density estimation with a K-Nearest Neighbors (KNN) functioned kernel to achieve stable and robust density estimation, (ii) adding a noise parameter Lambda ( λ ) to better identify and distinguish noise and boundary points, and (iii) using fuzzy membership functions to allow for probabilistic and soft cluster assignments of data points. Comparative analyses of four fuzzy membership functions (i.e., Gaussian, Combination Gaussian, Trapezoidal, and Triangular) were performed on real, synthetic, and high-dimensional datasets. The experimental results indicate that the proposed algorithm with the Gaussian fuzzy membership function outperforms other membership functions and traditional clustering algorithms including the original DPC, FKNN, and DPC-DBFN in terms of clustering accuracy, stability, adaptability, and noise resistance, and is therefore part of our final suggested method. The proposed method overall relieves original DPC shortcomings by strengthening adaptability, accuracy, and applicability to complex real-world clustering tasks. • Substituting standard cutoff-distance-based kernel with KNN for enhanced clustering. • Introducing λ parameter to manage noise and boundary points for separating clusters. • Introducing fuzzy neighborhood kernel for soft cluster membership assignment. • Investigating the impact of several fuzzy membership functions on clustering. • Proposed strategy outperforms original DPC, FKNN, and DPC-DBFN on datasets.

Fuzzy clustering is a popular unsupervised learning method that is used in cluster analysis. Fuzzy clustering allows a data point to belong to two or more clusters. Fuzzy c-means is the most well-known method that is applied to cluster analysis, however, the shortcoming is that the number of clusters need to be predefined. This paper proposes a clustering approach based on Particle Swarm Optimization (PSO). This PSO approach determines the optimal number of clusters automatically with the help of a threshold vector. The algorithm first randomly partitions the data set within a preset number of clusters, and then uses a reconstruction criterion to evaluate the performance of the clustering results. The experiments conducted demonstrate that the proposed algorithm automatically finds the optimal number of clusters. Furthermore, to visualize the results principal component analysis projection, conventional Sammon mapping, and fuzzy Sammon mapping were used

The real-time system of the JT-60 has a cascade structure with the major and the minor control loops. The multi-variable feedback control is adopted in the major control loop in order to adjust non-linear effects which will appear in the high-/beta/region. By applying the concept of 'phase shift control' to the control program of the major control loop, a flexible tokamak discharge control is expected to be performed with taking account of machine protection.