Self-organized hierarchical k-means file clustering algorithm based on P2P sharing directories and its application

This study aims to cluster 65 countries based on PISA results. In the study, PISA results (Science-Mathematics-Reading) published by OECD in 2015 and 2018 were used. The main purpose of the analysis is to apply cluster analysis using a multivariate data structure to identify similarities and differences in education systems between countries. In this analysis, the k-means method and the hierarchical clustering algorithm were used to group countries into specific groups, so that countries with similar educational performance were included in the same cluster. In addition, Dunn, Connectivity and Silhouette indexes were used to increase the reliability of the analysis and to determine the optimal number of clusters. According to the validation indexes, k-means method with k=2 was used for 2015 PISA scores while hierarchical clustering algorithm with k=2 was used for 2018 PISA scores. In 2015, Turkey was the only country that changed clusters between the countries clustered according to their PISA scores and the countries clustered according to their PISA scores in 2018, and the reasons for this change were discussed. It is also observed that Turkey was in Cluster-1 in 2015, which includes countries with lower performance, and in Cluster-2 in 2018, which includes countries with higher performance. The clustering methods and indexes used provide a more robust and informed interpretation of the results obtained and make an important contribution to understanding the education systems of countries based on PISA results and grouping countries with similar performance.

Clustering is one of the techniques used to obtain useful information from web log file for better understanding of customer behaviour. Two clustering techniques that commonly used are Greedy Hierarchical Item Set-Based Clustering (GHIC) algorithm and Hierarchical Clustering Algorithm (HCA). The algorithms, however, have its weaknesses in terms of processing times and time complexity. This paper proposes a new approach called Hierarchical Pattern-Based Clustering (HPBC) algorithm to improve the processing times based on the difference of mean support values of each cluster. The simulation revealed that the proposed algorithm outperformed the HCA and GHIC up to 100% and 50% respectively, with less time complexity.

Currently, the data that dealt by traditional text clustering methods is small. Text representation model that text clustering used is traditional vector space model (VSM). The traditional text clustering has defect of low efficiency when processing big data. The quality is bad when using traditional VSM model for text representation. To solve these two problems, this paper puts forward a VSM model for text representation, which is improved by Term Frequency(TF), and using Mahout to text clustering. Experimental results show that use Mahout for text clustering has a higher efficiency than traditional text clustering; this novel VSM model can improve the text clustering quality.

This work aims to expand the knowledge of the area of data analysis through persistence homology and representations of directed graphs. To be specific, we looked for how we can analyze homology cluster groups using agglomerative Hierarchical Clustering algorithms and methods. Additionally, the Wine data, which is offered in R studio, was analyzed using various cluster algorithms such as Hierarchical Clustering, K-Means Clustering, and PAM Clustering. The goal of the analysis was to find out which cluster's method is proper for a given numerical dataset. We tried to find the agglomerative hierarchical clustering method by testing the data that will be the optimal clustering algorithm among these three; K-Means, PAM, and Random Forest methods. By comparing each model's accuracy value with cultivar coefficients, we concluded that K-Means methods are the most helpful when working with numerical variables. On the other hand, PAM clustering and Gower with Random Forest are the most beneficial approaches when using categorical variables. These tests can determine the optimal number of clustering groups, given the data set, and by doing the proper analysis. Using those the project, we can apply our method to several industrial areas such that clinical, business, and others. For example, people can make different groups based on each patient who has a common disease, required therapy, and other things in the clinical society. Additionally, people can expect to get several clustered groups based on the marginal profit, marginal cost, or other economic indicators for the business area.

Since most bio-medical Linked Data Sets are simply extracted from the Relational database, lots of them are lack of ontology or concept hierarchy structure for user better understanding the data sets. This problem also limited usage of bio-medical Linked Data Sets. To resolve the problem, this paper introduced a method to dynamically generate the concept hierarchy from the Linked Data Sets. Based on the hierarchical clustering algorithm, we applied Vector Space Model(VSM) and Jaccard's Coefficient(JC) to formalize the hierarchy structure after pre-processing data. We implemented our method using two Linked Data Sets: DrugBank and Diseasome from Linked Life Data and evaluated performance with the gold standard.

In this paper, new parallel algorithms for single and complete link hierarchical agglomerative clustering are presented. The parallel algorithms have been mapped on a SIMD machine model with a linear interconnection network. The model consists of a linear array of N PE's, where N is the number of patterns, interfaced with a global host machine and the interconnection network provides inter-PE and PE-to-host/host-to-PE communication. The proposed algorithms are faster than previously known algorithms for hierarchical clustering. For clustering a data set with N patterns, using N PE's, the computation time for the single link clustering algorithm is shown to be O(NlogN) and that for the complete link clustering algorithm is shown to be O(N/sup 2/). The parallel algorithms have been verified through simulations on the Intel's iPSC/2 concurrent supercomputer.

Compressional and shear travel time logs (DTC and DTS) acquired using sonic logging tools are crucial for subsurface geomechanical characterization. In this study, 13 ‘easy-to-acquire’ conventional logs were processed using 6 shallow learning models, namely ordinary least squares (OLS), partial least squares (PLS), elastic net (EN), LASSO, multivariate adaptive regression splines (MARS), and artificial neural network (ANN), to successfully synthesize DTC and DTS logs. Among the 6 models, ANN outperforms other models with R2 of 0.87 and 0.85 for the syntheses of DTC and DTS logs, respectively. The 6 shallow learning models are trained and tested with 8481 data points acquired from a 4240-feet depth interval of a shale reservoir in Well 1, and the trained models are deployed in Well 2 for purposes of blind testing against 2920 data points from 1460-feet depth interval. Following that, 5 clustering algorithms are applied on the 13 ‘easy-to-acquire’ logs to identify clusters and compare them with the prediction performance of the shallow learning models used for log synthesis. Dimensionality reduction algorithm is used to visualize the characteristics of the clustering algorithm. Hierarchical clustering, DBSCAN, and self-organizing map (SOM) algorithms are sensitive to outliers and did not effectively differentiate the input data into consistent clusters. Gaussian mixture model can well differentiate the various formations, but the clusters do not have a strong correlation with the prediction performance of the log-synthesis models. Clusters identified using K-means method have a strong correlation with the prediction performance of the shallow learning models. By combining the predictive shallow learning models for log synthesis with the K-means clustering algorithm, we propose a reliable workflow that can synthesize the DTC and DTS logs, as well as generate a reliability indicator for the predicted logs to help an user better understand the performance of the shallow learning models during deployment.

Clustering is an effective unsupervised machine learning method that can be used as a stand-alone heuristic or as a part of a data mining process. The goal of clustering analysis is to partition data into groups with high intra-cluster association, and low inter-cluster association. Hierarchical clustering requires minimal parameters, has flexibility with similarity measure, and has strong visualization power, all of which makes it ideal for exploratory analysis. Hierarchical clustering is a particular branch of clustering algorithms where the results are not given as partitions, but rather a nested structure, which can represent the ordering among elements within a dataset. The study of the performance of hierarchical structure on time series data is limited. In this paper, we examine the hierarchical structure of time series datasets. The most popular hierarchical structured clustering heuristics include the Hierarchical Agglomerative Clustering, which is distance based; and Ordering Points To Identify the Clustering Structure, which is density based. Both share many similar characteristics and are suitable for time series data processing. We examine the performance of different hierarchical clustering algorithms with time series both internally as well as externally.

This work presents a study about dissimilarity measures for seismic signals, and their relation to clustering in the particular problem of the identification of earthquake focal mechanisms, i.e. the physical phenomena which have generated an earthquake. Starting from the assumption that waveform similarity implies similarity in the focal parameters, important details about them can be determined by studying waveforms related to the wave field produced by earthquakes and recorded by a seismic network. Focal mechanisms identification is currently investigated by clustering of seismic events, using mainly cross-correlation dissimilarity in conjunction with hierarchical clustering algorithm. By the way, it results that such adoptions have not been sufficiently validated. To shed light on this we have studied the cross correlation dissimilarity on simulated seismic signals in conjunction with hierarchical and partitional clustering algorithms, and compared its performance with a newly one recently introduced for the purpose called cumulative shape. In particular, we have properly created synthetic waveforms related to two types of focal mechanisms, showing that the cumulative shape perform better than cross-correlation in the identification of the expected clustering solution.

Detailed comparison is one important sub-task of external plagiarism detection. Seed heuristic between two documents is often used in this task. Vector space model (VSM) and Jaccard coefficient are commonly used in plagiarism detection. VSM can produce high recall performance; Jaccard coefficient can produce high precision performance. In this paper, we propose a hybrid similarity measure model on the basis of the fitting function of the optimal dividing line between plagiarism and none-plagiarism where we integrates VSM and Jaccard coefficient into a unified one, our method make full use of the advantage of VSM and the Jaccard coefficient, and it can extract more reasonable heuristic seeds in the plagiarism detection. Our method is evaluated at PAN corpus of CLEF (Cross-Language Evaluation Forum) and compared with the methods based on VSM or Jaccard coefficient. Experimental results show our method can produce better performance.

Hierarchical clustering algorithm is a very important method in data mining. The disadvantage of hierarchical clustering lies in the time complexity of the algorithm and the one-way irreversibility of the algorithm. The inaccuracy of the conditions for cluster termination is another major disadvantage of hierarchical clustering. Hierarchical clustering requires the final cluster number. But for most datasets, the number of clusters cannot be known in advance. Therefore, a method is proposed to combine the split-based and agglomeration-based hierarchical clustering algorithms to first quickly and uniformly partition the original dataset, and then make similar partitions adaptively merge based on the partition density and partition distance on the basis of these partitions. In this paper, aiming at these defects of hierarchical clustering, a hierarchical clustering algorithm based on fast and uniform segmentation is proposed.

Knowledge graph is the cornerstone of artificial intelligence. Entity alignment in multi-source online encyclopedias is an important part of data integration to construct the knowledge graph. In order to solve the problem that traditional methods are not effective enough for entity alignment in online encyclopedias tasks, this paper proposes the Chinese Online Encyclopedia Aligner (COEA) based on the combination of entity attributes and context. In this paper, we focus on (1) extracting attribute information and context of entities from the infobox of online encyclopedias and normalizing them, (2) computing the similarity of entity attributes based on Vector Space Model, and (3) further considering the entity similarity based on the topic model over entity context when the similarity of attributes is between the lower bound and the upper bound. Finally, data sets of entity alignment in online encyclopedias are constructed for simulation experiments. The experimental results, which show the method proposed in this paper outperforms traditional entity alignment algorithms, verify that our method can significantly improve the performance of entity alignment in online encyclopedias in the construction of Chinese knowledge graphs.

Current many information retrieval methods are based on purely keywords for representing the user needs. One of the main problems with this method is that it does not formally capture the explicit meaning of a keywords query, and ignores the documents that may be different in content but related with them. To improve the precision of the information retrieval, this paper proposes a hybrid method for information retrieval in digital libraries, and gives a corresponding framework and the algorithm of the approach. The proposed method combining the ontology technique and vector space model can improve effectively the drawback existed in present retrieval system.

In text classification based on a vector space model, the high dimension of the feature may pose some problems. These problems occur not only for computational reasons, but also because of overfitting. Feature selection is an important preprocessing step used for text classification applications to reduce the vector space size, control the computational time, and maintain or improve performance. In this study, we used an embedded approach in feature selection in which the Chi-square (CHI) feature selector is a filter step. In this step, the less discriminative features are discarded. In the wrapper step, a novel algorithm is proposed based on the combination of the fast global search ability of the genetic algorithm (GA) and the positive feedback mechanism of ant colony optimization (ACO). In order to validate our approach, we carried out a series of experiments on Reuters-21578 corpus, and we compare the achieved results with some other well-known techniques. The evaluation results are such that our method obtained a better performance compared with the other methods in the majority of cases.

There exist noisy, unparallel sentences in parallel web pages. Web page structure is subjected to some limitation for sentences alignment task for web page text. The most straightforward way of aligning sentences is using a translation lexicon. However, a major obstacle to this approach is the lack of dictionary for training. This paper presents a method for automatically align Mongolian-Chinese parallel text on the Web via vector space model. Vector space model is an algebraic model for representing any object as vectors of identifiers, such as index terms. In the statistically based vector-space model, a sentence is conceptually represented by a vector of keywords extracted from the text. Extracted keywords are composed by content words, known as terms and the weight of a term in a sentence vector can be determined tf-idf method. CHI is used to compute the association between bilingual words. Once the term weights are determined, the similarity between sentence vectors is computed via cosine measure. The experimental results indicate that the method is accurate and efficient enough to apply without human intervention.