- Research Article
24
- 10.1016/j.procs.2014.05.314
Feature Extension for Short Text Categorization Using Frequent Term Sets
- Jan 01, 2014
- Procedia Computer Science
- Yuan Man
Feature Extension for Short Text Categorization Using Frequent Term Sets
A short text feature extension algorithm based on improved frequent word set is proposed. By calculating support and confidence, the same category tendencies of frequent term sets are extracted. Correlations based frequent term sets are defined to further extend the term set. Meanwhile, information gain is introduced to traditional TF-IDF, better expressing the category distribution information and the weight of word for each category is enhanced. All term pairs with external relations are extracted and the frequent term set is expanded. Finally, the word similarity matrix is constructed via the frequent word set, and the symmetric non-negative matrix factorization technique is applied to extend the feature space. Experiments show that the constructed short text model can improve the performance of short text clustering.
Feature Extension for Short Text Categorization Using Frequent Term Sets
Feature Extension for Short Text Categorization Using Frequent Term Sets
A Study on Text Clustering Algorithms Based on Frequent Term Sets
In this paper, a new text-clustering algorithm named Frequent Term Set-based Clustering (FTSC) is introduced. It uses frequent term sets to cluster texts. First, it extracts useful information from documents and inserts into databases. Then, it uses the Apriori algorithm based on association rules mining efficiently to discover the frequent items sets. Finally, it clusters the documents according to the frequent words in subsets of the frequent term sets. This algorithm can reduce the dimension of the text data efficiently for very large databases, thus it can improve the accuracy and speed of the clustering algorithm. The results of clustering texts by the FTSC algorithm cannot reflect the overlap of texts' classes. Based on the FTSC algorithm, an improved algorithm—Frequent Term Set-based Hierarchical Clustering algorithm (FTSHC) is given. This algorithm can determine the overlap of texts' classes by the overlap of the frequent words sets, and provide an understandable description of the discovered clusters by the frequent terms sets. The FTSC, FTSHC and K-Means algorithms are evaluated quantitatively by experiments. The results of the experiments prove that FTSC and FTSHC algorithms are more efficient than K-Means algorithm in the performance of clustering.
Read moreThe research of text clustering algorithms based on frequent term sets
In this paper, we present a text-clustering algorithm of frequent term set-based clustering (FTSC), which uses frequent term sets for texts clustering. This algorithm can reduce the dimensionality of the text data efficiently, thus it can improve accurate rate and running speed of the clustering algorithm. The results of clustering texts by the FTSC algorithm cannot reflect the overlap of texts' classes. Based on the FTSC algorithm, its improved algorithm - frequent term set-based hierarchical clustering algorithm (FTSHC) is given. This algorithm can determine the overlap of texts' classes by the overlap of frequent words sets, and provide an understandable description of the discovered clusters by the frequent terms sets. The experiment results prove that FTSC and FTSHC algorithms are more efficient than K-Means algorithm in the performance of clustering.
Read moreStudy on frequent term set-based hierarchical clustering algorithm
This paper, we present a text-clustering algorithm of Frequent Term Set-based Clustering (FTSC), which uses frequent term sets for texts clustering. This algorithm can reduce the dimensionality of the text data efficiently, thus it can improve accurate rate and running speed of the clustering algorithm. The results of clustering texts by the FTSC algorithm cannot reflect the overlap of texts' classes. Based on the FTSC algorithm, its improved algorithm-Frequent Term Set-based Hierarchical Clustering algorithm (FTSHC) is given. This algorithm can determine the overlap of texts' classes by the overlap of frequent words sets, and provide an understandable description of the discovered clusters by the frequent terms sets. The experiment results prove that FTSC and FTSHC algorithms are more efficient than K-Means algorithm in the performance of clustering.
Read moreA Hybrid Algorithm for Web Document Clustering Based on Frequent Term Sets and k-Means
In order to conquer the major challenges of current web document clustering, i.e. huge volume of documents, high dimensional process and understandability of the cluster, we propose a simple hybrid algorithm (SHDC) based on top-k frequent term sets and k-means. Top-k frequent term sets are used to produce k initial means, which are regarded as initial clusters and further refined by k-means. The final optimal clustering is returned by k-means while the understandable description of clustering is provided by k frequent term sets. Experimental results on two public datasets indicate that SHDC outperforms other two representative clustering algorithms (the farthest first k-means and random initial k-means) both on efficiency and effectiveness.
Read moreAn Integrated Approach and Framework for Document Clustering Using Graph Based Association Rule Mining
Growth in number of documents increases day by day, and for managing this growth the document clustering techniques are used document clustering is a significant tool to allocating web search engines for data mining and knowledge discovery. In this paper, we have introduced a new framework graph-based frequent Term set for document clustering (GBFTDC). In this study, document clustering has been performed for extraction of useful information from document dataset based on frequent term set. We have generated association rules to perform pre-processing and then have applied clustering approach.
Read moreMulti-Instance Learning Based Web Mining
In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. In this paper, a web mining problem, i.e. web index recommendation, is investigated from a multi-instance view. In detail, each web index page is regarded as a bag, while each of its linked pages is regarded as an instance. A user favoring an index page means that he or she is interested in at least one page linked by the index. Based on the browsing history of the user, recommendation could be provided for unseen index pages. An algorithm named Fretcit-kNN, which employs the Minimal Hausdorff distance between frequent term sets and utilizes both the references and citers of an unseen bag in determining its label, is proposed to solve the problem. Experiments show that in average the recommendation accuracy of Fretcit-kNN is 81.0% with 71.7% recall and 70.9% precision, which is significantly better than the best algorithm that does not consider the specific characteristics of multi-instance learning, whose performance is 76.3% accuracy with 63.4% recall and 66.1% precision.
Read moreAn Improved Method of Short Text Feature Extraction Based on Words Co-Occurrence
In Chinese text clustering, short text is very different from traditional long text, principally in the low frequency of words. As a result, traditional text feature extraction and the method for weight calculating is not directly suitable for short text clustering .To solve the problem of clustering drift in short text segments ,this paper proposes an method for feature extraction through improving the method of weight calculating based on words co-occurrence. Experiments show the method can get better performance in Chinese short-text clustering compared with the traditional method TF-IDF.
Read moreA new metric for feature selection on short text datasets
In recent years, short texts are everywhere, especially in social media networks. Short text classification is an essential task for various applications related to the operations on short text documents. In many cases, using the entire feature set causes the high dimensionality problem in short text data. This problem reason of time‐consuming and negatively impacts the performance of classifiers. This study presents an effective feature selection algorithm called XY method, which represents the features on XY line and calculates the distance of a feature to the XY line. Also, a value named λ is calculated. According to this value, the terms are divided into different regions such as negative, positive, and third to determine their discrimination capability. The novel XY method aims to select as few terms as possible in the negative region. The proposed method is evaluated using four different short text datasets with Macro‐F1 success measure. In comparisons with other existing feature selection algorithms such as chi‐square, information gain, deviation from Poisson distribution, recently proposed max‐min ratio, and distinguishing feature selector demonstrate that the XY method achieves either better or competitive performance in significantly reduced various feature sizes.
Read moreEnhancing Topic Modeling for Short Texts with Auxiliary Word Embeddings
Many applications require semantic understanding of short texts, and inferring discriminative and coherent latent topics is a critical and fundamental task in these applications. Conventional topic models largely rely on word co-occurrences to derive topics from a collection of documents. However, due to the length of each document, short texts are much more sparse in terms of word co-occurrences. Recent studies show that the Dirichlet Multinomial Mixture (DMM) model is effective for topic inference over short texts by assuming that each piece of short text is generated by a single topic. However, DMM has two main limitations. First, even though it seems reasonable to assume that each short text has only one topic because of its shortness, the definition of “shortness” is subjective and the length of the short texts is dataset dependent. That is, the single-topic assumption may be too strong for some datasets. To address this limitation, we propose to model the topic number as a Poisson distribution, allowing each short text to be associated with a small number of topics (e.g., one to three topics). This model is named PDMM. Second, DMM (and also PDMM) does not have access to background knowledge (e.g., semantic relations between words) when modeling short texts. When a human being interprets a piece of short text, the understanding is not solely based on its content words, but also their semantic relations. Recent advances in word embeddings offer effective learning of word semantic relations from a large corpus. Such auxiliary word embeddings enable us to address the second limitation. To this end, we propose to promote the semantically related words under the same topic during the sampling process, by using the generalized Pólya urn (GPU) model. Through the GPU model, background knowledge about word semantic relations learned from millions of external documents can be easily exploited to improve topic modeling for short texts. By directly extending the PDMM model with the GPU model, we propose two more effective topic models for short texts, named GPU-DMM and GPU-PDMM. Through extensive experiments on two real-world short text collections in two languages, we demonstrate that PDMM achieves better topic representations than state-of-the-art models, measured by topic coherence. The learned topic representation leads to better accuracy in a text classification task, as an indirect evaluation. Both GPU-DMM and GPU-PDMM further improve topic coherence and text classification accuracy. GPU-PDMM outperforms GPU-DMM at the price of higher computational costs.
Read moreShort Text Similarity Measurement Using Context from Bag of Word Pairs and Word Co-occurrence
With the rapid development of social networks, short texts have become a prevalent form of social communications on the Internet. Measuring the similarity between short texts is a fundamental task to many applications, such as social network text querying, short text clustering and geographical event detection for smart city. However, short texts in social media always show limited contextual information and they are sparse, noisy and ambiguous. Hence, effectively measuring the distance between short texts is a challenging task.
Read moreTopic extraction from extremely short texts with variational manifold regularization
With the emerging of massive short texts, e.g., social media posts and question titles from Q&A systems, discovering valuable information from them is increasingly significant for many real-world applications of content analysis. The family of topic modeling can effectively explore the hidden structures of documents through the assumptions of latent topics. However, due to the sparseness of short texts, the existing topic models, e.g., latent Dirichlet allocation, lose effectiveness on them. To this end, an effective solution, namely Dirichlet multinomial mixture (DMM), supposing that each short text is only associated with a single topic, indirectly enriches document-level word co-occurrences. However, DMM is sensitive to noisy words, where it often learns inaccurate topic representations at the document level. To address this problem, we extend DMM to a novel Laplacian Dirichlet Multinomial Mixture (LapDMM) topic model for short texts. The basic idea of LapDMM is to preserve local neighborhood structures of short texts, enabling to spread topical signals among neighboring documents, so as to modify the inaccurate topic representations. This is achieved by incorporating the variational manifold regularization into the variational objective of DMM, constraining the close short texts with similar variational topic representations. To find nearest neighbors of short texts, before model inference, we construct an offline document graph, where the distances of short texts can be computed by the word mover’s distance. We further develop an online version of LapDMM, namely Online LapDMM, to achieve inference speedup on massive short texts. Carrying this implications, we exploit the spirit of stochastic optimization with mini-batches and an up-to-date document graph that can efficiently find approximate nearest neighbors instead. To evaluate our models, we compare against the state-of-the-art short text topic models on several traditional tasks, i.e., topic quality, document clustering and classification. The empirical results demonstrate that our models achieve very significant performance gains over the baseline models.
Read moreShort text similarity calculation method based on word vector attention mechanism and CNN-BiPLSTM
Similarity calculation of short text is an important part of text classification. In view of the problem that statistical learning methods can only extract shallow features of short texts, but cannot express their deep semantics through the extracted features, a new method based on two-way mirror hole connection is proposed. A short text semantic similarity calculation model combining long short-term memory network (BiP-LSTM) and convolutional neural network (CNN). The word vector is pre-trained with the word vector attention mechanism, and the distance factor is introduced to solve the influence of the distance between words on the word sense association. When combining sentence-level features, an attention mechanism is added to make it contain more representative features. The experimental results show that, based on the Rotten_imdb and Quora Question Pairs datasets, the proposed short text similarity calculation method has an accuracy of 84.37% and an F1 value of 83.67%, which is better than other benchmark methods.
Read moreShort Text Clustering Based on Word Semantic Graph with Word Embedding Model
Nowadays, a number of short messages or short text contents created on the Internet are rapidly increasing. Tasks to manipulate, analyze, and extract knowledge from them lead mining techniques such as text clustering to become more important. However, applying traditional text clustering algorithms which consider only common words or phrases to group short texts is inefficient due to the problem of sparsity. In this paper, we propose a new clustering technique, called word semantic graph clustering, based on the use of text concepts. We apply the word embedding model from Word2Vec to capture the semantic meaning of words and later construct semantic subgraphs in which those words represented as vertices are connected by some high semantic similarities. Finally, short text documents will be assigned to the same cluster if they contain at least one word belonging to the same semantic subgraph. Experimental results conducted on two real datasets show that the proposed approach outperforms the state-of-the-art text clustering algorithms. In addition, it can also produce more appropriate label for each cluster than the comparative algorithms do.
Read moreShort Text Mapping Based on Fast Clustering Using Minimum Spanning Trees
Due to short length and limited content, short text representation has the problem of high-dimension and high-sparsity. For the purpose of achieving the goal of reducing the dimension and eliminate the sparseness while preserve the semantics of the information in the text to be represented, a method of short text mapping based on fast clustering using minimum spanning trees is proposed. First, we remove the irrelevant terms, then a clustering method based on minimum spanning tree is adopted to identify the relevant term set and remove the redundant terms to get the short text mapping space. Finally, a matrix mapping method is designed to represent the original short text on a highly correlated and non-redundant short text mapping space. The proposed method not only has low time complexity but also produces higher quality short text mapping space. The experiments prove that our method is feasible and effective.
Read more