An Overview of VSM-Based Text Clustering Approaches

Currently, the data that dealt by traditional text clustering methods is small. Text representation model that text clustering used is traditional vector space model (VSM). The traditional text clustering has defect of low efficiency when processing big data. The quality is bad when using traditional VSM model for text representation. To solve these two problems, this paper puts forward a VSM model for text representation, which is improved by Term Frequency(TF), and using Mahout to text clustering. Experimental results show that use Mahout for text clustering has a higher efficiency than traditional text clustering; this novel VSM model can improve the text clustering quality.

QuantNet is an integrated web-based environment consisting of different types of statistics-related documents and program codes. Its goal is creating reproducibility and offering a platform for sharing validated knowledge native to the social web. To increase the information retrieval (IR) efficiency there is a need for incorporating semantic information. Three text mining models will be examined: vector space model (VSM), generalized VSM (GVSM) and latent semantic analysis (LSA). The LSA has been successfully used for IR purposes as a technique for capturing semantic relations between terms and inserting them into the similarity measure between documents. Our results show that different model configurations allow adapted similarity-based document clustering and knowledge discovery. In particular, different LSA configurations together with hierarchical clustering reveal good results under M3 evaluation. QuantNet and the corresponding Data-Driven Documents (D3) based visualization can be found and applied under quantlet. The driving technology behind it is Q3-D3-LSA, which is the combination of “GitHub API based QuantNet Mining infrastructure in R”, LSA and D3 implementation.

Statistical Analysis for Marketing on Facebook : Text Mining and Clustering Approaches(Competition 1)

Blockchain technology forecasting by patent analytics and text mining

This paper studies the classification of the coal supply-demand information that comes from the Internet by using text mining technology. Firstly, it introduces the concept of Web text mining and the basic theories of the classification process. Then, it makes a further discussion about related technologies in all aspects of Web text mining process, and focuses on the analysis of the text classification algorithm. At the same time, this paper explores the current development of coal industry and the application of Web text mining on the coal supply-demand market. At last, it creates the information classification model based on the text classification algorithm of Web text mining, which visualizes the process of classifying the online coal supply-demand information.

In this paper, we propose a method of genetic algorithm (GA) for text clustering based on singular value decomposition technique. The main difficulty in the application of GA to text clustering is its long string representation in high dimensional space. Because the most straightforward and popular approach represents texts with vector space model (VSM), that is, each unique term in the vocabulary represents one dimension. Singular value decomposition (SVD) is a successful technique arising from numerical linear algebra that is used in latent semantic indexing (LSI). Employing the SVD-based document representation, LSI can overcome the problems by using statistically derived conceptual indices instead of individual words and provide a dimension reduced space. Genetic algorithm belongs to search techniques which could automatically exploit the optimal solution for objective or fitness function of an optimization problem. GA can be used in conjunction with the reduced latent semantic structure and improve clustering efficiency and accuracy. Our algorithm is performed on Reuter documents collection. The results show that the performance of SVD-based GA is significantly superior to that of conventional GA in vector space model.

The fast growth of Internet web documents has posed new challenges on how to efficiently and accurately manage and retrieve the textual collections, text clustering plays a significant role. Traditional document clustering is an unsupervised categorization of a given document collection based on vector space model, which is a high sparse vector. In this paper, we propose a means to fight the existing shortcomings with a word vector in distributed representation which is obtained from a neural probabilistic language model. To improve the representation of document vector and enhance the accuracy of text clustering, we first computing semantic similarities between words using word embedded vector, and then expanding the keywords of each document. The experiment results show the method can improve the accuracy of clustering.

This paper improves the traditional text mining technology which cannot understand the text semantics. The author discusses the text mining methods based on ontology and puts forward text mining model based on domain ontology. Ontology structure is built firstly and the “concept-concept” similarity matrix is introduced, then a conception vector space model based on domain ontology is used to take the place of traditional vector space model to represent the documents in order to realize text mining. Finally, the author does a case and draws some conclusions.KeywordsOntologytext miningdomain ontologyvector space model

Mining is a research area of retrieving high quality hidden information such as patterns, trends, or distributions through analyzing unformatted text.Basically, since text mining assumes an unstructured text, it needs to be represented as a simple text model for analyzing it.So far, most frequently used model is VSM(Vector Space Model), in which a text is represented as a bag of words.However, recently much researches tried to apply a graph-based text model for representing semantic relationships between words.In this paper, we survey research trends of graph-based text representation models for text mining.Additionally, we also discuss about future models of graph-based text mining.

It is seen that distance education environments are more frequently used due to the progress of internet today. The data acquired from these environments are commonly irregular journal records and text-based communication messages. Digital mining applications are used in order to convert these irregular data into regular, and cleaning out the information stacks. The purpose of this research is to reveal the usage of the types of digital mining application, which are being applied within the distance education environments. For this purpose, a situation-based study was carried out which was related to the subjects of data mining, web mining and text mining, being used in the distance education environments. As a result of the research, a relationship was determined between the types of digital mining. Key words: Text mining, data mining, web mining.

At present, computer technology is widely used in medicine, such as text mining and text clustering. Text clustering is a common application in the field of natural semantic processing, and can also be applied to the processing of Traditional Chinese Medicine medical records. The clustered medical records are convenient for doctors to conduct more targeted disease analysis. This paper adopts the method of building Term Frequency–Inverse Document Frequency matrix, combined with technical means such as word frequency statistics, to achieve clustering of limited medical case texts. The main work of this paper: 1. Extract key information from Traditional Chinese Medicine medical records, and compare the extraction results with artificial symptom analysis. 2. Use the k-means text clustering algorithm to perform cluster analysis on a small number of medical records, and generate subject words for each category. Results: The Jieba library can effectively extract the keywords of medical records, and the medical records clustering can achieve the expected results.

Phrase2Vec: Phrase embedding based on parsing

With the continuous improvement of product technical performance, more and more attention has been paid to the emotional needs of users. As a result, understanding the relationship between emotional impression and design elements is a vital procedure in product design, which is also an important part of Kansei Engineering. However, traditional questionnaire method has some shortcomings in this aspect, because it may cause misunderstanding of word meaning. Therefore, this paper uses a method called text mining technology, which is based on big data and can truly and effectively reflect the user’s point of view. 3367 online reviews about reclining chairs were crawled from JingDong, one of the largest online shopping service platforms in China. Through word frequency statistics and text clustering, we can find the users’ concerns, and study the relationship between design elements and emotional impression in a complete context. The results show that ‘comfort’ is the most frequently used emotional variable when using the recliner, which is related to various emotional variables. In addition, the reclining chair comments extracted from JingDong are divided into 15 clusters by python. The findings indicate that text mining technology and text clustering can effectively collect and analyze the users’ emotional experience, and provide some guidance for the design of reclining chair.KeywordsKansei engineeringText miningReclinerUser reviewsPython

This study presents an overview of the research aimed at optimizing library information retrieval through the utilization of the Vector Space Model (VSM) method in a computer science context. Libraries, as publicly financed collections, provide extensive knowledge resources, eliminating the need for individual book purchases. However, the challenge lies in efficiently navigating the expanding library collections. To tackle this issue, the study employs information retrieval techniques, particularly the VSM method, which assesses term similarity by assigning weights to terms, enabling document and query representation as vectors. The relevance between documents and queries is measured through vector similarity. This approach, integrated with indexing, streamlines collection retrieval in libraries. Employing the Waterfall model for system development, the research outlines phases like analysis, design, coding, testing, and implementation. While effective, the model's rigidity in accommodating evolving requirements poses limitations. The VSM method's numerical representation of text documents facilitates precise similarity calculations, supported by TF-IDF values indicating term importance in documents relative to the corpus. The study further extends to system design using UML diagrams and a visitor interface, integrating VSM for efficient search functionality. Black-box testing confirms the robustness of the system components and interfaces. Overall, this research presents a systematic approach to enhance information retrieval in libraries, emphasizing the VSM's pivotal role in optimizing document searches within expansive collections.

A Study on the Characteristics and Significance of the Boycott Japan based on Text Mining and Semantic Network Analysis