A Document Classifier Based on Word Semantic Association

Vector Space Models (VSMs) represent documents as points in a vector space derived from term frequencies in the corpus. This level of abstraction provides a flexible way to represent complex semantic concepts through vectors, matrices, and higher-order tensors. In this paper we utilize a number of VSMs on a corpus of judicial decisions in order to classify cases in terms of legal factors, stereotypical fact patterns that tend to strengthen or weaken a side's argument in a legal claim. We apply different VSMs to a corpus of trade secret misappropriation cases and compare their classification results. The experiment shows that simple binary VSMs work better than previously reported techniques but that more complex VSMs including dimensionality reduction techniques do not improve performance.

This study presents an overview of the research aimed at optimizing library information retrieval through the utilization of the Vector Space Model (VSM) method in a computer science context. Libraries, as publicly financed collections, provide extensive knowledge resources, eliminating the need for individual book purchases. However, the challenge lies in efficiently navigating the expanding library collections. To tackle this issue, the study employs information retrieval techniques, particularly the VSM method, which assesses term similarity by assigning weights to terms, enabling document and query representation as vectors. The relevance between documents and queries is measured through vector similarity. This approach, integrated with indexing, streamlines collection retrieval in libraries. Employing the Waterfall model for system development, the research outlines phases like analysis, design, coding, testing, and implementation. While effective, the model's rigidity in accommodating evolving requirements poses limitations. The VSM method's numerical representation of text documents facilitates precise similarity calculations, supported by TF-IDF values indicating term importance in documents relative to the corpus. The study further extends to system design using UML diagrams and a visitor interface, integrating VSM for efficient search functionality. Black-box testing confirms the robustness of the system components and interfaces. Overall, this research presents a systematic approach to enhance information retrieval in libraries, emphasizing the VSM's pivotal role in optimizing document searches within expansive collections.

Chapter 1 - Why Geometric Algebra?

Huge digital data is typical nowadays. Consequently, text clustering has become a crucial text mining technique nowadays. Organizing search information into groups (or clusters) not only makes the search results more meaningful, but makes the system more efficient since the user views only the relevant information and ignores the rest. An example is clustering web search engine result into meaningful results. Many text clustering approaches and their corresponding algorithms exist, but none has been found to be sufficient. There is also insufficient understanding of the algorithms as well as lack of agreed formal classification of the algorithms. There is thus, need for an in-depth study of the various algorithms. In this paper, we describe the Vector Space Model (VSM) method of text representation. We also give an overview of the text clustering approaches that apply the VSM .These include distance based approach, feature extraction approach, density-based approach, grid-based approach, and neural networks approach. We describe the characteristics of each accompanied by a representative algorithm. The paper thus informs researchers of text mining concerning the current state of affairs of text clustering algorithm. Keywords: clusters, text clustering, text mining, vector space model.

Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning

QuantNet is an integrated web-based environment consisting of different types of statistics-related documents and program codes. Its goal is creating reproducibility and offering a platform for sharing validated knowledge native to the social web. To increase the information retrieval (IR) efficiency there is a need for incorporating semantic information. Three text mining models will be examined: vector space model (VSM), generalized VSM (GVSM) and latent semantic analysis (LSA). The LSA has been successfully used for IR purposes as a technique for capturing semantic relations between terms and inserting them into the similarity measure between documents. Our results show that different model configurations allow adapted similarity-based document clustering and knowledge discovery. In particular, different LSA configurations together with hierarchical clustering reveal good results under M3 evaluation. QuantNet and the corresponding Data-Driven Documents (D3) based visualization can be found and applied under quantlet. The driving technology behind it is Q3-D3-LSA, which is the combination of “GitHub API based QuantNet Mining infrastructure in R”, LSA and D3 implementation.

Currently, the data that dealt by traditional text clustering methods is small. Text representation model that text clustering used is traditional vector space model (VSM). The traditional text clustering has defect of low efficiency when processing big data. The quality is bad when using traditional VSM model for text representation. To solve these two problems, this paper puts forward a VSM model for text representation, which is improved by Term Frequency(TF), and using Mahout to text clustering. Experimental results show that use Mahout for text clustering has a higher efficiency than traditional text clustering; this novel VSM model can improve the text clustering quality.

Vector space models (VSMs) represent word meanings as points in a high dimensional space. VSMs are typically created using a large text corpora, and so represent word semantics as observed in text. We present a new algorithm (JNNSE) that can incorporate a measure of semantics not previously used to create VSMs: brain activation data recorded while people read words. The resulting model takes advantage of the complementary strengths and weaknesses of corpus and brain activation data to give a more complete representation of semantics. Evaluations show that the model 1) matches a behavioral measure of semantics more closely, 2) can be used to predict corpus data for unseen words and 3) has predictive power that generalizes across brain imaging technologies and across subjects. We believe that the model is thus a more faithful representation of mental vocabularies.

Document categorization is the process of classifying documents from many mixed documents automatically, and the main problem is how to express document content in vector space completely. This paper proposes a new model named Latent Semantic Analysis (LSA) + word2vec to categorize documents. This is the first attempt of combining word2vec with LSA at document categorization and it can map document to vector space under the premise of keeping document contents fully. At first, we create a term by document matrix and the element of which is decided by Term Frequency-Inverse Document Frequency (TF-IDF) weighting and word vector trained by word2vec. This matrix is a 3-dimensional matrix and it can describe the meaning of every word and the content of every document exactly. Secondly, Singular Value Decomposition (SVD) is executed on the matrix and lower computational complexity is gained from this. The model is named LSA + word2vec. Then, document vector gained from the new model are put into Convolutional Neural Network (CNN) to train. CNN is an efficient deep learning algorithm, which improves the accuracy of classification greatly. We evaluate the performance based on the 20newsgroups corpus. The results show that our new model achieves better effects on document categorization tasks, and the accuracy made about 15% improvement than traditional methods, such as LSA and Vector Space Model (VSM).

Minkowski distances are frequently used to indicate the similarity of two vectors in a n-dimensional space. This paper is about the probability distributions of Minkowski distances (e.g., City-block distances and Euclidean distances) between vectors in spaces spanned by n orthogonal, discrete valued axes. Formulas to compute the distributions of Minkowski distances are developed, critical values for tests of significance are tabled, and a normal approximation is examined. With the given information about the distributions of Minkowski distances a proper interpretation of empirical distance values should be facilitated.

In text classification based on a vector space model, the high dimension of the feature may pose some problems. These problems occur not only for computational reasons, but also because of overfitting. Feature selection is an important preprocessing step used for text classification applications to reduce the vector space size, control the computational time, and maintain or improve performance. In this study, we used an embedded approach in feature selection in which the Chi-square (CHI) feature selector is a filter step. In this step, the less discriminative features are discarded. In the wrapper step, a novel algorithm is proposed based on the combination of the fast global search ability of the genetic algorithm (GA) and the positive feedback mechanism of ant colony optimization (ACO). In order to validate our approach, we carried out a series of experiments on Reuters-21578 corpus, and we compare the achieved results with some other well-known techniques. The evaluation results are such that our method obtained a better performance compared with the other methods in the majority of cases.

Thus far, we have dealt with indexes that support Boolean queries: A document either matches or does not match a query. In the case of large document collections, the resulting number of matching documents can far exceed the number a human user could possibly sift through. Accordingly, it is essential for a search engine to rank-order the documents matching a query. To do this, the search engine computes, for each matching document, a score with respect to the query at hand. In this chapter, we initiate the study of assigning a score to a (query, document) pair. This chapter consists of three main ideas. We introduce parametric and zone indexes in Section 6.1, which serve two purposes. First, they allow us to index and retrieve documents by metadata, such as the language in which a document is written. Second, they give us a simple means for scoring (and thereby ranking) documents in response to a query. Next, in Section 6.2 we develop the idea of weighting the importance of a term in a document, based on the statistics of occurrence of the term. In Section 6.3, we show that by viewing each document as a vector of such weights, we can compute a score between a query and each document. This view is known as vector space scoring. Section 6.4 develops several variants of term-weighting for the vector space model. Chapter 7 develops computational aspects of vector space scoring and related topics.

Word2vec word vector algorithm and Doc2vec text vectoring algorithm calculate the similarity between words and between texts from the perspective of vector space without considering the semantic similarity between texts. Thus, in this paper, an automatic evaluation algorithm is proposed to combine semantic similarity based on synonym lexical forest and text similarity based on vector space. Meanwhile, Sentiment analysis was used to obtain the emotional score of the reference answers and students' answers in the subjective questions. Finally, the final score of students' answers was obtained by combining text similarity and emotion scores. In this paper, logistics professional education is the experimental object of the experiment. The experimental results show that the algorithm has achieved good scoring results.

Hyponymy is used to build a taxonomic hierarchy. But the terms in hyponymy may have multiple senses. It will cause the problem of polysemy and affect the building of taxonomic hierarchy. In order to solve the problem, we present a method of sense recognition of hyponymy based on vector space model. Firstly we acquire the contexts of hyponymy from Chinese free corpus. Secondly we use Cilin to construct a relation-word vector space. Then we use latent semantic analysis to reduce the dimension of the vector space. In the final phase, we recognize the senses of hyponymy using average-group clustering. Experimental results show that the method can provide adequate discrimination of the different senses.

Since the inception of the concept of social networking, communication patterns have shifted drastically with the unmitigated trend in socializing over the Internet, especially when people began connecting via mobile devices. Nowadays people tend to use these modern communication systems to share their emotions with each other. Human emotions play a vital role in human relationships and people share their emotions through facial expressions, gestures, speech and text messages. However, text messaging is the most common and widely accepted method to exchange information among peers through the Internet and mobile networks. In comparison to other methods, identifying emotions from text messages is rather difficult for the recipient. Therefore, the need of automating the emotion recognition from textual content has increased. Utilization of text classification techniques can be considered as the most common approach of identifying emotions from textual content. Prior to applying a text classifier, the textual data should be transformed into a data structure that the classifier understands by conforming to a document representation model and term weighting method. For this research Vector Space Model (VSM) is used as the document representation model. This paper proposes an extension to the Term Frequency - Inverse Document Frequency (TF-IDF) weighting method to increase classification accuracy and explains experiments conducted to discover the best term weighting method in vector space to be used in feature (text term) extraction from Aman's emotion text corpus. The text classification is done using Oracle's ODM SVM tool and LibSVM tool.