Interpretable Semantic Vectors from a Joint Model of Brain- and Text-Based Meaning.

In this paper, we propose a method of genetic algorithm (GA) for text clustering based on singular value decomposition technique. The main difficulty in the application of GA to text clustering is its long string representation in high dimensional space. Because the most straightforward and popular approach represents texts with vector space model (VSM), that is, each unique term in the vocabulary represents one dimension. Singular value decomposition (SVD) is a successful technique arising from numerical linear algebra that is used in latent semantic indexing (LSI). Employing the SVD-based document representation, LSI can overcome the problems by using statistically derived conceptual indices instead of individual words and provide a dimension reduced space. Genetic algorithm belongs to search techniques which could automatically exploit the optimal solution for objective or fitness function of an optimization problem. GA can be used in conjunction with the reduced latent semantic structure and improve clustering efficiency and accuracy. Our algorithm is performed on Reuter documents collection. The results show that the performance of SVD-based GA is significantly superior to that of conventional GA in vector space model.

Vector Space Models (VSMs) represent documents as points in a vector space derived from term frequencies in the corpus. This level of abstraction provides a flexible way to represent complex semantic concepts through vectors, matrices, and higher-order tensors. In this paper we utilize a number of VSMs on a corpus of judicial decisions in order to classify cases in terms of legal factors, stereotypical fact patterns that tend to strengthen or weaken a side's argument in a legal claim. We apply different VSMs to a corpus of trade secret misappropriation cases and compare their classification results. The experiment shows that simple binary VSMs work better than previously reported techniques but that more complex VSMs including dimensionality reduction techniques do not improve performance.

For information retrieval and text-mining, a robust scalable framework is required to represent the information extracted from documents and enable visualization and query of such information. One very widely used model is the vector space model which is based on the bag-of-words approach. However, it suffers from the fact that it loses important information about the original text, such as information about the order of the terms in the text or about the frontiers between sentences or paragraphs. In this paper, we propose a graph-based text representation, which is capable of capturing (i) Term order (ii) Term frequency (iii) Term co-occurrence (iv) Term context in documents. We also apply the graph model into our text mining task, which is to discover unapparent associations between two and more concepts (e.g. individuals) from a large text corpus. Counterterrorism corpus is used to evaluate the performance of various retrieval models, which demonstrates feasibility and effectiveness of graphic text representation in information retrieval and text mining.

An improved focused crawler based on Semantic Similarity Vector Space Model

Vector space models of word meaning typically represent the meaning of a word as a vector computed by summing over all its corpus occurrences. Words close to this point in space can be assumed to be similar to it in meaning. But how far around this point does the region of similar meaning extend? In this paper we discuss two models that represent word meaning as regions in vector space. Both representations can be computed from traditional point representations in vector space. We find that both models perform at over 95% F-score on a token classification task.

The present study attempts to measure and compare the morphological productivity of five Mandarin Chinese suffixes: the verbal suffix -hua, the plural suffix -men, and the nominal suffixes -r, -zi, and -tou. These suffixes are predicted to differ in their degree of productivity : -hua and -men appear to be productive, being able to systematically form a word with a variety of base words, whereas -zi and -tou (and perhaps also -r) may be limited in productivity. Baayen [1989, 1992] proposes the use of corpus data in measuring productivity in word formation. Based on word-token frequencies in a large corpus of texts, his token-based measure of productivity expresses productivity as the probability that a new word form of an affix will be encountered in a corpus. We first use the token-based measure to examine the productivity of the Mandarin suffixes. The present study, then, proposes a type-based measure of productivity that employs the deleted estimation method [Jelinek & Mercer, 1985] in defining unseen words of a corpus and expresses productivity by the ratio of unseen word types to all word types. The proposed type-based measure yields the productivity ranking “-men, -hua, -r, -zi, -tou,” where -men is the most productive and -tou is the least productive. The effects of corpus-data variability on a productivity measure are also examined. The proposed measure is found to obtain a consistent productivity ranking despite variability in corpus data.

A procedure that processes a corpus of text and produces numeric vectors containing information about its meanings for each word is presented. This procedure is applied to a large corpus of natural language text taken from Usenet, and the resulting vectors are examined to determine what information is contained within them. These vectors provide the coordinates in a high-dimensional space in which word relationships can be analyzed. Analyses of both vector similarity and multidimensional scaling demonstrate that there is significant semantic information carried in the vectors. A comparison of vector similarity with human reaction times in a single-word priming experiment is presented. These vectors provide the basis for a representational model of semantic memory, hyperspace analogue to language (HAL).

Hierarchical and Non-Hierarchical Linear and Non-Linear Clustering Methods to Shakespeare Authorship Questionn

Semantic measures are used in handling different issues in several research areas, such as artificial intelligence, natural language processing, knowledge engineering, bioinformatics, and information retrieval. Hierarchical feature-based semantic measures have been proposed to estimate the semantic similarity between two concepts/words depending on the features extracted from a semantic taxonomy (hierarchy) of a given lexical source. The central issue in these measures is the constant weighting assumption that all elements in the semantic representation of the concept possess the same relevance. In this paper, a new weighting-based semantic similarity measure is proposed to address the issues in hierarchical feature-based measures. Four mechanisms are introduced to weigh the degree of relevance of features in the semantic representation of a concept by using topological parameters (edge, depth, descendants, and density) in a semantic taxonomy. With the semantic taxonomy of WordNet, the proposed semantic measure is evaluated for word semantic similarity in four gold-standard datasets. Experimental results show that the proposed measure outperforms hierarchical feature-based semantic measures in all the datasets. Comparison results also imply that the proposed measure is more effective than information-content measures in measuring semantic similarity.

Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning

A technique for the construction of exaggerated human movements was developed and its effectiveness tested for the case of categorising tennis serves as flat, slice, or topspin. The technique involves treating movements as points in a high-dimensional space and uses average movements as the basis for constructing exaggerated movements. Exaggerated movements of a particular style are defined as those points in the space of movements which lie on a line originating at the style average and in the direction defined by the difference between the style average and the grand average. In order to visualise the movements, computer animation techniques were employed to transform the three-dimensional coordinates of the movement into the motion of a solid-body figure. These solid-body models were used in perceptual experiments to assess the effectiveness of the exaggeration technique. After an initial training session on the exemplars from the original library, subjects viewed the synthetic tennis-serve motions and in two separate sessions either made three-alternative, categorisation judgments after viewing a single serve or rated dissimilarity after viewing a pair of serves. Results from both accuracy in the categorisation task and structure of a multidimensional scaling solution of the matrix of dissimilarities indicated that, as distance from the grand average increased, the service motion became more distinct and more accurately identified.

High-dimensional semantic space accounts of priming

This article presents the Semantic Vector Space Model (SVSM), a text representation and searching technique based on the combination of Vector Space Model (VSM) with heuristic syntax parsing and distributed representation of semantic case structures. In this model, both documents and queries are represented as semantic matrices. A search mechanism is designed to compute the similarity between two semantic matrices to predict relevancy. A prototype system was built to implement this model by modifying the SMART system and using the Xerox Part-Of-Speech (P-O-S) tagger as the pre-processor of the indexing process. The prototype system was used in an experimental study to evaluate this technique in terms of precision, recall, and effectiveness of relevance ranking. The results of the study showed that if documents and queries were too short (typically less than 2 lines in length), the technique was less effective than VSM. But with longer documents and queries, especially when original documents were used as queries, we found that the system based on our technique had significantly better performance than SMART. © 1997 John Wiley & Sons, Inc.

Most of text classification methods are based on Vector Space Model (VSM). In the model, each document is mapped to a point (represented by a vector) in Vector Space. A metric measuring the distance of two vectors is then designed to decide the category of any new vector. If the distance is less than some threshold or satisfies some other criterion, the category is assigned to the new vector.KeywordsAssociation RuleFeature SubsetRule MiningDistance ComputationAssociation Rule MiningThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In view of the lack of semantic information description in commonly-used knowledge representation modes such as framework and object-oriented mode, the domain ontology of electronic protective equipment fault diagnosis is designed by analyzing and summarizing the characteristics of the equipment fault cases. Considering that the traditional VSM (vector space model) ignores the role of the position of document entries and structural elements in describing the semantic correlation between features, K-NN (K-nearest neighbor) similarity matching algorithm based on SVSM (semantic vector space model) is proposed by making full use of the advantages of ontology in semantic knowledge representation. For electronic protective equipment fault diagnosis algorithm based on CBR (case based reasoning), a detailed description is given and its feasibility and advantage in performance is proved through experiment.