Representing words as regions in vector space

It is widely accepted that traditional word embedding models, which rely on distributional semantics hypothesis, are relatively limited for contrast meaning problem. Distributional semantics hypothesis indicates that words lying in similar contexts have similar representations in vector space. Nevertheless, synonyms and antonyms often locate in similar contexts, which means they appear close to each other in vector space. Hence, it is of great difficulty to distinguish antonyms from synonyms. To address this challenge, we propose an optimization model, named Lexicon-based Word Embedding Tuning (LWET) model. The goal of LWET is to incorporate reliable semantic lexicons to tune the distributions of pre-trained word embeddings in the vector space so as to improve their ability of distinguishing antonyms from synonyms. To speed up the training process of LWET, we propose two approximation algorithms, including positive sampling and quasi-hierarchical softmax. Compared with quasi-hierarchical softmax, positive sampling is faster, however, at the cost of worse performance. In experiments, LWET and other state-of-the-art models are tested on antonyms recognition, distinguishing antonyms from synonyms and word similarity. The results of the first two experiments show that LWET significantly improves the ability of word embeddings to detect antonyms, thus achieving the state-of-the-art performance. On word similarity, LWET gets slightly better performance than the state-of-the-art models. It means that LWET can remain and strengthen the semantic structure rather than destroy it when tuning word distributions in vector space. In general, compared with related work, LWET can not only achieve similar or even better performance, but also speed up the training process.

Vector space models (VSMs) represent word meanings as points in a high dimensional space. VSMs are typically created using a large text corpora, and so represent word semantics as observed in text. We present a new algorithm (JNNSE) that can incorporate a measure of semantics not previously used to create VSMs: brain activation data recorded while people read words. The resulting model takes advantage of the complementary strengths and weaknesses of corpus and brain activation data to give a more complete representation of semantics. Evaluations show that the model 1) matches a behavioral measure of semantics more closely, 2) can be used to predict corpus data for unseen words and 3) has predictive power that generalizes across brain imaging technologies and across subjects. We believe that the model is thus a more faithful representation of mental vocabularies.

This paper proposes an adaptive document representation (concept vector space model) using Markov Chain model. The vector space representation is one of the most common models for representing documents in classification process. The document classification based on ontology classification approach is represented as a vector, whose components are ontology concepts and their relevance. The relevance is represented the by frequency of concepts’ occurrence. These concepts make various contributions in classification process. The contributions depend on the position of concepts where they are depicted in the ontology hierarchy. The hierarchy such as classes, subclasses and instances may have different values to represent the concepts’ importance. The weights to define concepts’ importance are generally selected by empirical analysis and are usually kept fixed. Thus, making it less effective and time consuming. We therefore propose a new model to automatically estimate weights of concepts within the ontology. This model initially maps the ontology to a Markov chain model and then calculates the transition probability matrix for this Markov chain. Further, the transition probability matrix is used to compute the probability of steady states based on left eigenvectors. Finally, the importance is calculated for each ontology concept. And, an enhanced concept vector space representation is created with concepts’ importance and concepts’ relevance. The concept vector space representation can be adapted for new ontology concepts.

The performance of text classification can be affected by the choice of appropriate term weighting scheme as well as other parameters. The terminology supervised term weighting scheme has become popular in recent years, as it may provide discriminative representation in vector space for text documents belonging to different classes. A term weighting scheme generally consists of three factors, namely term frequency factor, collection frequency factor, and length normalization factor. The researchers mostly have been focused on developing new collection frequency factors in term weighting studies. However, the term frequency factor has an important role, especially in supervised term weighting. In this study, we extensively analyzed the effects of using different term frequency factors on seven supervised term weighting schemes. While six of these supervised term weighting schemes were applied in the previous studies in the literature, we derived one of them from an existing feature selection method and it was not used as a weighting method before. This analysis is performed using SVM and Roccio classifiers on two widely known benchmark datasets with different characteristics. Experimental results showed that modification of term frequency factor in supervised term weighting schemes increased the performance of almost all weighting schemes. Also, term weighting schemes using square root function-based term frequency factor (SQRT_TF) are more successful than the ones using term frequency (TF) and logarithmic function-based term frequency (LOG_TF) factors. TF term frequency factor seems as the least effective one among three different term frequency factors according to the experimental results and statistical analysis.

Image similarity is widely applicable in image understanding and object tracking. It is easy for human to fulfill while difficult for machines. In this paper, we present a simple but efficient end-to-end mechanism to transfer an image into its corresponding representation in vector space based on Convolutional Neural Networks supervised by word2vec, which can then be applied to applications such as image classification and object detection, and a further work of image caption/description. We describe how we train the model to achieve a deep semantic understanding of the image along with its caption. We train our method on Flickr8k and Flickr30k datasets respectively, and evaluate on Corel1k benchmark dataset. Through the visualization of how our model extracts the features of images and produces similar vectors for similar images, we demonstrate the effectiveness of our proposed model.

Vector Space Models (VSMs) represent documents as points in a vector space derived from term frequencies in the corpus. This level of abstraction provides a flexible way to represent complex semantic concepts through vectors, matrices, and higher-order tensors. In this paper we utilize a number of VSMs on a corpus of judicial decisions in order to classify cases in terms of legal factors, stereotypical fact patterns that tend to strengthen or weaken a side's argument in a legal claim. We apply different VSMs to a corpus of trade secret misappropriation cases and compare their classification results. The experiment shows that simple binary VSMs work better than previously reported techniques but that more complex VSMs including dimensionality reduction techniques do not improve performance.

In this article, we propose a review of studies evaluating the distribution of distances between elements of a random set independently and uniformly distributed over a region of space in a normed R -vector space (for example, point events generated by a homogeneous Poisson process in a compact set). The distribution of distances between individuals is present in many situations when interaction depends on distance and concerns many disciplines, such as statistical physics, biology, ecology, geography, networking, etc. After reviewing the solutions proposed in the literature, we present a modern, general and unified resolution method using convolution of random vectors. We apply this method to typical compact sets: segments, rectangles, disks, spheres and hyperspheres. We show, for example, that in a hypersphere the distribution of distances has a typical shape and is polynomial for odd dimensions. We also present various applications of these results and we show, for example, that variance of distances in a hypersphere tends to zero when space dimension increases.

Analysis of word embedding properties to inform their use in downstream NLP tasks has largely been studied by assessing nearest neighbors. However, geometric properties of the continuous feature space contribute directly to the use of embedding features in downstream models, and are largely unexplored. We consider four properties of word embedding geometry, namely: position relative to the origin, distribution of features in the vector space, global pairwise distances, and local pairwise distances. We define a sequence of transformations to generate new embeddings that expose subsets of these properties to downstream models and evaluate change in task performance to understand the contribution of each property to NLP models. We transform publicly available pretrained embeddings from three popular toolkits (word2vec, GloVe, and FastText) and evaluate on a variety of intrinsic tasks, which model linguistic information in the vector space, and extrinsic tasks, which use vectors as input to machine learning models. We find that intrinsic evaluations are highly sensitive to absolute position, while extrinsic tasks rely primarily on local similarity. Our findings suggest that future embedding models and post-processing techniques should focus primarily on similarity to nearby points in vector space.

Most of text classification methods are based on Vector Space Model (VSM). In the model, each document is mapped to a point (represented by a vector) in Vector Space. A metric measuring the distance of two vectors is then designed to decide the category of any new vector. If the distance is less than some threshold or satisfies some other criterion, the category is assigned to the new vector.KeywordsAssociation RuleFeature SubsetRule MiningDistance ComputationAssociation Rule MiningThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Fake news detection on job advertisements has grabbed the attention of many researchers over past decade. Various classifiers such as Support Vector Machine (SVM), XGBoost Classifier and Random Forest (RF) methods are greatly utilized for fake and real news detection pertaining to job advertisement posts in social media. Bi-Directional Long Short-Term Memory (Bi-LSTM) classifier is greatly utilized for learning word representations in lower-dimensional vector space and learning significant words word embedding or terms revealed through Word embedding algorithm. The fake news detection is greatly achieved along with real news on job post from online social media is achieved by Bi-LSTM classifier and thereby evaluating corresponding performance. The performance metrics such as Precision, Recall, F1-score, and Accuracy are assessed for effectiveness by fraudulency based on job posts. The outcome infers the effectiveness and prominence of features for detecting false news. .

An affine algebraic group

Cross-lingual word embeddings display words from different languages in the same vector space. They provide reasoning about semantics, compare the meaning of words across languages and word meaning in multilingual contexts, necessary to bilingual lexicon induction, machine translation, and cross-lingual information retrieval. This paper proposes an efficient approach to learn bilingual transform mapping between monolingual word embeddings in language pairs. We choose ten different languages from three different language families and downloaded their last update Wikipedia dumps1 1. https://dumps.wikimedia.org. Then, with some pre-processing steps and using word2vec, we produce word embeddings for them. We select seven language pairs from chosen languages. Since the selected languages are relative, they have thousands of identical words with similar meanings. With these identical dictation words and word embedding models of each language, we create training, validation and, test sets for the language pairs. We then use a generative adversarial network (GAN) to learn the transform mapping between word embeddings of source and target languages. The average accuracy of our proposed method in all language pairs is 71.34%. The highest accuracy is achieved for the Turkish-Azerbaijani language pair with the accuracy 78.32%., which is noticeably higher than prior methods.

Anomaly Detection Using Integration Model of Vector Space and Network Representation

This paper describes the effects of a complex scalar scaling field on quantum mechanics. The field origin is an extension of the gauge freedom for basis choice in gauge theories to the underlying scalar field. The extension is based on the idea that the value of a number at one space time point does not determine the value at another point. This, combined with the description of mathematical systems as structures of different types, results in the presence of separate number fields and vector spaces, both as structures, at different space time locations. Complex number structures and vector spaces at each location, are scaled by a complex space time dependent scaling factor. The effect of this scaling factor on several physical and geometric quantities has been described in other work. Here the emphasis is on quantum mechanics of one and two particles, their states and properties. Multiparticle states are also briefly described. The effect shows as a complex, nonunitary, scalar field connection on a fiber bundle description of nonrelativistic quantum mechanics. The lack of physical evidence for the presence of this field so far means that the coupling constant of this field to fermions is very small. It also means that the gradient of the field must be very small in a local region of cosmological space and time. Outside this region there are no restrictions on the field gradient.

This paper presents our experimental work on mining of opinions from a large number of blog posts and its relevance for socio-political research. The experimental work involves collecting blog data on three interesting topics, transforming the collected blog data into vector space representation, and then performing opinion mining using both a machine learning text classifier and an unsupervised semantic orientation approach. We implemented Naïve Bayes and SO-PMI-IR algorithms for opinion mining. We obtained interesting results, which have been evaluated for correctness and also cross-validated with the outcomes of multiple techniques employed. The paper concludes with a short discussion of the results and relevance of the experimental work.