Challenging distributional models with a conceptual network of philosophical terms

This demonstration presents an infrastructure for computing multilingual semantic relatedness and correlation for twelve natural languages by using three distributional semantic models (DSMs). Our demonsrator - DInfra (Distributional Infrastructure) provides researchers and developers with a highly useful platform for processing large-scale corpora and conducting experiments with distributional semantics. We integrate several multilingual DSMs in our webservice so the end user can obtain a result without worrying about the complexities involved in building DSMs. Our webservice allows the users to have easy access to a wide range of comparisons of DSMs with different parameters. In addition, users can configure and access DSM parameters using an easy to use API.

Distributional semantics develops theories and methods to represent the meaning of natural language expressions, with vectors encoding their statistical distribution in linguistic contexts. It is at once a theoretical model to express meaning, a practical methodology to construct semantic representations, a computational framework for acquiring meaning from language data, and a cognitive hypothesis about the role of language usage in shaping meaning. This book aims to build a common understanding of the theoretical and methodological foundations of distributional semantics. Beginning with its historical origins, the text exemplifies how the distributional approach is implemented in distributional semantic models. The main types of computational models, including modern deep learning ones, are described and evaluated, demonstrating how various types of semantic issues are addressed by those models. Open problems and challenges are also analyzed. Students and researchers in natural language processing, artificial intelligence, and cognitive science will appreciate this book.

This work describes SemR-11, a multi-lingual dataset for evaluating semantic similarity and relatedness for 11 languages (German, French, Russian, Italian, Dutch, Chinese, Portuguese, Swedish, Spanish, Arabic and Persian). Semantic similarity and relatedness gold standards have been initially used to support the evaluation of semantic distance measures in the context of linguistic and knowledge resources and distributional semantic models. SemR-11 builds upon the English gold-standards of Miller & Charles (MC), Rubenstein & Goodenough (RG), WordSimilarity 353 (WS-353), and Simlex-999, providing a canonical translation for them. The final dataset consists of 15,917 word pairs and can be used to support the construction and evaluation of semantic similarity/relatedness and distributional semantic models. As a case study, the SemR-11 test collections was used to investigate how different distributional semantic models built from corpora in different languages and with different sizes perform in computing semantic relatedness similarity and relatedness tasks.

Multi-Ontology Refined Embeddings (MORE): A hybrid multi-ontology and corpus-based semantic representation model for biomedical concepts.

Schizophrenia is widely known to manifest in language disturbance. Namely, speech incoherence, tangentiality, derailment are indicative of thought disorder characteristic of schizophrenia. Recent advances in distributional semantics have made it possible to measure coherence in text in a unified and objective manner. It has been shown that semantic coherence measures based on distributional semantic models in English speech significantly contribute to schizophrenia diagnosis prediction and correlate with thought disorder measures. However, information on other languages and modes is either contradictory or unavailable. The goal of the current paper is to analyze semantic coherence in schizophrenia in Russian written texts. We present a dataset of short texts written by patients diagnosed with schizophrenia and matched healthy control subjects. We have developed a number of semantic coherence measures, both replicating findings in other languages and novel ones. Our results show that in Russian written texts by patients diagnosed with schizophrenia semantic coherence values are contradictory to the findings obtained for spoken English texts. However, semantic coherence in our dataset provides an effective diagnosis predictor. We discuss our results in terms of possible theoretic interpretation and outline further steps to semantic coherence measurement in schizophrenia.

The present paper intends to draw the conception of language implied in the technique of word embeddings that supported the recent development of deep neural network models in computational linguistics. After a preliminary presentation of the basic functioning of elementary artificial neural networks, we introduce the motivations and capabilities of word embeddings through one of its pioneering models, word2vec. To assess the remarkable results of the latter, we inspect the nature of its underlying mechanisms, which have been characterized as the implicit factorization of a word-context matrix. We then discuss the ordinary association of the “distributional hypothesis” with a “use theory of meaning,” often justifying the theoretical basis of word embeddings, and contrast them to the theory of meaning stemming from those mechanisms through the lens of matrix models (such as vector space models and distributional semantic models). Finally, we trace back the principles of their possible consistency through Harris’s original distributionalism up to the structuralist conception of language of Saussure and Hjelmslev. Other than giving access to the technical literature and state of the art in the field of natural language processing to non-specialist readers, the paper seeks to reveal the conceptual and philosophical stakes involved in the recent application of new neural network techniques to the computational treatment of language.

Noun compounds, consisting of two nouns (the head and the modifier) that are combined into a single concept, differ in terms of their plausibility: school bus is a more plausible compound than saddle olive. The present study investigates which factors influence the plausibility of attested and novel noun compounds. Distributional Semantic Models (DSMs) are used to obtain formal (vector) representations of word meanings, and compositional methods in DSMs are employed to obtain such representations for noun compounds. From these representations, different plausibility measures are computed. Three of those measures contribute in predicting the plausibility of noun compounds: The relatedness between the meaning of the head noun and the compound (Head Proximity), the relatedness between the meaning of modifier noun and the compound (Modifier Proximity), and the similarity between the head noun and the modifier noun (Constituent Similarity). We find non-linear interactions between Head Proximity and Modifier Proximity, as well as between Modifier Proximity and Constituent Similarity. Furthermore, Constituent Similarity interacts non-linearly with the familiarity with the compound. These results suggest that a compound is perceived as more plausible if it can be categorized as an instance of the category denoted by the head noun, if the contribution of the modifier to the compound meaning is clear but not redundant, and if the constituents are sufficiently similar in cases where this contribution is not clear. Furthermore, compounds are perceived to be more plausible if they are more familiar, but mostly for cases where the relation between the constituents is less clear.

Recent years have seen a growing interest within the natural language processing (NLP) community in evaluating the ability of semantic models to capture human meaning representation in the brain. Existing research has mainly focused on applying semantic models to decode brain activity patterns associated with the meaning of individual words, and, more recently, this approach has been extended to sentences and larger text fragments. Our work is the first to investigate metaphor processing in the brain in this context. We evaluate a range of semantic models (word embeddings, compositional, and visual models) in their ability to decode brain activity associated with reading of both literal and metaphoric sentences. Our results suggest that compositional models and word embeddings are able to capture differences in the processing of literal and metaphoric sentences, providing support for the idea that the literal meaning is not fully accessible during familiar metaphor comprehension.

Distributional social semantics: Inferring word meanings from communication patterns

This article aims to explore the nuanced semantic differentiation between the terms “path” and “road” by employing the analytical framework established by Berejan S.G. alongside contemporary distributional semantics models. The focus of the research lies in evaluating the semantic similarities and distinctions inherent in the formula-syntagm “path – road”, utilizing both traditional methodologies and modern computational techniques. Through this comparative analysis, the study seeks to shed light on how varying approaches can yield different insights into the meanings and associations of these two concepts. By combining Berejan's analytical framework with distributional semantics models, researchers aim to uncover subtleties in meaning that may not be apparent through traditional methods alone. This interdisciplinary approach allows for a more comprehensive understanding of the nuances between “path” and “road”, ultimately contributing to the larger conversation surrounding language and cognition. By delving into both the historical context and contemporary applications of these terms, the study offers valuable insights into the dynamic nature of semantic relationships and the ways in which they shape our perceptions of the world around us.

Distributional models of lexical semantics are capable of acquiring sophisticated representations of word meanings. The main theoretical insight provided by these models is that they demonstrate the systematic connection between the knowledge that people acquire and the experience that they have with the natural language environment. However, linguistic experience is inherently variable and differs radically across people due to demographic and cultural variables. Recently, distributional models have been used to examine how word meanings vary across languages and it was found that there is considerable variability in the meanings of words across languages for most semantic categories. The goal of this article is to examine how variable word meanings are across individual language users within a single language. This was accomplished by assembling 500 individual user corpora attained from the online forum Reddit. Each user corpus ranged between 3.8 and 32.3 million words each, and a count-based distributional framework was used to extract word meanings for each user. These representations were then used to estimate the semantic alignment of word meanings across individual language users. It was found that there are significant levels of relativity in word meanings across individuals, and these differences are partially explained by other psycholinguistic factors, such as concreteness, semantic diversity, and social aspects of language usage. These results point to word meanings being fundamentally relative and contextually fluid, with this relativeness being related to the individualized nature of linguistic experience.

Reports an error in "Determining the optimal environmental information for training computational models of lexical semantics and lexical organization" by Brendan T. Johns (Canadian Journal of Experimental Psychology / Revue canadienne de psychologie expérimentale, 2024[Sep], Vol 78[3], 163-173; see record 2025-18520-001). In the article, the bar graph for Figure 4 is not the correct graph. The correct graph is provided in the erratum. (The following abstract of the original article appeared in record 2025-18520-001.) Experiential theories of cognition propose that the external environment shapes cognitive processing, shifting emphasis from internal mechanisms to the learning of environmental structure. Computational modelling, particularly distributional models of lexical semantics (e.g., Landauer & Dumais, 1997) and models of lexical organization (e.g., Johns, 2021a), exemplifies this, highlights the influence of language experience on cognitive representations. While these models have been successful, comparatively less attention has been paid to the training materials used to train these models. Recent research has explored the role of social/communicatively oriented training materials on models of lexical semantics and organization (Johns, 2021a, 2021b, 2023, 2024), introducing discourse- and user-centred text training materials. However, determining the optimal training materials for these two model types remains an open question. This article addresses this problem by using experiential optimization (Johns, Jones, & Mewhort, 2019), which selects the materials that maximize model performance. This study will use experiential optimization to compare user-based and discourse-based corpora in optimizing models of lexical organization and semantics, offering insight into pathways towards integrating cognitive models in these areas. (PsycInfo Database Record (c) 2025 APA, all rights reserved).

Tasks such as question answering and semantic search are dependent on the ability of querying & reasoning over large-scale commonsense knowledge bases (KBs). However, dealing with commonsense data demands coping with problems such as the increase in schema complexity, semantic inconsistency, incompleteness and scalability. This paper proposes a selective graph navigation mechanism based on a distributional relational semantic model which can be applied to querying & reasoning over heterogeneous knowledge bases (KBs). The approach can be used for approximative reasoning, querying and associational knowledge discovery. In this paper we focus on commonsense reasoning as the main motivational scenario for the approach. The approach focuses on addressing the following problems: (i) providing a semantic selection mechanism for facts which are relevant and meaningful in a specific reasoning & querying context and (ii) allowing coping with information incompleteness in large KBs. The approach is evaluated using ConceptNet as a commonsense KB, and achieved high selectivity, high scalability and high accuracy in the selection of meaningful navigational paths. Distributional semantics is also used as a principled mechanism to cope with information incompleteness.

This article presents Luppar, an Information Retrieval tool for closed collections of documents which uses a local distributional semantic model associated to each corpus. The system performs automatic query expansion using a combination of distributional semantic model and local context analysis and supports relevancy feedback. The performance of the system was evaluated in databases of different domains and presented results equal to or higher than those published in the literature.

The process of searching and understanding existing vocabularies (terminological artifacts) on the Linked Data Web is an intrinsic activity to the consumption and production of Linked Data. Data consumers trying to find and understand the vocabularies behind datasets in order to query them, or data producers searching for existing resources to describe their data, face the challenge of semantically searching existing concepts in vocabularies. Traditional search mechanisms do not address the level of semantic matching necessary to match users' information needs to vocabulary elements, bringing an additional barrier to the consumption and production of Linked Data on the Web. This work describes a terminological search mechanism which uses a distributional semantic model to provide a best-effort semantic matching solution. The distributional semantic model leverages the semantic information present in large volumes of unstructured text to improve the semantic matching capabilities of the search process. A quantitative evaluation of the quality of the search results shows that the approach provides an effective semantic matching mechanism for terminological search.