High-dimensional semantic space accounts of priming

Effects of word order and morphological information on Japanese sentence comprehension in nonfluent/agrammatic variant of primary progressive aphasia

Latent semantic analysis (LSA) has been used in several intelligent tutoring systems(ITS's) for assessing students' learning by evaluating their answers to questions in the tutoring domain. It is based on word-document co-occurrence statistics in the training corpus and a dimensionality reduction technique. However, it doesn't consider the word-order or syntactic information, which can improve the knowledge representation and therefore lead to better performance of an ITS. We present here an approach called Syntactically Enhanced LSA (SELSA) which generalizes LSA by considering a word along with its syntactic neighborhood given by the part-of-speech tag of its preceding word, as a unit of knowledge representation. The experimental results on Auto-Tutor task to evaluate students' answers to basic computer science questions by SELSA and its comparison with LSA are presented in terms of several cognitive measures. SELSA is able to correctly evaluate a few more answers than LSA but is having less correlation with human evaluators than LSA has. It also provides better discrimination of syntactic-semantic knowledge representation than LSA.

Text representation in a vector, known as embedding, is crucial for various classification tasks including sentiment analysis. It helps to process and understand natural language text more effectively. It has evolved from static approaches, such as bag-of-words and n-grams, to more dynamic approaches that consider the context and meaning of words, such as word embeddings and contextualized embeddings. Word embeddings use neural networks to learn vector representations of words based on their co-occurrence patterns in large text corpora. On the other hand, contextualized embeddings, such as BERT, consider the context of each word within a sentence or document to generate more nuanced representations. Numerous researchers have suggested modifying the original Word2Vec and BERT embeddings to include sentiment information. This paper provides a comprehensive overview of these methods by including a detailed discussion of various evaluation techniques. The paper also outlines several challenges related to embeddings that can be addressed in order to improve the results of sentiment analysis.

Job matching is a hiring process that involves a thorough understanding of the context and meaning of words in different languages. The updated and expanded latent semantic indexing (LSI) Framework seeks to improve the precision and relevance of job matching analysis of word meanings in multi-languages. Because they only compare related terms, conventional LSIs are often insufficient to address the complexity of context in job matching. Extending the LSI approach can improve the vector representation of words and help you understand the context and semantic relationships in the text. Improved LSI analyzes context more precisely by using word vector representation. Improved LSI focuses on understanding semantic relationships between words in many languages to produce more accurate and relevant job matches. This paper describes the steps involved in improving LSI, such as data collection, pre-processing, linguistic feature extraction, LSI model training, and evaluation of matching results. The results show that the examined classification model has much better performance in terms of word classification. Conventional LSI has an average prediction value of 79%, once the enhanced LSI can accurately predict about 84% of the entire word, it has a reasonable capacity to recognize the actual words in a natural context.

We apply an exemplar model of memory to explain performance in the artificial grammar task. The model blends the convolution-based method for representation developed in Jones and Mewhort's BEAGLE model of semantic memory (Psychological Review 114:1-37, 2007) with the storage and retrieval assumptions in Hintzman's MINERVA 2 model of episodic memory (Behavior Research Methods, Instruments, and Computers, 16:96-101, 1984). The model captures differences in encoding to fit data from two experiments that document the influence of encoding on implicit learning. We provide code so that researchers can adapt the model and techniques to their own experiments.

We describe a CV recommender system built for the purpose of connecting candidates with projects that are relevant to their skills. Each candidate and each project is described by a textual document (CV or a project description) from which we extract a set of skills and convert this set to a numeric representation using two known models: Latent Semantic Indexing (LSI) and Global Vectors for Word Representation (GloVe) model. Indexes built from these representations enable fast search of similar entities for a given candidate/project and the empirical results demonstrate that the obtained l2 distances correlate with the number of common skills and Jaccard similarity.

Identifying dominant topics appearing in the Journal of Cleaner Production

Implementing Latent Semantic Analysis in Learning Environments with Conversational Agents and Tutorial Dialog Arthur C. Graesser (a-graesser@memphis.edu) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Xiangen Hu (xhu@memphis.edu) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Brent A. Olde (baolde@memphis.edu) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Matthew Ventura (mventura45@hotmail.com) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Andrew Olney (aolney@hotmail.com) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Max Louwerse (mlouwers@memphis.edu) Department of Psychology, 202 Psychology Building, University of Memphis, Memphis, TN 38152 USA Donald R. Franceschetti (dfrncsch@memphis.edu) Department of Physics, University of Memphis, CAMPUS BOX 523390, Memphis, TN 38152 USA Natalie Person (person@rhodes.edu) Department of Psychology, Rhodes College, 2000 N. Parkway, Memphis, TN 38112 USA We have been developing learning environments with animated conversational agents. The agents manage a mixed-initiative dialog between the learner and the computer system either by a direct conversational interaction or by serving as a navigational guide on a web site. Two of the systems simulate human tutors by (a) presenting difficult questions that require deep reasoning, (b) attempting to comprehend the learner’s typed input, (c) formulating dialog acts that are sensitive to the learner’s contributions, and (d) speaking to the student with the animated agent. AutoTutor teaches computer literacy whereas Why/AutoTutor teaches conceptual physics (Graesser, VanLehn, Rose, Jordan, & Harter, 2001). The Human Use Regulatory Affairs Advisor (HURAA) teaches officers in the military about the ethical use of human subjects on a web site with a search facility that accesses documents through questions posed in natural language. All three systems have used Latent Semantic Analysis (LSA) as its primary representation of world knowledge. LSA is a statistical technique that compresses a large corpus texts into a space of 100-500 dimensions (Landauer, Foltz, & Laham, 1998). The K-dimensional space is used when evaluating the similarity between any two bags of words, with values ranging from 0 to 1. From the standpoint of AutoTutor and Why/AutoTutor, one bag of words is the set of assertions that a student expresses within a dialog turn; the other bag of words is the content of the curriculum script for a particular topic. From the standpoint of HURAA, one bag of words is the learner’s query in natural language and the other is a paragraph in the document space. LSA has generally been successful in evaluating the quality of student explanations, in evaluating the quality of student assertions in tutorial dialog, and in the retrieval of documents from natural language queries. Successes and failures of LSA are identified in these three learning environments. Acknowledgements This research was supported by the Office of Naval Research (N61339-01-C1006), the Institute for Defense Analyses (AK-2-1801), the National Science Foundation (REC 0106965), and the DoD Multidisciplinary University Research Initiative (MURI) administered by ONR under grant N00014-00-1-0600. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of ONR, IDA, DoD or NSF. References Graesser, A.C., VanLehn, K., Rose, C., Jordan, P., & Harter, D. (2001). Intelligent tutoring systems with conversational dialogue. AI Magazine, 22, 39-51. Landauer, T.K., Foltz, P.W., Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25, 259-284.

A procedure that processes a corpus of text and produces numeric vectors containing information about its meanings for each word is presented. This procedure is applied to a large corpus of natural language text taken from Usenet, and the resulting vectors are examined to determine what information is contained within them. These vectors provide the coordinates in a high-dimensional space in which word relationships can be analyzed. Analyses of both vector similarity and multidimensional scaling demonstrate that there is significant semantic information carried in the vectors. A comparison of vector similarity with human reaction times in a single-word priming experiment is presented. These vectors provide the basis for a representational model of semantic memory, hyperspace analogue to language (HAL).

Unstructured text data, such as emails, blogs, contracts, academic publications, organizational documents, transcribed interviews, and even tweets, are important sources of data in Information Systems research. Various forms of qualitative analysis of the content of these data exist and have revealed important insights. Yet, to date, these analyses have been hampered by limitations of human coding of large data sets, and by bias due to human interpretation. In this paper, we compare and combine two quantitative analysis techniques to demonstrate the capabilities of computational analysis for content analysis of unstructured text. Specifically, we seek to demonstrate how two quantitative analytic methods, viz., Latent Semantic Analysis and data mining, can aid researchers in revealing core content topic areas in large (or small) data sets, and in visualizing how these concepts evolve, migrate, converge or diverge over time. We exemplify the complementary application of these techniques through an examination of a 25-year sample of abstracts from selected journals in Information Systems, Management, and Accounting disciplines. Through this work, we explore the capabilities of two computational techniques, and show how these techniques can be used to gather insights from a large corpus of unstructured text.

The identification of the particular meaning for a word based on the context of its usage is commonly referred to as Word Sense Disambiguation (or WSD). Although considered a complex task, WSD is an important component of language processing and information analysis systems in several fields. Current methods for WSD rely on human input and are usually limited to a finite set of words. Complicating matters further, language is dynamic with (current) word usage changes and the introduction of new words. Static definitions created by previously defined analyses become outdated or are inadequate to deal with current usage. Fully automated methods for WSD are needed both for sense discovery and for distinguishing the sense being used for a word in context. Latent Semantic Analysis (LSA) is a candidate automated unsupervised learning system that has not been widely applied in this area. In this chapter, advanced LSA techniques are deployed as an unsupervised learning approach to the WSD tasks of sense discovery and distinguishing senses in use.

Most universities in Belgium do research in AI and offer AI courses to students. The largest part of the research is into the more classical AI subjects, such as logic programming and theorem proving. However, sub-symbolic AI is also investigated. It seems that all Belgian Universities have their own distinct flavour of research. Below, we present the different Belgian universities in alphabetical order. The Facultes Universitaires Notre Dame de la Paix Namur (University of Namur) has a group called ‘semantics and computational logic’ led by B. Le Charlier and P. Schobbens. This group is theoretically oriented and although their interests are broader than just AI, some of their research is into computational logic, automated theorem proving and formalisation of agentbased systems. The Katholieke Universiteit Leuven (Flemish catholic university of Leuven) has different groups working on artificial-intelligence subjects. First of all, there is the group on declarative languages and artificial intelligence led by M. Bruynooghe. The artificialintelligence interest of this group is into knowledge representation and machine learning, with research themes such as data mining, constraint logic programming, reasoning with time, natural-language processing and learning in agents. They are also involved in the Robocup simulation league. Second, there is a group doing research into intelligent mobile robots led by H. Van Brussel and J. De Schutter. This group is part of the mechanical engineering department. Finally, there is a group at the economy department, led by J. Vanthienen and M. Verhelst, that investigates knowledge systems and modelling. At the Limburgs Universitair Centrum (Limburg University Centre) M. Gyssens of the theoretical computer science group works on the structure and complexity of constraint satisfaction problems. J. Van den Bussche of the same group works on implementation of query languages for data mining. The Universite Catholique de Louvain (French catholic university of Louvain) has several small groups investigating different AI subjects. Computational linguistics is investigated by E. Milgrom and S. Granger. Logic programming is investigated by Y. Deville. A. Thayse investigates natural-language processing, P.J. Courtois investigates self-organising systems, P. Van Hentenryck works on optimisation and constraint satisfaction and Y. Kamp investigates collective intelligence. The Universite de Liege (Montefiore) (University of Liege) has D. Gribomont working on verification methods, automatic theorem proving and formal logic. The Universitaire Instelling Antwerpen (University of Antwerp) has the centre for Dutch language and speech, led by W. Daelemans. Part of the research of this institute is purely linguistic, but a great deal of research is also performed into (memory-based) machine learning of natural language, using large corpora of text, as well as into intelligent processing of text. The Universit Libre de Bruxelles (French Free University of Brussels) has the IRIDIA (Institut de Recherches Interdisciplinaires et de Developpements en Intelligence Artificielle) group led by M. Dorigo and H. Bersini. The main research interests are: ‘belief representation and AI-techniques for process control and classification’ and ‘nature-inspired heuristics for the solution of combinatorial and continuous space optimisation problems’. Sub-symbolic techniques are

BackgroundThe ability to automatically detect anxiety disorders from speech could be useful as a screening tool for an anxiety disorder. Prior studies have shown that individual words in textual transcripts of speech have an association with anxiety severity. Transformer-based neural networks are models that have been recently shown to have powerful predictive capabilities based on the context of more than one input word. Transformers detect linguistic patterns and can be separately trained to make specific predictions based on these patterns.ObjectiveThis study aimed to determine whether a transformer-based language model can be used to screen for generalized anxiety disorder from impromptu speech transcripts.MethodsA total of 2000 participants provided an impromptu speech sample in response to a modified version of the Trier Social Stress Test (TSST). They also completed the Generalized Anxiety Disorder 7-item (GAD-7) scale. A transformer-based neural network model (pretrained on large textual corpora) was fine-tuned on the speech transcripts and the GAD-7 to predict whether a participant was above or below a screening threshold of the GAD-7. We reported the area under the receiver operating characteristic curve (AUROC) on the test data and compared the results with a baseline logistic regression model using the Linguistic Inquiry and Word Count (LIWC) features as input. Using the integrated gradient method to determine specific words that strongly affect the predictions, we inferred specific linguistic patterns that influence the predictions.ResultsThe baseline LIWC-based logistic regression model had an AUROC value of 0.58. The fine-tuned transformer model achieved an AUROC value of 0.64. Specific words that were often implicated in the predictions were also dependent on the context. For example, the first-person singular pronoun “I” influenced toward an anxious prediction 88% of the time and a nonanxious prediction 12% of the time, depending on the context. Silent pauses in speech, also often implicated in predictions, influenced toward an anxious prediction 20% of the time and a nonanxious prediction 80% of the time.ConclusionsThere is evidence that a transformer-based neural network model has increased predictive power compared with the single word–based LIWC model. We also showed that the use of specific words in a specific context—a linguistic pattern—is part of the reason for the better prediction. This suggests that such transformer-based models could play a useful role in anxiety screening systems.

Latent semantic analysis (LSA) is a model of knowledge representation for words. It works by applying dimension reduction to local co-occurrence data from a large collection of documents after performing singular value decomposition on it. When the reduction is applied, the system forms condensed representations for the words that incorporate higher order associations. The higher order associations are primarily responsible for any semantic similarity between words in LSA. In this article, a memory model is described that creates semantic representations for words that are similar in form to those created by LSA. However, instead of applying dimension reduction, the model builds the representations by using a retrieval mechanism from a well-known account of episodic memory.

Clinical information is dominated by natural language representation of data and knowledge. To bring quantitative methods to bear in the empiric analysis of clinical episodes, they must be classified into reasonably homogenous categories that sustain inference and generalization. A tangible, if trivial, example of a classification requirement is the retrieval of patient cases relevant to the testing of a clinical hypothesis, so that they can be further scrutinized. Reliance on text word retrieval alone, drawn from natural language summaries, is fraught with contextual ambiguity and defeated by an expressively rich sub-language.