Disambiguating prepositional phrase attachment sites with sense information captured in contextualized distributional data

To save time, healthcare providers frequently use abbreviations while authoring clinical documents. Nevertheless, abbreviations that authors deem unambiguous often confuse other readers, including clinicians, patients, and natural language processing (NLP) systems. Most current clinical NLP systems "post-process" notes long after clinicians enter them into electronic health record systems (EHRs). Such post-processing cannot guarantee 100% accuracy in abbreviation identification and disambiguation, since multiple alternative interpretations exist. Authors describe a prototype system for real-time Clinical Abbreviation Recognition and Disambiguation (rCARD) - i.e., a system that interacts with authors during note generation to verify correct abbreviation senses. The rCARD system design anticipates future integration with web-based clinical documentation systems to improve quality of healthcare records. When clinicians enter documents, rCARD will automatically recognize each abbreviation. For abbreviations with multiple possible senses, rCARD will show a ranked list of possible meanings with the best predicted sense at the top. The prototype application embodies three word sense disambiguation (WSD) methods to predict the correct senses of abbreviations. We then conducted three experments to evaluate rCARD, including 1) a performance evaluation of different WSD methods; 2) a time evaluation of real-time WSD methods; and 3) a user study of typing clinical sentences with abbreviations using rCARD. Using 4,721 sentences containing 25 commonly observed, highly ambiguous clinical abbreviations, our evaluation showed that the best profile-based method implemented in rCARD achieved a reasonable WSD accuracy of 88.8% (comparable to SVM - 89.5%) and the cost of time for the different WSD methods are also acceptable (ranging from 0.630 to 1.649 milliseconds within the same network). The preliminary user study also showed that the extra time costs by rCARD were about 5% of total document entry time and users did not feel a significant delay when using rCARD for clinical document entry. The study indicates that it is feasible to integrate a real-time, NLP-enabled abbreviation recognition and disambiguation module with clinical documentation systems.

This paper investigates whether high-quality annotations for tasks involving semantic disambiguation can be obtained without a major investment in time or expense. We examine the use of untrained human volunteers from Amazons Mechanical Turk in disambiguating prepositional phrase (PP) attachment over sentences drawn from the Wall Street Journal corpus. Our goal is to compare the performance of these crowdsourced judgments to the annotations supplied by trained linguists for the Penn Treebank project in order to indicate the viability of this approach for annotation projects that involve contextual disambiguation. The results of our experiments on a sample of the Wall Street Journal corpus show that invoking majority agreement between multiple human workers can yield PP attachments with fairly high precision. This confirms that a crowdsourcing approach to syntactic annotation holds promise for the generation of training corpora in new domains and genres where high-quality annotations are not available and difficult to obtain.

This paper presents a method of word sense disambiguation that assigns a target word the sense that is most related to the senses of its neighbor words. We explore the use of measures of relatedness between word senses based on a novel hybrid approach. First, we investigate how to “literally” and “regularly” express a “concept”. We apply set algebra to Wordnet's synsets cooperating with Wordnet's word ontology. In this way we establish regular rules for constructing various representations (lexical notations) of a concept using Boolean operators and word forms in various synset(s) defined in Wordnet. Then we establish a formal mechanism for quantifying and estimating the semantic relatedness between concepts – we facilitate “concept distribution statistics” to determine the degree of semantic relatedness between two lexically expressed concepts. Human languages have words that can mean different things in different contexts, such words with multiple meanings are potentially “ambiguous”. The process of “deciding which of several meanings of a term is intended in a given context” is known as “Word Sense Disambiguation (WSD)”. The proposed method is not supervised, and does not require any manually created sense‐tagged training examples. The experimental results showed good performance on Semcor, a subset of the Brown corpus. We observe that measures of semantic relatedness are useful sources of information for word sense disambiguation.

Word-sense recognition and disambiguation (WERD) is the task of identifying word phrases and their senses in natural language text. Though it is well understood how to disambiguate noun phrases, this task is much less studied for verbs and verbal phrases. We present Werdy, a framework for WERD with particular focus on verbs and verbal phrases. Our framework first identifies multi-word expressions based on the syntactic structure of the sentence; this allows us to recognize both contiguous and non-contiguous phrases. We then generate a list of candidate senses for each word or phrase, using novel syntactic and semantic pruning techniques. We also construct and leverage a new resource of pairs of senses for verbs and their object arguments. Finally, we feed the so-obtained candidate senses into standard word-sense disambiguation (WSD) methods, and boost their precision and recall. Our experiments indicate that Werdy significantly increases the performance of existing WSD methods.

Word sense disambiguation is important for many applications in natural language processing fields including machine translation, information retrieval and automatic summarization. In this paper, left word unit and right word unit are extracted for improving the quality of word sense disambiguation (WSD) starting from the target polysemous word. Their semantic knowledge is mined from Tongyici Cilin which is a Chinese semantic lexicon. A new method of word sense disambiguation is proposed with semantic information of left word unit and right word unit. The classifier of word sense disambiguation is built based on bayesian model. SemEval-2007: Task#5 is used as training corpus and test corpus. Experimental results show that the disambiguation classifier’s precision is improved and demonstrate the effectiveness of the method.

Word sense disambiguation (WSD) is a fundamental natural language processing task. Unsupervised knowledge-based WSD only relies on a lexical knowledge base as the sense inventory and has wider practical use than supervised WSD that requires a mass of sense-annotated data. HowNet is the most widely used lexical knowledge base in Chinese WSD. Because of its uniqueness, however, most of existing unsupervised WSD methods cannot work for HowNet-based WSD, and the tailor-made methods have not obtained satisfying results. In this paper, we propose a new unsupervised method for HowNet-based Chinese WSD, which exploits the masked language model task of pre-trained language models. In experiments, considering existing evaluation dataset is small and out-of-date, we build a new and larger HowNet-based WSD dataset. Experimental results demonstrate that our model achieves significantly better performance than all the baseline methods. All the code and data of this paper are available at https://github.com/thunlp/SememeWSD.

BackgroundIn recent years, deep learning methods have been applied to many natural language processing tasks to achieve state-of-the-art performance. However, in the biomedical domain, they have not out-performed supervised word sense disambiguation (WSD) methods based on support vector machines or random forests, possibly due to inherent similarities of medical word senses.ResultsIn this paper, we propose two deep-learning-based models for supervised WSD: a model based on bi-directional long short-term memory (BiLSTM) network, and an attention model based on self-attention architecture. Our result shows that the BiLSTM neural network model with a suitable upper layer structure performs even better than the existing state-of-the-art models on the MSH WSD dataset, while our attention model was 3 or 4 times faster than our BiLSTM model with good accuracy. In addition, we trained “universal” models in order to disambiguate all ambiguous words together. That is, we concatenate the embedding of the target ambiguous word to the max-pooled vector in the universal models, acting as a “hint”. The result shows that our universal BiLSTM neural network model yielded about 90 percent accuracy.ConclusionDeep contextual models based on sequential information processing methods are able to capture the relative contextual information from pre-trained input word embeddings, in order to provide state-of-the-art results for supervised biomedical WSD tasks.

A neurocomputational model based on emergent massively overlapping neural cell assemblies (CAs) for resolving prepositional phrase (PP) attachment ambiguity is described. PP attachment ambiguity is a well-studied task in natural language processing and is a case where semantics is used to determine the syntactic structure. A large network of biologically plausible fatiguing leaky integrate-and-fire neurons is trained with semantic hierarchies (obtained from WordNet) on sentences with PP attachment ambiguity extracted from the Penn Treebank corpus. During training, overlapping CAs representing semantic similarities between the component words of the ambiguous sentences emerge and then act as categorizers for novel input. The resulting average resolution accuracy of 84.56% is on par with known machine learning algorithms.

For most Web searching applications, queries are commonly ambiguous because words usually contain several meanings. Traditional Word Sense Disambiguation (WSD) methods use statistic models or ontology-based knowledge models to find the most appropriate sense for the ambiguous word. Since queries are usually short, the contexts of the queries may not always provide enough information for disambiguating queries. Thus, more than one interpretation may be found for one ambiguous query. In this paper, we propose a cluster-based WSD method, which finds out all appropriate interpretations for the query. Because some senses of one ambiguous word usually have very close semantic relations, we group those similar senses together for explaining the ambiguous word in one interpretation. If the cluster-based WSD method generates several contradictory interpretations for one ambiguous query, we extract users’ preferences from clickthrough data, and determine suitable concepts or concepts’ clusters that meet users’ interests for explaining the ambiguous query.

For most Web searching applications, queries are commonly ambiguous because words usually contain several meanings. Traditional Word Sense Disambiguation (WSD) methods use statistic models or ontology-based knowledge models to find the most appropriate sense for the ambiguous word. Since queries are usually short, the contexts of the queries may not always provide enough information for disambiguating queries. Thus, more than one interpretation may be found for one ambiguous query. In this paper, we propose a cluster-based WSD method, which finds out all appropriate interpretations for the query. Because some senses of one ambiguous word usually have very close semantic relations, we group those similar senses together for explaining the ambiguous word in one interpretation. If the cluster-based WSD method generates several contradictory interpretations for one ambiguous query, we extract users’ preferences from clickthrough data, and determine suitable concepts or concepts’ clusters that meet users’ interests for explaining the ambiguous query.

This paper presents a method of word sense disambiguation that assigns a target word the sense that is most related to the senses of its neighbor words. We explore the use of measures of relatedness between word senses based on a novel hybrid approach. First, we investigate how to literally and regularly express a concept. We apply set algebra to Wordnet’s synsets cooperating with Wordnet’s word ontology. In this way we establish regular rules for constructing various representations (lexical notations) of a using Boolean operators and various word forms in synset(s). Then we construct a formal mechanism for quantifying and estimating the semantic relatedness between concepts - we facilitate concept distribution statistics to determine the degree of semantic relatedness between two lexically expressed concepts. Then we applied the measure of semantic relatedness to the WSD task. The experimental results showed good performance on Semcor, a subset of Brown corpus. We observe that measures of semantic relatedness are useful sources of information for word sense disambiguation.

Sprinkled semantic diffusion kernel for word sense disambiguation

This paper describes our system participating in the SemEval-2017 Task 7, for the subtasks of homographic pun location and homographic pun interpretation. For pun interpretation, we use a knowledge-based Word Sense Disambiguation (WSD) method based on sense embeddings. Pun-based jokes can be divided into two parts, each containing information about the two distinct senses of the pun. To exploit this structure we split the context that is input to the WSD system into two local contexts and find the best sense for each of them. We use the output of pun interpretation for pun location. As we expect the two meanings of a pun to be very dissimilar, we compute sense embedding cosine distances for each sense-pair and select the word that has the highest distance. We describe experiments on different methods of splitting the context and compare our method to several baselines. We find evidence supporting our hypotheses and obtain competitive results for pun interpretation.

Word sense disambiguation (WSD) refers to the task of finding the intended meaning of a word in a given sentence. WordNet® (lexical database based on psychological principals) based on WSD methods have been discussed extensively. WordNet® relates words through various relations and all the WSD approaches available in the literature till date rely on the notion of assigning equal importance to each relation. But actually in real-world situations, these relations do not exhibit equal importance, so they should not be given equal weight. The paper presents a novel method for exploiting the idea of WordNet® relations weight based on its importance and thus assigns the weights to the edges of the WordNet® graph, µ ∊ (0, 1]. This helps in fuzzifying the WordNet® graph, where nodes represent the words and weighted edges represent the relation of the words with strength∈ (0,1]. The values for edge weight assignment are obtained using test cases based on a simulated annealing method. The proposed method utilizes various Fuzzy graph connectivity measures to determine the significance of each node in the Fuzzy graph, resulting in identification of the intended meaning of the word. This method is tested on the SemCor dataset and observed that it gives better results when compared to state-of-the-art approaches.

Word sense disambiguation (WSD) is a fundamental natural language processing task. We propose a new unsupervised method for HowNet-based Chinese WSD, which exploits the masked language model task of pre-trained language models. In experiments, considering the existing evaluation dataset is small and out-of-date, we build a new and larger HowNet-based WSD dataset. Experimental results demonstrate that our model achieves significantly better performance than all the baseline methods. In this presentation, we introduce both our method and the new dataset.