VLX-Stories: Building an Online Event Knowledge Base with Emerging Entity Detection

Zhekun Luo, Shalini Ghosh, Devin Guillory, Keizo Kato, Trevor Darrell, Huijuan Xu. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Knowledge enhancement-based large-scale natural language training model is an advanced language model that combines deep learning and knowledge enhancement. By learning from massive unlabeled data and combining with external knowledge such as knowledge graph, it breaks through the limitations of traditional models in interpretability and reasoning ability. Introducing knowledge into data-driven artificial intelligence model is an important way to realize human-machine hybrid intelligence. However, since most pre-trained models are trained on large-scale unstructured corpus data, the defects in certainty and explainability can be remedied to some extent by introducing external knowledge. To solve the above problems, we present a knowledge-enhanced large-scale natural language training model that integrates deep learning with external knowledge sources (e.g., knowledge graphs) to improve interpretability and reasoning ability. This approach addresses the limitations of traditional models trained on unstructured data by incorporating external knowledge to enhance certainty and explainability. We propose a new knowledge enhancement method and demonstrate its effectiveness through a long text representation model. This model processes structured, knowledge-rich long texts by extracting and integrating knowledge and semantic information at the sentence and document levels. It then fuses these representations to generate an enhanced long text representation. Experiments on legal case matching tasks show that our model significantly outperforms existing methods, highlighting its innovation and practical value.

Recently, neural language representation models pre-trained on large corpus can capture rich co-occurrence information and be fine-tuned in downstream tasks to improve the performance. As a result, they have achieved state-of-the-art results in a large range of language tasks. However, there exists other valuable semantic information such as similar, opposite, or other possible meanings in external knowledge graphs (KGs). We argue that entities in KGs could be used to enhance the correct semantic meaning of language sentences. In this paper, we propose a new method CKG: Dynamic Representation Based on Context and Knowledge Graph. On the one side, CKG can extract rich semantic information from a large corpus. On the other side, it can make full use of inside information such as co-occurrence in large corpus and outside information such as similar entities in KGs. We conduct extensive experiments on a wide range of tasks, including QQP, MRPC, SST-5, SQuAD, CoNLL 2003, and SNLI. The experiment results show that CKG achieves SOTA 89.2 on SQuAD compared with SAN (84.4), ELMo (85.8), and BERT_Base (88.5).

LUKE-Graph: A Transformer-based Approach with Gated Relational Graph Attention for Cloze-style Reading Comprehension

The increasing reliance on Large Language Models (LLMs) for health information seeking can pose severe risks due to the potential for misinformation and the complexity of these topics. This paper introduces KnowNet a visualization system that integrates LLMs with Knowledge Graphs (KG) to provide enhanced accuracy and structured exploration. Specifically, for enhanced accuracy, KnowNet extracts triples (e.g., entities and their relations) from LLM outputs and maps them into the validated information and supported evidence in external KGs. For structured exploration, KnowNet provides next-step recommendations based on the neighborhood of the currently explored entities in KGs, aiming to guide a comprehensive understanding without overlooking critical aspects. To enable reasoning with both the structured data in KGs and the unstructured outputs from LLMs, KnowNet conceptualizes the understanding of a subject as the gradual construction of graph visualization. A progressive graph visualization is introduced to monitor past inquiries, and bridge the current query with the exploration history and next-step recommendations. We demonstrate the effectiveness of our system via use cases and expert interviews.

Aiming at the problems of lack of background knowledge and the inconsistent response of robots in the current human-computer interaction system, we proposed a human-computer interaction model based on a knowledge graph ripple network. The model simulated the natural human communication process to realize a more natural and intelligent human-computer interaction system. This study had three contributions: first, the affective friendliness of human-computer interaction was obtained by calculating the affective evaluation value and the emotional measurement of human-computer interaction. Then, the external knowledge graph was introduced as the background knowledge of the robot, and the conversation entity was embedded into the ripple network of the knowledge graph to obtain the potential entity content of interest of the participant. Finally, the robot replies based on emotional friendliness and content friendliness. The experimental results showed that, compared with the comparison models, the emotional friendliness and coherence of robots with background knowledge and emotional measurement effectively improve the response accuracy by 5.5% at least during human-computer interaction.

Knowledge-aware multi-center clinical dataset adaptation: Problem, method, and application

Traditional Chinese Medicine (TCM), with its extensive knowledge base documented in ancient texts, offers a unique resource for contemporary drug discovery, particularly in combatting microbial infections. The success of antimalarial drugs like artemisinin and artesunate, derived from the TCM herb Artemisia annua L., exemplifies the potential of TCM-derived small molecules. This rich repository of natural products and intricate molecular structures could reveal novel compounds with unexplored mechanisms of action. Our study employs a multifaceted approach that combines text mining, detailed textual analysis, and modern antibacterial molecular prediction methodologies to unlock the potential of ancient TCM remedies. We use external knowledge maps, which include databases of known bioactive compounds and their targets, to identify promising TCM candidates. This approach leverages both historical texts and contemporary scientific data to explore the therapeutic potential of TCM. We discovered that herb patterns DiYu→ZeXie and Kushen→ShengJiang potentially combat both Grams-positive and Grams-negative bacteria. We utilized the AntiBac-Pred online tool to identify and analyze the chemical components of herbs, integrating data from ancient texts and TCMDB@Taiwan external knowledge graph. The DiYu→ZeXie groups showed antimicrobial potential against resistant Staphylococcus simulans, while the Kushen→ShengJiang groups exhibited dual antimicrobial effects against Bacillus subtilis. Exploring TCM's extensive repository offers numerous opportunities for discovering therapeutically active compounds. Our synergistic approach, which combines ancient wisdom with modern science, holds significant promise for enhancing our ability to combat infectious diseases. This method could pave the way for a new era of personalized medicine, addressing the urgent need for innovative treatments against multidrug-resistant bacteria and viruses.

Construct a knowledge graph is time-consuming and the knowledge graph in the scientific domain requires extremely high labor costs due to it requires high prior knowledge to extract knowledge from resources. To build a scientific research knowledge graph, the most of input are papers, patent, the description of their project and some national program (such as National High Technology Research and Development Program of China, Major State Basic Research Development Program of China, General Program, Key Program and Major Program) which all of them are unstructured data, that make human participation are mostly necessary to measure the quality. In this paper, we design and proposed a framework using active learning; this framework can be used to extract entity and relation from unstructured science and technology research data. This framework combines the human and machine learning approach together, which is active learning, to help user extract entity from those unstructured data with less time cost. By using those data to construct a CKG as annotation label, it further implements active learning tools and helps the expert to rapidly annotate the data with high accuracy. Those knowledge graph constructed by this framework can be used to finding similar research area, finding similar researchers, finding popular research areas and so on.

Integrating commonsense knowledge with GPT embeddings for emotion classification☆

Event Detection (ED) aims to identify event trigger words from a given text and classify it into an event type. Most current methods to ED rely heavily on training instances, and almost ignore the correlation of event types. Hence, they tend to suffer from data scarcity and fail to handle new unseen event types. To address these problems, we formulate ED as a process of event ontology population: linking event instances to pre-defined event types in event ontology, and propose a novel ED framework entitled OntoED with ontology embedding. We enrich event ontology with linkages among event types, and further induce more event-event correlations. Based on the event ontology, OntoED can leverage and propagate correlation knowledge, particularly from data-rich to data-poor event types. Furthermore, OntoED can be applied to new unseen event types, by establishing linkages to existing ones. Experiments indicate that OntoED is more predominant and robust than previous approaches to ED, especially in data-scarce scenarios.

Decision-making tasks usually follow five steps: identifying the problem, collecting data, extracting evidence, iden-tifying arguments, and making the decision. This paper focuses on two steps of decision-making: extracting evidence by building knowledge graphs (KGs) of specialized topics and identifying sentences' arguments through sentence-level argument mining. We present a hybrid model that combines topic modeling using latent Dirichlet allocation (LDA) and word embeddings to obtain external knowledge from structured and unstructured data. We use a topic model to extract topic- and sentence-specific evidence from the structured knowledge base Wikidata. A knowledge graph is constructed based on the cosine similarity between the entity word vectors of Wikidata and the vector of the given sentence. A second graph based on topic-specific articles found via Google supplements the general incompleteness of the structured knowledge base. Combining these graphs, we obtain a graph-based model that, as our evaluation shows, successfully capitalizes on both structured and unstructured data.

Until recently, structured (e.g., relational) and unstructured (e.g., textual) data were managed very differently: Structured data was queried declaratively using languages such as SQL, while unstructured data was searched using boolean queries over inverted indices. Today, we witness the rapid emergence of Big Data Integration techniques leveraging knowledge graphs to bridge the gap between different types of contents and integrate both unstructured and structured information more effectively. I will start this talk by giving a few examples of Big Data Integration. I will then describe two recent systems built in my lab and leveraging such techniques: ZenCrowd, a socio-technical platform that automatically connects Web documents to semi-structured entities in a knowledge graph, and Guider, a Big Data Integration system for the cloud.

In this paper, we propose a dual-process cognitive recommendation system for sequential recommendations. The framework includes an intuitive representation module (System 1) and an inference module (System 2). System 1 is designed to understand the user’s historical interaction sequences with external knowledge graph. System 2 is built to make recommendations by reinforcement learning to consider long-term returns and diversity. We demonstrate the performance of our method on a wide range of recommendation datasets. Experiments show significant improvement over the state-of-the-art models regarding both relevance and diversity.KeywordsRecommendation systemKnowledge graphReinforcement learningDiversity in recommendationDual-process theory

Open-set action recognition (OSAR) aims to learn a recognition framework capable of both classifying known classes and identifying unknown actions in open-set scenarios. Existing OSAR methods typically reside in a data-driven paradigm, which ignore the rich semantics in both known and unknown categories. In fact, we humans have the capability of leveraging the captured semantic information, i.e., knowledge and experience, to incisively distinguish samples from known and unknown classes. Motivated by this observation, in this paper, we propose a Unified Semantic Exploration (USE) framework for recognizing actions in open-set scenarios. Specifically, we explore the explicit knowledge semantics by simulating the unknown classes with knowledge-guided virtual classes based on an external knowledge graph, which enables the model to simulate open-set perception during model training. Besides, we propose to learn the implicit data semantics by transferring the knowledge structure of action categories to the visual prototype space for semantic structure preservation. Extensive experiments on several action recognition benchmarks validate the effectiveness of our proposed method.