Ontology and Knowledge Graphs for Natural Language Processing

Knowledge graphs have shown increasing value in semantic search, intelligent Q&A, data analysis, natural language processing, visual understanding, IoT devices, etc. It is undeniable that knowledge graphs have become the mandatory path for the artificial intelligence fields development. However, the information contained in existing knowledge graphs is incomplete, which attracts a large number of researchers to enhance the completeness of knowledge graphs utilizing knowledge graph completion methods. Most traditional embedding-based knowledge graph completion models use structural information within data-rich triples from the knowledge graph is limited by the long-tail distribution of the relations in the triples. To address this problem, we propose a Bert-based knowledge graph completion algorithm for few-shot knowledge graphs, with the main goal of implementing the knowledge graph completion task with only a few sample triples of training instances. We improve the baseline model GMatching for handling few-shot knowledge graphs by introducing the Bert pre-trained linguistic representation model to enhance the semantic representation of entities and relations in the triples. Through experiments, we demonstrate that our improved model B-GMatching achieves good results.

The research results presented in this paper were obtained as part of the D2KAB project (Data to Knowledge in Agriculture and Biodiversity) which aims to develop semantic web-based tools to describe and make agronomical data actionable and accessible following the FAIR principles. We focus on constructing domain-specific Knowledge Graphs (KGs) from textual data sources, using Natural Language Processing (NLP) techniques to extract and structure relevant entities. Our approach is based on the formalization of a semantic data model using common linked open vocabularies such as the Web Annotation Ontology (OA) and the Provenance Ontology (PROV). The model was developed by formulating motivating scenarios and competency questions from domain experts. This model has been used to construct three different KGs from three distinct corpora: PubMed scientific publications on wheat and rice genetics and phenotyping, and French agricultural alert bulletins. The named entities to be recognized include genes, phenotypes, traits, genetic markers, taxa and phenological stages normalized using semantic resources such as the Wheat Trait and Phenotype Ontology (WTO), the French Crop Usage (FCU) thesaurus and the Plant Phenological Description Ontology (PPDO). Named entities were extracted using different NLP approaches and tools. The relevance of the semantic model was validated by implementing experts questions as SPARQL queries to be answered on the constructed RDF knowledge graphs. Our work demonstrates how domain-specific vocabularies and systematic querying of KGs can reveal hidden interactions and support agronomists in navigating vast amounts of data. The resources and transformation pipelines developed are publicly available in Git repositories.

In recent years, the ever-increasing quantities of entities in large knowledge bases on the Web, such as DBpedia, Freebase and YAGO, pose new challenges but at the same time open up new opportunities for intelligent information access. These knowledge bases (KBs) have become valuable resources in many research areas, such as natural language processing (NLP) and information retrieval (IR). Recently, almost every major commercial Web search engine has incorporated entities into their search process, including Google’s Knowledge Graph, Yahoo!’s Web of Objects and Microsoft’s Satori Graph/Bing Snapshots. The goal is to bridge the semantic gap between natural language text and formalized knowledge. Within the context of globalization, multilingual and cross-lingual access to information has emerged as an issue of major interest. Nowadays, more and more people from different countries are connecting to the Internet, in particular the Web, and many users can understand more than one language. While the diversity of languages on the Web has been growing, for most people there is still very little content in their native language. As a consequence of the ability to understand more than one language, users are also interested in Web content in other languages than their mother tongue. There is an impending need for technologies that can help in overcoming the language barrier for multilingual and cross-lingual information access. In this thesis, we face the overall research question of how to allow for semantic-aware and cross-lingual processing of Web documents and user queries by leveraging knowledge bases. With the goal of addressing this complex problem, we provide the following solutions: (1) semantic annotation for addressing the semantic gap between Web documents and knowledge; (2) semantic search for coping with the semantic gap between keyword queries and knowledge; (3) the exploitation of cross-lingual semantics for overcoming the language barrier between natural language expressions (i.e., keyword queries and Web documents) and knowledge for enabling cross-lingual semantic annotation and search. We evaluated these solutions and the results showed advances beyond the state-of-the-art. In addition, we implemented a framework of cross-lingual semantic annotation and search, which has been widely used for cross-lingual processing of media content in the context of our research projects.

With the continuous development of new media technology, the spiritual needs of the masses have been greatly satisfied and the aesthetic ability has also been significantly improved compared with the past. From the current point of view, “literary works,” as the spiritual food of contemporary people, are promoting social spirit. The use of natural language processing and knowledge graph technology can improve cultural cognition to promote the dissemination and development of classic English literature, which has become a necessary means of dissemination of classic English literature. Most of the existing classic English literary works are appreciated based on modern literature datasets. Nowadays, with the continuous development of new media technology, there are fewer studies on the dissemination and cultural cognition of classic English literary works. This makes it impossible for readers to obtain cultural cognition from classic English literary works, making it difficult for the dissemination and development of classic English literary works. In view of the above problems, using natural language processing and knowledge graph technology, taking Shakespeare's play “Hamlet” represented by classic English literary works as an example, the research on the construction method of knowledge graph is carried out and the cultural characteristics in literary works are extracted and analyzed. In parsing, a bidirectional gated recurrent unit network model based on hybrid character embedding is proposed. Based on n-gram embedding, by combining pretraining embedding and radical embedding, it can fully consider the rich semantic information in English literature works to extract. Feature: in terms of named entity recognition, based on the existing iterative atrous convolutional network model, an iterative atrous convolutional network model is proposed. To get the best sequence label and get the last labeled entity information, in terms of knowledge graph construction and visual query, a workflow method for building knowledge graph from unstructured text is proposed and a flask-based knowledge graph visual query system is designed, which applies the best model of the above two tasks. We decode the complete “Hamlet” text, extract entities and their semantic links as nodes and relationships in the knowledge graph, store knowledge through the graph database, and finally form a visual query system that combines the front and back end.

Knowledge graph completion is an important research issue in knowledge graph construction, knowledge engineering, and natural language processing. A knowledge graph is a knowledge support for realizing accurate knowledge services in general and professional fields. It is also an important breakthrough foundation in information retrieval, question-and-answer interactions, and information recommendation. The low quality and small scale of the knowledge graph are the main bottlenecks that hinder its wide applications. The purpose of knowledge graph completion is to build a large-scale and high-quality knowledge graph for continuously updating and expanding the knowledge graph. Aiming at the difficulty of knowledge graph completion methods to extract deep semantic features from auxiliary information, such as unstructured texts, this study proposes a knowledge graph completion method based on parsing graph embedding and weighted graph convolutional network. This method uses the weighted graph convolutional network to model the semantic dependency parsing of the entity description and construct the semantic dependency parsing graph embedding. Furthermore, it introduces a multi-grained sentence-embedding generation method of the entity description, which is intended to build entity representation learning that can capture multi-grained semantics and deep-level semantic features. The experimental results on two public datasets show that the proposed knowledge graph completion approach outperforms the existing methods, thereby demonstrating its effectiveness and superiority.

A knowledge graph (KG) consists of numerous triples, in which each triple, i.e., (head entity, relation, tail entity), denotes a real-world assertion. Many large-scale KGs have been developed, e.g., general-purpose KGs Freebase and YAGO. Also, lots of domain-specific KGs are emerging, e.g., COVID-19 KGs, biomedical KGs, and agricultural KGs. By embedding KGs into low-dimensional vectors, i.e., representations of entities and relations, we could integrate KGs into machine learning models and enhance the performance of many prediction tasks, including search, recommendations, and question answering. During the construction, refinement, embedding, and application of KGs, a variety of KG learning algorithms have been developed to handle challenges in various real-world scenarios. Moreover, graph neural networks have also brought new opportunities to KG learning. This workshop aims to engage with active researchers from KG communities, recommendation communities, natural language processing communities, and other communities, and deliver state-of-the-art research insights into the core challenges in KG learning.

Most of the biomedical knowledge the research community has acquired during the past few decades has been deposited in scientific literature as unstructured text. Converting the unstructured text into the structured form will enable novel methodologies and applications for scientific discovery that can fully harness the power of the existing knowledge. To this end, two fundamental questions need to be addressed: named entity recognition (NER) and relation extraction (RE). NER deals with identifying the concepts or entities in texts, such as diseases, genes/proteins, chemical compounds, etc. while RE aims to extract the relations among these entities. Together, the extracted information forms a knowledge graph (KG) where the nodes are entities in the texts and the edges represent their relationships. KGs can link concepts within existing research to allow researchers to find connections that may have been difficult to discover without them. The LitCoin Natural Language Processing (NLP) Challenge was recently organized by NCATS of NIH and NASA to spur innovation by rewarding the most creative and high-impact uses of biomedical text to create KGs. Our team participated in the challenge and ranked first place. We have applied the methods we developed for the LitCoin NLP challenge to all PubMed abstracts and constructed the largest-scale biomedical KG to date. We show that powerful and versatile query functions can be implemented on top of the KG to enable highly specific and accurate knowledge retrieval and inference of causal and indirect relationships. Citation Format: Xin Sui, Yuan Zhang, Feng Pan, Donghu Sun, Menghan Chung, Jinfeng Zhang. Constructing the largest-scale knowledge graph using all PubMed abstracts and its application for highly specific and accurate knowledge retrieval. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5421.

Knowledge graphs have become a vital tool for structuring and accessing cultural heritage data, offering new ways to connect and interpret historical information. As the volume and complexity of heritage data grow, integrating advanced computational methods becomes essential to enhance their accuracy, usability, and accessibility. This paper explores emerging trends in cultural heritage knowledge graphs, focusing on the role of artificial intelligence and machine learning in improving entity recognition, contextualization, and knowledge enrichment. It examines how natural language processing and deep learning techniques can refine data interpretation and automate updates, leading to dynamic, self-sustaining knowledge graphs. Additionally, the study highlights the integration of knowledge graphs with immersive technologies such as augmented reality and virtual reality, which offer interactive ways to engage with heritage content. Finally, it discusses the impact of linked open data initiatives in fostering crossinstitutional collaboration and global accessibility. These advancements collectively redefine how cultural heritage is studied and experienced, making it more interconnected, interactive, and adaptable to new discoveries.

Top-[formula omitted] star queries on knowledge graphs through semantic-aware bounding match scores

Purpose This work aims to normalize the N lp C ontributions scheme (henceforward, N lp C ontribution G raph ) to structure, directly from article sentences, the contributions information in Natural Language Processing (NLP) scholarly articles via a two-stage annotation methodology: 1) pilot stage—to define the scheme (described in prior work); and 2) adjudication stage—to normalize the graphing model (the focus of this paper). Design/methodology/approach We re-annotate, a second time, the contributions-pertinent information across 50 prior-annotated NLP scholarly articles in terms of a data pipeline comprising: contribution-centered sentences, phrases, and triple statements. To this end, specifically, care was taken in the adjudication annotation stage to reduce annotation noise while formulating the guidelines for our proposed novel NLP contributions structuring and graphing scheme. Findings The application of N lp C ontribution G raph on the 50 articles resulted finally in a dataset of 900 contribution-focused sentences, 4,702 contribution-information-centered phrases, and 2,980 surface-structured triples. The intra-annotation agreement between the first and second stages, in terms of F1-score, was 67.92% for sentences, 41.82% for phrases, and 22.31% for triple statements indicating that with increased granularity of the information, the annotation decision variance is greater. Research limitations N lp C ontribution G raph has limited scope for structuring scholarly contributions compared with STEM (Science, Technology, Engineering, and Medicine) scholarly knowledge at large. Further, the annotation scheme in this work is designed by only an intra-annotator consensus—a single annotator first annotated the data to propose the initial scheme, following which, the same annotator reannotated the data to normalize the annotations in an adjudication stage. However, the expected goal of this work is to achieve a standardized retrospective model of capturing NLP contributions from scholarly articles. This would entail a larger initiative of enlisting multiple annotators to accommodate different worldviews into a “single” set of structures and relationships as the final scheme. Given that the initial scheme is first proposed and the complexity of the annotation task in the realistic timeframe, our intra-annotation procedure is well-suited. Nevertheless, the model proposed in this work is presently limited since it does not incorporate multiple annotator worldviews. This is planned as future work to produce a robust model. Practical implications We demonstrate N lp C ontribution G raph data integrated into the Open Research Knowledge Graph (ORKG), a next-generation KG-based digital library with intelligent computations enabled over structured scholarly knowledge, as a viable aid to assist researchers in their day-to-day tasks. Originality/value N lp C ontribution G raph is a novel scheme to annotate research contributions from NLP articles and integrate them in a knowledge graph, which to the best of our knowledge does not exist in the community. Furthermore, our quantitative evaluations over the two-stage annotation tasks offer insights into task difficulty.

In natural language processing (NLP) and computer vision (CV), the successful application of foundation models across diverse tasks has demonstrated their remarkable potential.However, despite the rich structural and textual information embedded in knowledge graphs (KGs), existing research of foundation model for KG has primarily focused on their structural aspects, with most efforts restricted to in-KG tasks (e.g., knowledge graph completion, KGC).This limitation has hindered progress in addressing more challenging out-of-KG tasks.In this paper, we introduce MERRY, a foundation model for general knowledge graph reasoning, and investigate its performance across two task categories: in-KG reasoning tasks (e.g., KGC) and out-of-KG tasks (e.g., KG question answering, KGQA).We not only utilize the structural information, but also the textual information in KGs.Specifically, we propose a multi-perspective Conditional Message Passing (CMP) encoding architecture to bridge the gap between textual and structural modalities, enabling their seamless integration.Additionally, we introduce a dynamic residual fusion module to selectively retain relevant textual information and a flexible edge scoring mechanism to adapt to diverse downstream tasks.Comprehensive evaluations on 28 datasets demonstrate that MERRY outperforms existing baselines in most scenarios, showcasing strong reasoning capabilities within KGs and excellent generalization to out-of-KG tasks such as KGQA.

Textual entailment is a fundamental task in natural language processing. Most approaches for solving this problem use only the textual content present in training data. A few approaches have shown that information from external knowledge sources like knowledge graphs (KGs) can add value, in addition to the textual content, by providing background knowledge that may be critical for a task. However, the proposed models do not fully exploit the information in the usually large and noisy KGs, and it is not clear how it can be effectively encoded to be useful for entailment. We present an approach that complements text-based entailment models with information from KGs by (1) using Personalized PageRank to generate contextual subgraphs with reduced noise and (2) encoding these subgraphs using graph convolutional networks to capture the structural and semantic information in KGs. We evaluate our approach on multiple textual entailment datasets and show that the use of external knowledge helps the model to be robust and improves prediction accuracy. This is particularly evident in the challenging BreakingNLI dataset, where we see an absolute improvement of 5-20% over multiple text-based entailment models.

Reasoning over knowledge graphs (KGs) has received increasing attention recently due to its promising applications in many areas, such as semantic search and recommendation systems. Subsequently, most reasoning models are inherently transductive and ignore uncertainties of KGs, making it difficult to generalize to unseen entities. Moreover, existing approaches usually require each entity in the KG to have sufficient training samples, which leads to the overfitting of the entity having few instances. In fact, long-tail distributions are quite widespread in KGs, and newly emerging entities will tend to have only a few related triples. In this work, we aim at studying knowledge graph reasoning under a challenging setting where only limited training samples are available. Specifically, we propose a Bayesian inductive reasoning method and incorporate meta-learning techniques in few-shot learning to solve data deficiency and uncertainties. We design a Bayesian graph neural network as a meta-learner to achieve Bayesian inference, which can extrapolate meta-knowledge from observed KG to emerging entities. We conduct extensive experiments on two large-scale benchmark datasets, and the results demonstrate considerable performance improvement with the proposed approach over other baselines.

Extracting information from unstructured and semi-structured geoscience literature is a crucial step in conducting geological research. The traditional machine learning extraction paradigm requires a substantial amount of high-quality manually annotated data for model training, which is time-consuming, labor-intensive, and not easily transferable to new fields. Recently, large language models (LLMs) (e.g., ChatGPT, GPT-4, and LLaMA), have shown great performance in various natural language processing (NLP) tasks, such as question answering, machine translation, and text generation. A substantial body of work has demonstrated that LLMs possess strong in-context learning (ICL) and even zero-shot learning capabilities to solve downstream tasks without specifically designed supervised fine-tuning. In this paper, we propose utilizing LLMs for geoscience literature information extraction. Specifically, we design a hierarchical PDF parsing pipeline and an automated knowledge extraction process, which can significantly reduce the need for manual data annotation, assisting geoscientists in literature data mining. For the hierarchical PDF parsing pipeline, firstly, a document layout detection model fine-tuned on geoscience literature is employed for layout detection, obtaining layout detection information for the document. Secondly, based on the document layout information, an optical character content parsing model is used for content parsing, obtaining the text structure and plain text corresponding to the content. Finally, the text structure and plain text are combined and reconstructed to ultimately obtain the parsed structured data. For the automated knowledge extraction process, firstly, the parsed long text is segmented into paragraphs to adapt to the input length limit of LLMs. Subsequently, a few-shot prompting method is employed for structured knowledge extraction, encompassing two tasks: attribute value extraction and triplet extraction. In attribute value extraction, prompts are generated automatically by the LLMs based on the subdomain and attribute names, facilitating the location and extraction of values related to subdomain attribute names in the text. For triplet extraction, the LLMs employ a procedural approach to entity extraction, entity type extraction, and relation extraction, following the knowledge graph structure pattern. Finally, the extracted structured knowledge is stored in the form of knowledge graphs, facilitating further analysis and integration of various types of knowledge from the literature. Our proposed approach turns out to be simple, flexible, and highly effective in geoscience literature information extraction. Demonstrations of information extraction in subdomains such as radiolarian fossils and fluvial facies have yielded satisfactory results. The extraction efficiency has significantly improved, and feedback from domain experts indicates a relatively high level of accuracy in the extraction process. The extracted results can be used to construct a foundational knowledge graph for geoscience literature, supporting the comprehensive construction and efficient application of a geoscience knowledge graph.

The enormous volume of medical data has resulted in the development of sophisticated systems that facilitate information search and enable clinicians in decision-making process. Driven by artificial intelligence, knowledge graphs (KGs) provide a solid structure for organising and evaluating vast volumes of diverse medical data, therefore enabling wiser question development and improved decision-making. This article presents a whole strategy for integrating knowledge graphs with artificial intelligence-based approaches to improve medical information search and decision support systems performance. Graph-based reasoning, natural language processing (NLP), and machine learning all help the proposed approach to enhance semantic comprehension. It achieves this by tying together unorganised and organised medical data sources to provide pertinent analysis. Using predictive analytics, personalised healthcare recommendations, and real-time clinical decision support, the AI-powered knowledge graph architecture helps you It achieves this by continuously shifting the relationships among illnesses, symptoms, therapies, pasts of patients. This approach also ensures that many healthcare systems may cooperate better, which facilitates information search and reduces the diagnostic error count. Including reinforcement learning techniques enhances question results depending on user interaction, therefore enhancing the search process. The results of experiments show that KGs with AI work better than traditional database-driven methods when it comes to getting medical information quickly, correctly, and usefully. The suggested method helps healthcare workers a lot by making it easier for them to get accurate, evidence-based information more quickly. This will eventually lead to better patient results. This study shows that knowledge graphs driven by AI have the ability to completely change how medical information is managed and how decisions are made. This could lead to smarter and more flexible healthcare systems.