Topic-Guided Knowledge Graph Construction for Argument Mining

Analysis of Consumer buying patterns is crucial for vendors to understand the necessities and favorites of their audience. With the increasing availability of digital data, collecting large amounts of consumer feedback and opinions has become easier. However, analyzing such data is challenging due to the presence of unstructured text data. Latent Dirichlet Allocation (LDA) is a prevalent topic modeling procedure that can be used to analyze unstructured text data. LDA allows for the identification of latent topics that are present in the data, enabling businesses to gain insights into consumer behavior. By applying LDA to consumer feedback data, businesses can identify the key topics that are most important to their customers. Additionally, LDA can be used to track changes in consumer preferences over time, allowing businesses to stay up-to-date with evolving market trends. Overall, LDA is a powerful tool for analyzing consumer behavior and can provide valuable insights for businesses looking to improve their products and services. In this work, LDA is employed to analyze consumer tweets using topic modeling. LDA has successfully generated topic and

With the development of technology, Earth Science has entered a new era. Continuous research has generated a large amount of Earth Science data, including a significant amount of semi-structured and unstructured data, which contain information about locations, geographical concepts, geological characteristics of mineral deposits, and relationships. Efficient management of these Earth Science data is crucial for the development of digital earth systems, rational planning of resource industries, and resource security. By representing entities, relationships, and attributes through graph structures, knowledge graphs capture and present concepts and facts about the real world, facilitating efficient data management. However, due to the highly specialized and complex nature of Earth Science data and disciplinary differences, the methods used to construct general-purpose knowledge graphs cannot be directly applied to building knowledge graphs in the field of geological science. Therefore, this paper summarizes a “pipeline” approach to constructing an Earth Science knowledge graph in order to clarify the complete construction process and reduce barriers between data and technology. This approach divides the construction of the Earth Science knowledge graph into two parts and designs functional modules under each part to specify the construction process of the knowledge graph. In addition to proposing this approach, a knowledge graph of iron ore deposits is automatically constructed by integrating geographic and geological data related to iron ore deposits using deep learning techniques. The systematic approach presented in this paper reduces the threshold for constructing geological science knowledge graphs, provides methodological support for specific disciplines or research objects in Earth Science, and also lays the foundation for the construction of large-scale Earth Science knowledge graphs that combine crowdsourcing and expert decision-making, as well as the development of intelligent question-answering systems and intelligent decision-making systems covering the entire field of Earth Science.

Topic modelling is an important technique for extracting meaningful insights from large volumes of unstructured text data. The paper presents a federated technique for topic modelling that is based on a novel approach of the Latent Dirichlet Allocation (LDA) method for topic model generation. The proposed approach enables a topic model to be developed in a distributed environment using data continually generated from multiple sources without the need for sharing actual data. The first iteration of the topic modelling uses unsupervised LDA at each device generating the data. The results of each device are aggregated at a central server to generate a set of seed words that are used for guided LDA by the subsequent iterations of topic modelling. The proposed work, Federated LDA (F-LDA) has been evaluated using two datasets: a text dataset of dialogues between patients and doctors based on factual conversations and another comprising tweets related to depression. Comparing the performance of F-LDA with that of a centralized LDA, it was observed that F-LDA results in improved coherence score as well as diversity score in comparison to centralized LDA. This indicates that F-LDA achieves better interpretable topics covering a wide range of themes without redundancy.

This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data. The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines. The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery. Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.

Construct a knowledge graph is time-consuming and the knowledge graph in the scientific domain requires extremely high labor costs due to it requires high prior knowledge to extract knowledge from resources. To build a scientific research knowledge graph, the most of input are papers, patent, the description of their project and some national program (such as National High Technology Research and Development Program of China, Major State Basic Research Development Program of China, General Program, Key Program and Major Program) which all of them are unstructured data, that make human participation are mostly necessary to measure the quality. In this paper, we design and proposed a framework using active learning; this framework can be used to extract entity and relation from unstructured science and technology research data. This framework combines the human and machine learning approach together, which is active learning, to help user extract entity from those unstructured data with less time cost. By using those data to construct a CKG as annotation label, it further implements active learning tools and helps the expert to rapidly annotate the data with high accuracy. Those knowledge graph constructed by this framework can be used to finding similar research area, finding similar researchers, finding popular research areas and so on.

Although there are a small number of work to conduct patent research by building knowledge graph, but without constructing patent knowledge graph using patent documents and combining latest natural language processing methods to mine hidden rich semantic relationships in existing patents and predict new possible patents. In this paper, we propose a new patent vacancy prediction approach named PatentMiner to mine rich semantic knowledge and predict new potential patents based on knowledge graph (KG) and graph attention mechanism. Firstly, patent knowledge graph over time (e.g. year) is constructed by carrying out named entity recognition and relation extrac-tion from patent documents. Secondly, Common Neighbor Method (CNM), Graph Attention Networks (GAT) and Context-enhanced Graph Attention Networks (CGAT) are proposed to perform link prediction in the constructed knowledge graph to dig out the potential triples. Finally, patents are defined on the knowledge graph by means of co-occurrence relationship, that is, each patent is represented as a fully connected subgraph containing all its entities and co-occurrence relationships of the patent in the knowledge graph; Furthermore, we propose a new patent prediction task which predicts a fully connected subgraph with newly added prediction links as a new pa-tent. The experimental results demonstrate that our proposed patent predic-tion approach can correctly predict new patents and Context-enhanced Graph Attention Networks is much better than the baseline. Meanwhile, our proposed patent vacancy prediction task still has significant room to im-prove.

Given that unstructured data is increasing exponentially everyday, extracting and understanding the information, themes, and relationships from large collections of documents is increasingly important to researchers in many disciplines including biomedicine. Latent Dirichlet Allocation (LDA) is an unsupervised topic modeling technique based on the "bag-of-words" assumption that has been applied extensively to unveil hidden semantic themes within large sets of textual documents. Recently, it was extended using the "bag-of-n-grams" paradigm to account for word order. In this paper, we present an alternative phrase based LDA model to move from a bag of words or n-grams paradigm to a "bag-of-key-phrases" setting by applying a key phrase extraction technique, the C-value method, to further explore latent themes. We evaluate our approach by using a phrase intrusion user study and demonstrate that our model can help LDA generate better and more interpretable topics than those generated using the bag-of-n-grams approach. Given topic models essentially are statistical tools, an important problem in topic modeling is that of visualizing and interacting with the models to understand and extract new information from a collection. To evaluate our phrase based modeling approach in this context, we incorporate it in an open source interactive topic browser. Qualitative evaluations of this browser with biomedical experts demonstrate that our approach can aid biomedical researchers gain better and faster understanding of their document collections.

This paper examines the performance of topic modelling techniques on a newly compiled dataset of Malaysian business news, aiming to address the challenge of extracting meaningful insights from unstructured data. We compare traditional methods, such as Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF), with emerging methods or models, including <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> op2Vec and BERTopic. Each model is trained or fine-tuned on five yearly datasets (2019–2023) to account for temporal variations. Our goal is to identify the most effective technique for topic discovery in this domain. The dataset includes 35,667 articles collected through web scraping from New Straits Times (Malaysia) and other Malaysian news portals. The results show that NMF consistently achieves the highest coherence scores, ranging between 0.65 until 0.80, which indicate strong topic quality and interpretability. On the other hand, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> op2Vec performs the worst, while LDA and BERTopic produce inconsistent results. These findings provide useful guidance, particularly the strengths and limitations of each technique, for researchers and NLP practitioners applying topic modelling to similar textual corpora.

Finding similar entities among knowledge graphs is an essential research problem for knowledge integration and knowledge graph connection. This paper aims at finding semantically similar entities between two knowledge graphs. This can help end users and search agents more effectively and easily access pertinent information across knowledge graphs. Given a query entity in one knowledge graph, the proposed approach tries to find the most similar entity in another knowledge graph. The main idea is to leverage graph embedding, clustering, regression and sentence embedding. In this approach, RDF2Vec has been employed to generate vector representations of all entities of the second knowledge graph and then the vectors have been clustered based on cosine similarity using K medoids algorithm. Then, an artificial neural network with multilayer perception topology has been used as a regression model to predict the corresponding vector in the second knowledge graph for a given vector from the first knowledge graph. After determining the cluster of the predicated vector, the entities of the detected cluster are ranked through sentence-BERT method and finally the entity with the highest rank is chosen as the most similar one. To evaluate the proposed approach, experiments have been conducted on real-world knowledge graphs. The experimental results demonstrate the effectiveness of the proposed approach.

We present an online multilingual system for event detection and comprehension from media feeds. The system retrieves information from news sites, aggregates them into events (event detection), and summarizes them by extracting semantic labels of its most relevant entities (event representation) in order to answer the journalism Ws: who, what, when and where. The generated events populate VLX-Stories -an event ontology- transforming unstructured text data to a structured knowledge base representation. Our system exploits an external entity Knowledge Graph (VKG) to help populate VLX-Stories. At the same time, this external knowledge graph can also be extended with a Dynamic Entity Linking (DEL) module, which detects emerging entities (EE) on unstructured data. The system is currently deployed in production and used by media producers in the editorial process, providing real-time access to breaking news. Each month, VLX-Stories detects over 9000 events from over 4000 news feeds from seven different countries and in three different languages. At the same time, it detects over 1300 EE per month, which populate VKG.

Knowledge enhancement-based large-scale natural language training model is an advanced language model that combines deep learning and knowledge enhancement. By learning from massive unlabeled data and combining with external knowledge such as knowledge graph, it breaks through the limitations of traditional models in interpretability and reasoning ability. Introducing knowledge into data-driven artificial intelligence model is an important way to realize human-machine hybrid intelligence. However, since most pre-trained models are trained on large-scale unstructured corpus data, the defects in certainty and explainability can be remedied to some extent by introducing external knowledge. To solve the above problems, we present a knowledge-enhanced large-scale natural language training model that integrates deep learning with external knowledge sources (e.g., knowledge graphs) to improve interpretability and reasoning ability. This approach addresses the limitations of traditional models trained on unstructured data by incorporating external knowledge to enhance certainty and explainability. We propose a new knowledge enhancement method and demonstrate its effectiveness through a long text representation model. This model processes structured, knowledge-rich long texts by extracting and integrating knowledge and semantic information at the sentence and document levels. It then fuses these representations to generate an enhanced long text representation. Experiments on legal case matching tasks show that our model significantly outperforms existing methods, highlighting its innovation and practical value.

A domain expert can process heterogeneous data to make meaningful interpretations or predictions from the data. For example, by looking at research papers and patent records, an expert can determine the maturity of an emerging technology and predict the geographic location(s) and time (e.g., in a certain year) where and when the technology will be a success. However, this is an expert- and manual-intensive task. This paper presents an end-to-end system that integrates heterogeneous data sources into a knowledge graph in the RDF (Resource Description Framework) format using an ontology. Then the user can easily query the knowledge graph to prepare the required data for different types of predictive analysis tools. We show a case study of predicting the (geographic) center(s) of fuel cell technologies using data collected from public sources to demonstrate the feasibility of our system. The system extracts, cleanses, and augments data from public sources including research papers and patent records. Next, the system uses an ontology-based data integration method to generate knowledge graphs in the RDF format to enable users to switch quickly between machine learning models for predictive analytic tasks. We tested the system using the Support Vector Machine and Multiple Hidden Markov Models and achieved 66.7% and 83.3% accuracy on the city and year levels of spatial and temporal resolutions, respectively.

Knowledge Graph is a technology for more effective knowledge sharing and makes the web smarter. By building Knowledge Graphs, information resources can be more easily calculated, understood and evaluated, effectively and accurately contributing to further development in various fields. Research on Knowledge Graph fusion aims to generate new knowledge by using models to fuse knowledge graphs from various domains, in which entity alignment is one of the fundamental researches in this field. For state-of-the-art studies, the hotspot of entity alignment primarily focus on cross-lingual and graph learning approaches. In this paper, we first introduce the main problems faced in the field of entity alignment research. Then the acquisition and usage of relevant models of entity alignment for cross-lingual and graph learning approaches are introduced. Next, the most frequently used evaluation indexes of entity alignment are presented. Finally, the challenges and future research directions of entity alignment in these two fields are further discussed.

In an era dominated by an unprecedented deluge of textual information, the need for effective methods to make sense of large datasets is more pressing than ever. This article takes a pragmatic approach to unraveling the intricacies of topic modeling, with a specific focus on the widely used Latent Dirichlet Allocation (LDA) algorithm. The initial segment of the article lays the groundwork by exploring the practical relevance of topic modeling in real-world scenarios. It addresses the everyday challenges faced by researchers and professionals dealing with vast amounts of unstructured text, emphasizing the potential of topic modeling to distill meaningful insights from seemingly chaotic data. Moving beyond theoretical abstraction, the article then delves into the mechanics of Latent Dirichlet Allocation. Developed in 2003 by Blei, Ng, and Jordan, LDA provides a probabilistic framework to identify latent topics within documents. The article takes a step-by-step approach to demystify LDA, offering a practical understanding of its components and the Bayesian principles governing its operation. A significant portion of the article is dedicated to the practical implementation of LDA. It provides insights into preprocessing steps, parameter tuning, and model evaluation, offering readers a hands-on guide to applying LDA in their own projects. Real-world examples and case studies showcase how LDA can be a valuable tool for tasks such as document clustering, topic summarization, and sentiment analysis. However, the journey through LDA is not without challenges, and the article candidly addresses these hurdles. Topics such as determining the optimal number of topics, the sensitivity of results to parameter settings, and the interpretability of outcomes are discussed. This realistic appraisal adds depth to the article, helping readers navigate the nuances and potential pitfalls of employing LDA in practice. Beyond the technical intricacies, the article explores the broad spectrum of applications where LDA has proven its efficacy. From text mining and information retrieval to social network analysis and healthcare informatics, LDA has left an indelible mark on diverse domains. Through practical examples, the article illustrates how LDA can be adapted to different contexts, showcasing its versatility as a tool for uncovering latent patterns. Keywords: Topic Modeling, Latent Dirichlet Allocation, Text Mining, Natural Language Processing, Document Clustering, Bayesian Inference.

A massive of text that is generated every minute is increasing dramatically. Therefore, it is more and more important to find an effective model to automatic classify the amount of text. Topic models is the most powerful techniques in text classification. There are many research results in the field of topic model have been published in scholarly journals. The Latent Dirichlet Allocation (LDA) is one of the most popular topic models in text classification. Researchers have proposed many topic evolution models based on LDA to solve some specific problems in applications of text classification. And some joint models which based on topic models combined other algorithms have been studied to enhance the performance of text classification. In this paper, we investigated three categories topic models for text classification and briefly introduced their advantages and disadvantages in the applications of text mining. Also, we introduce the generated process of documents and illustrate the graphical model for each topic models.