Semantic and Engineering-Based Embedding for Classification List Development

The Beyond 2022 project aims to create a virtual archive by digitally reconstructing and digitizing historical records lost in a catastrophic fire which consumed items in the Public Record Office of Ireland in 1922. The project is developing a knowledge graph (KG) to facilitate information retrieval and discovery over the reconstructed items. The project decided to adopt Semantic Web technologies to support its distributed KG and reasoning. In this article, we present our approach to KG generation and management. We elaborate on how we help historians contribute to the KG (via a suite of spreadsheets) and its ontology. We furthermore demonstrate how we use named graphs to store different versions of factoids and their provenance information and how these are serviced in two different endpoints. Modeling data in this manner allows us to acknowledge that history is, to some extent, subjective and different perspectives can exist in parallel. The construction of the KG is driven by competency questions elicited from subject matter experts within the consortium. We avail of CIDOC-CRM as our KG’s foundation, though we needed to extend this ontology with various qualifiers (types) and relations to support the competency questions. We illustrate how one can explore the KG to gain insights and answer questions. We conclude that CIDOC-CRM provides an adequate, albeit complex, foundation for the KG and that named graphs and Linked Data principles are a suitable mechanism to manage sets of factoids and their provenance.

Compressors in petrochemical plants are often crucial to process operations, and when a failure occurs, the outcome can be catastrophic. Many researches have been attempting to detect failure modes as early as possible to plan upfront repair and conceivably reduce maintenance time. A reciprocating compressor was selected as the target of this study, and a few years of historical records of maintenance parameters and maintenance work orders were gathered for analysis. The time history was divided into 13 events, and each event started with a normal operation and ended with a repair work order. Time‐domain features and wavelet decomposition features of the parameters were extracted, and the patterns stored within each event were identified using the artificial neural network and support vector machine. Moreover, a set of reasoning algorithms were developed to detect anomalies, and responsible failure modes were identified. For a specific type of compressor, the vibration signal was found to be related to most of the anomalies and thus used for evaluation. Results showed a >90% detection rate for failure mode diagnosis based on historical test data.

Multilingual knowledge graph (KG) embeddings provide latent semantic representations of entities and structured knowledge with cross-lingual inferences, which benefit various knowledge-driven cross-lingual NLP tasks. However, precisely learning such cross-lingual inferences is usually hindered by the low coverage of entity alignment in many KGs. Since many multilingual KGs also provide literal descriptions of entities, in this paper, we introduce an embedding-based approach which leverages a weakly aligned multilingual KG for semi-supervised cross-lingual learning using entity descriptions. Our approach performs co-training of two embedding models, i.e. a multilingual KG embedding model and a multilingual literal description embedding model. The models are trained on a large Wikipedia-based trilingual dataset where most entity alignment is unknown to training. Experimental results show that the performance of the proposed approach on the entity alignment task improves at each iteration of co-training, and eventually reaches a stage at which it significantly surpasses previous approaches. We also show that our approach has promising abilities for zero-shot entity alignment, and cross-lingual KG completion.

Natural Language Processing (NLP) has seen a surge in recent years, especially with the introduction of transformer architectures, relying on the now famous self-attention mechanism. Especially, with the rise of Large Language Models (LLM), propelled by the appearance of ChatGPT in 2022, a new hope of extracting relevant information from text has emerged. In the meantime, natural language data have not often been used in risk, resilience, and reliability tasks. However, text data containing reliability-related information, that can be used to monitor health information regarding complex systems, are available in several and diverse shapes. Indeed, text data can either contain theoretical expert knowledge (technical reports, documentation, Failure Modes and Effects Analysis (FMEA)), or in-practice expert knowledge (incident reports, maintenance work orders), or in-practice non-expert knowledge (customer feedback, news articles). Critical infrastructures, such as nuclear powerplants, railway networks, or electrical power grids, are complex systems for which any failure would induce severe consequences affecting many people. Such systems have the advantage of serving many users, thus having many possible text sources from which technical information and past incident data can be mined for anticipating future failures and generating responses to catastrophic scenarios. The goal of this work is to develop methods and apply state-of-the-art NLP techniques to text data relating to critical infrastructures and failures, to (1) mine information from unstructured language data, and (2) structure the extracted information. Preliminary experiments were conducted on customer review data and incident reports, and show promising performance for failure detection from text data with transformers, as well as incident-related information extraction using LLMs.

Extracting information from unstructured and semi-structured geoscience literature is a crucial step in conducting geological research. The traditional machine learning extraction paradigm requires a substantial amount of high-quality manually annotated data for model training, which is time-consuming, labor-intensive, and not easily transferable to new fields. Recently, large language models (LLMs) (e.g., ChatGPT, GPT-4, and LLaMA), have shown great performance in various natural language processing (NLP) tasks, such as question answering, machine translation, and text generation. A substantial body of work has demonstrated that LLMs possess strong in-context learning (ICL) and even zero-shot learning capabilities to solve downstream tasks without specifically designed supervised fine-tuning. In this paper, we propose utilizing LLMs for geoscience literature information extraction. Specifically, we design a hierarchical PDF parsing pipeline and an automated knowledge extraction process, which can significantly reduce the need for manual data annotation, assisting geoscientists in literature data mining. For the hierarchical PDF parsing pipeline, firstly, a document layout detection model fine-tuned on geoscience literature is employed for layout detection, obtaining layout detection information for the document. Secondly, based on the document layout information, an optical character content parsing model is used for content parsing, obtaining the text structure and plain text corresponding to the content. Finally, the text structure and plain text are combined and reconstructed to ultimately obtain the parsed structured data. For the automated knowledge extraction process, firstly, the parsed long text is segmented into paragraphs to adapt to the input length limit of LLMs. Subsequently, a few-shot prompting method is employed for structured knowledge extraction, encompassing two tasks: attribute value extraction and triplet extraction. In attribute value extraction, prompts are generated automatically by the LLMs based on the subdomain and attribute names, facilitating the location and extraction of values related to subdomain attribute names in the text. For triplet extraction, the LLMs employ a procedural approach to entity extraction, entity type extraction, and relation extraction, following the knowledge graph structure pattern. Finally, the extracted structured knowledge is stored in the form of knowledge graphs, facilitating further analysis and integration of various types of knowledge from the literature. Our proposed approach turns out to be simple, flexible, and highly effective in geoscience literature information extraction. Demonstrations of information extraction in subdomains such as radiolarian fossils and fluvial facies have yielded satisfactory results. The extraction efficiency has significantly improved, and feedback from domain experts indicates a relatively high level of accuracy in the extraction process. The extracted results can be used to construct a foundational knowledge graph for geoscience literature, supporting the comprehensive construction and efficient application of a geoscience knowledge graph.

Collaborative filtering and neural recommendation models have advanced substantially over the past decade, yet they frequently struggle to capture the fine-grained structure of user preferences and the rich, multi-relational dependencies among items. Knowledge graph-based methods provide a principled means of representing such structured information; however, systematic mechanisms for combining structured reasoning with heterogeneous recommendation signals remain relatively underexplored. In this work, we introduce KGERA (Knowledge Graph Enhanced Reasoning Architecture), a recommendation framework that augments a multi-relational knowledge graph with an explicit reasoning module to enhance ranking quality through test-time inference. We construct a unified knowledge graph encoding diverse item relationships, including genre co-membership, creator-level associations, and data-driven similarity links, and apply structured reasoning on this graph to compute contextualized recommendation scores. Unlike approaches that require expensive large language model inference, KGERA performs lightweight, interpretable reasoning exclusively at inference time using embedding-based computations, allowing the system to adapt dynamically to evolving user preference signals without retraining. The reasoning component is coupled with nine complementary recommendation strategies through a learned ensemble that integrates collaborative filtering (ItemKNN), graph neural networks (LightGCN), self-supervised graph learning (SGL, SimGCL), neural collaborative filtering (NCF), knowledge graph embeddings (TransE), content-based filtering (TextProxy), and popularity-based signals (PopRec) into a single predictive model. Experiments on MovieLens-1M demonstrate that KGERA significantly outperforms strong baselines, including recent self-supervised methods from SIGIR 2021-2022, achieving improvements of 52.83% in NDCG@10, 50.24% in Recall@10, and 54.62% in MRR@10 over the best baseline. Comprehensive validation through leave-one-out ablation studies, threshold sensitivity analysis, 5-fold cross-validation (CV = 4.84%), and stratification experiments across user activity levels (77.5%–85.6% improvement for low/medium activity users), item popularity tiers (114.2% improvement for medium-popularity items), and content genres demonstrates the robustness and generalizability of our approach. All improvements are statistically significant with $$p < 0.001$$ under paired t-tests. Collectively, these results indicate that explicit test-time reasoning over structured knowledge provides a computationally efficient and interpretable direction for advancing the design of next-generation recommender systems.

In this study, the application of the RCM methodology (which is a maintenance management model) is shown in a factory of tongue and groove floors, in the area of Advanced Transformation; in this section is where the final finish of all floor surfaces is given. The application of the RCM methodology allowed the analysis of the primary and secondary functions, the identification of the failure modes, the effects of the failure and the hierarchy of risks of the most critical equipment of the system or problem area. The AMEF was carried out to determine the maintenance plan and in turn, with the NPR, the most critical failure modes were selected. Subsequently, these failure modes were analyzed in the information sheet and in the decision sheet with the objective of determining the activities and frequency of maintenance tasks. As a consequence of this analysis, documents were developed that help with the proper maintenance management (Resume, technical file, novelty report, maintenance service request, work order and maintenance report). Also, instructions for maintenance activities of the machines were developed. Finally, some results of the simulation of the methodology are shown, such as: the reduction of the average times to repair (TMPR) and the increase of the maintenance tasks concluded within the established term. These results will be reflected in the increased availability of plant equipment.

Failure mode and effect analysis for dairy product manufacturing: Practical safety improvement action plan with cases from Turkey

A framework for differentially-private knowledge graph embeddings

Vital Signs: Deficiencies in Environmental Control Identified in Outbreaks of Legionnaires’ Disease—North America, 2000–2014

In recent years, interest in knowledge graphs (KGs) has increased exponentially in both the scientific and industrial communities. Integration of various KGs is a pressing problem and is used, for example, to develop complex digital twins of industrial systems. Knowledge graph integration is also necessary when combining KGs extracted from natural language texts using large language models. One component of solving the KG integration problem is entity alignment (EA), which attempts to identify entities in different KGs that describe the same real-world object. In reality, many entities in real KGs have no equivalents in other KGs. In particular, each knowledge graph fragment extracted from a single publication may have its own structure of entity names and identifiers, which significantly complicates the task of identifying entities. This paper describes experiments on entity alignment in the presence of unmatchable entities using a Russian-English dataset as an example.

In the previous chapter we saw how knowledge graph embedding algorithms can capture structured knowledge about concepts and relations in a graph as embeddings in a vector space, which then can be used in downstream tasks. However, this type of approaches can only capture the knowledge that is explicitly represented in the graph, hence lacking in recall and domain coverage. In this chapter, we focus on algorithms that address this limitation through the combination of information from both unstructured text corpora and structured knowledge graphs. The first approach is Vecsigrafo, which produces corpus-based word, lemma, and concept embeddings from large disambiguated corpora. Vecsigrafo jointly learns word, lemma, and concepts embeddings, bringing together textual and symbolic knowledge representations in a single, unified formalism for use in neural natural language processing architectures. The second and more recent approach is called Transigrafo, which adopts recent Transformer-based language models to derive concept-level contextual embeddings, providing state-of-the-art performance in word-sense disambiguation with reduced complexity.

As a result of limited resources and economic development acting as dual bottleneck constraints, optimizing industrial layout structures has been a general trend in heavy industry. The visual supervision of heat source factories based on integrated multidimensional information is important for optimizing an industrial layout. Based on Visible Infrared Imaging Radiometer Suite (VIIRS) I‐band 375‐m active fire product (VNP14IMG) data, point of interest data, enterprise attribute information and other data, combined with clustering regression, knowledge graphs (KGs), 3D geographic information systems, and other technical methods, the temporal and spatial variations in China's heat source industries are macroscopically analyzed, and a visual supervision platform for heat source industries with functions such as visualization, time‐series analysis, and knowledge discovery is established. The results show that: (1) overall, heat source factories exhibit a spatial pattern of dense in the east and sparse in the west, and the number of industrial heat source objects and the number of industrial fire hotspots decreased from 2013 to 2021, with rates of decline of 22.0 and 27.3%, respectively; (2) the enterprise KG, which contains basic enterprise information, dynamic enterprise risk information and enterprise equity structure information, can provide users with accurate and reliable enterprise knowledge; and (3) the remote sensing monitoring information platform for heat source factories performs well in terms of the discovery and management of heat source factories at large scale. In general, the platform constructed in this study can support the rapid monitoring and positioning of industrial heat sources over large areas to improve supervision in terms of finding problems and preventing risks and to provide a necessary reference for optimizing industrial spatial patterns and environmental governance.

Mobile mining equipment often operates in harsh environments characterised by remote locations and highly variable rock and operating conditions. This research explores the hypothesis that the failure behaviour of mining equipment is influenced by the physical properties of the ore and waste. We describe a method of examining this relationship via data mining on maintenance records and apply it to the hydraulic cylinders of two classes of earthmoving mobile equipment. Failure data for the analysis are drawn from maintenance work orders from 14 sites mining for haematite iron, nickel sulphide, and coking and thermal coal. The results show that the distributions of the estimated life parameters for hydraulic cylinders on earthmoving equipment are distinctly different for haematite iron, coal, and nickel sulphide sites. Analysis of the relationship between selected physical properties identified the influence of rock impact hardness number, abrasion index, and absolute hardness of the ore as significant factors for these hydraulic cylinders. Their effects are significant when parameters are considered in combination, for example, rock impact hardness number and abrasion index, and vary according to the cylinder type and asset class. The engineering implications of these results are considered with respect to known failure modes of the cylinders.

Causal feature selection using a knowledge graph combining structured knowledge from the biomedical literature and ontologies: A use case studying depression as a risk factor for Alzheimer’s disease