Information Extraction from Invoices

Extracting information from unstructured and semi-structured geoscience literature is a crucial step in conducting geological research. The traditional machine learning extraction paradigm requires a substantial amount of high-quality manually annotated data for model training, which is time-consuming, labor-intensive, and not easily transferable to new fields. Recently, large language models (LLMs) (e.g., ChatGPT, GPT-4, and LLaMA), have shown great performance in various natural language processing (NLP) tasks, such as question answering, machine translation, and text generation. A substantial body of work has demonstrated that LLMs possess strong in-context learning (ICL) and even zero-shot learning capabilities to solve downstream tasks without specifically designed supervised fine-tuning. In this paper, we propose utilizing LLMs for geoscience literature information extraction. Specifically, we design a hierarchical PDF parsing pipeline and an automated knowledge extraction process, which can significantly reduce the need for manual data annotation, assisting geoscientists in literature data mining. For the hierarchical PDF parsing pipeline, firstly, a document layout detection model fine-tuned on geoscience literature is employed for layout detection, obtaining layout detection information for the document. Secondly, based on the document layout information, an optical character content parsing model is used for content parsing, obtaining the text structure and plain text corresponding to the content. Finally, the text structure and plain text are combined and reconstructed to ultimately obtain the parsed structured data. For the automated knowledge extraction process, firstly, the parsed long text is segmented into paragraphs to adapt to the input length limit of LLMs. Subsequently, a few-shot prompting method is employed for structured knowledge extraction, encompassing two tasks: attribute value extraction and triplet extraction. In attribute value extraction, prompts are generated automatically by the LLMs based on the subdomain and attribute names, facilitating the location and extraction of values related to subdomain attribute names in the text. For triplet extraction, the LLMs employ a procedural approach to entity extraction, entity type extraction, and relation extraction, following the knowledge graph structure pattern. Finally, the extracted structured knowledge is stored in the form of knowledge graphs, facilitating further analysis and integration of various types of knowledge from the literature. Our proposed approach turns out to be simple, flexible, and highly effective in geoscience literature information extraction. Demonstrations of information extraction in subdomains such as radiolarian fossils and fluvial facies have yielded satisfactory results. The extraction efficiency has significantly improved, and feedback from domain experts indicates a relatively high level of accuracy in the extraction process. The extracted results can be used to construct a foundational knowledge graph for geoscience literature, supporting the comprehensive construction and efficient application of a geoscience knowledge graph.

Study question Can a machine learning (ML) model, developed using modern neural network architecture produce comparable annotation data; utilisable for algorithmic outcome prediction, to manual time-lapse annotations? Summary answer The model automatically annotated unseen embryos with comparable results to manual methods, generating morphokinetic data to enable comparably predictive outputs from an embryo selection algorithm. What is known already The application of artificial intelligence across healthcare industries, including fertility, is increasing. Several ML models are available that seek to generate or analyse embryo images and morphokinetic data, and to determine embryo viability potential. Along with photographic images, the use of time-lapse in IVF laboratories has amassed numeric data, resulting predominantly from annotated manual assessment of images over time. Embryo annotation practice is variable in quality, can be subjective and is time-consuming; commonly taking several minutes per embryo. The development of rapid, accurate automatic annotation would represent a significant time-saving as well as an increase in reproducibility and accuracy. Study design, size, duration Multicentre quality assured annotation data from 63,383 time-lapse monitored embryos (EmbryoScope®), comprising over 400 million individual images, were used to train a ML model to automatically generate morphokinetic annotations. Data was derived from 8 UK clinics within a cohesive group between 2012-2021. Accuracy was assessed using 900 unseen embryos (with live birth outcome) by comparing the output of an established in-house, prospectively validated embryo selection model when the input was either ML-automated, or manual annotations. Participants/materials, setting, methods Multi-focal plane images were processed on the Azure cloud (Microsoft) and resampled to 300x300 pixels. A Laplacian-based focal stacking algorithm merged frames into a single image. The model consisted of an EfficientNetB4 Convolutional Neural Network classifier to extract features and classify the stage of embryo images. A Temporal Convolutional Network interpreted a time-series of image features; producing annotations from pronuclear fading through to blastocyst. Soft localisation loss function used QA data to integrate annotation subjectivities. Main results and the role of chance The ML model rapidly and automatically generated annotations. Efficacy and comparability of the ML model to automate reliable, utilisable annotations was demonstrated by comparison with manual annotation data and the ML model’s ability to auto-generate annotations which could be used to predict live birth by providing annotation data to an established, validated in house embryo selection model. Live birth-predictive capability was measured, and benchmarked against manual annotation, using the area under the receiver operating characteristic curve (AUC). When tested on time-lapse images, collected from pronuclear fading to full blastulation, representing 900 previously unseen, transferred blastocysts where live birth outcomes were blinded, the in-house developed auto-annotation ML model resulted in an AUC of 0.686 compared with 0.661 for manual annotations, for live birth prediction. Auto annotation using the developed model took only milliseconds to complete per embryo. The developed auto-annotation model, built and tested on large data, is considered suitable for productionisation with the aim of being validated and integrated into an application to support IVF laboratory practice. Limitations, reasons for caution Whilst this model was trained to recognise key morphokinetic events, there are other morphokinetic variables that may be useful in the prediction of live birth and further improve embryo selection, or deselection, ability. Akin to manual interpretation, some embryos may fail to be annotated or need second opinion. Wider implications of the findings There is increasing evidence supporting the application of ML to utilise big data from time-lapse imaging and fertility care generally. Whilst promising benefits to IVF clinics and patients, responsible use of data is required alongside large high-quality datasets, and rigorous validation, to ensure safe and robust applications. Trial registration number N/A

Information extraction (IE) systems are costly to build because they require development texts, parsing tools, and specialized dictionaries for each application domain and each natural language that needs to be processed. We present a novel method for rapidly creating IE systems for new languages by exploiting existing IE systems via cross-language projection. Given an IE system for a source language (e.g., English), we can transfer its annotations to corresponding texts in a target language (e.g., French) and learn information extraction rules for the new language automatically. In this paper, we explore several ways of realizing both the transfer and learning processes using off-the-shelf machine translation systems, induced word alignment, attribute projection, and transformation-based learning. We present a variety of experiments that show how an English IE system for a plane crash domain can be leveraged to automatically create a French IE system for the same domain.

Information Extraction (IE) systems assist analysts to assimilate information from electronic documents. This paper focuses on IE tasks designed to support information discovery applications. Since information discovery implies examining large volumes of heterogeneous documents for situations that cannot be anticipated a priori, they require IE systems to have breadth as well as depth. This implies the need for a domain-independent IE system that can easily be customized for specific domains: end users must be given tools to customize the system on their own. It also implies the need for defining new intermediate level IE tasks that are richer than the subject-verb-object (SVO) triples produced by shallow systems, yet not as complex as the domain-specific scenarios defined by the Message Understanding Conference (MUC). This paper describes InfoXtract, a robust, scalable, intermediate-level IE engine that can be ported to various domains. It describes new IE tasks such as synthesis of entity profiles, and extraction of concept-based general events which represent realistic near-term goals focused on deriving useful, actionable information. Entity profiles consolidate information about a person/organization/location etc. within a document and across documents into a single template; this takes into account aliases and anaphoric references as well as key relationships and events pertaining to that entity. Concept-based events attempt to normalize information such as time expressions (e.g., yesterday) as well as ambiguous location references (e.g., Buffalo). These new tasks facilitate the correlation of output from an IE engine with structured data to enable text mining. InfoXtract's hybrid architecture comprised of grammatical processing and machine learning is described in detail. Benchmarking results for the core engine and applications utilizing the engine are presented.

Although Large language Model (LLM)-powered information extraction (IE) systems have shown impressive capabilities, current fine-tuning paradigms face two major limitations: high training costs and difficulties in aligning with LLM preferences. To address these issues, we propose a novel universal IE paradigm—the Self-Correcting Iterative Refinement (SCIR) framework—along with a Multi-task Bilingual (Chinese-English) Self-Correcting (MBSC) dataset containing over 100,000 entries. The SCIR framework achieves plug-and-play compatibility with existing LLMs and IE systems through its Dual-Path Self-Correcting module and feedback-driven optimization, thereby significantly reducing training costs. Concurrently, the MBSC dataset tackles the challenge of preference alignment by indirectly distilling GPT-4's capabilities into IE result detection models. Experimental results demonstrate that SCIR outperforms state-of-the-art IE methods across three key tasks— named entity recognition, relation extraction, and event extraction—achieving a 5.27 percent average improvement in span-based Micro-F1 while reducing training costs by 87 percent compared to baseline approaches. These advancements not only enhance the flexibility and accuracy of IE systems but also pave the way for lightweight and efficient IE paradigms.

Information Extraction (IE) is the key technology enabling analytics over unstructured and semi-structured data. Not surprisingly, it is becoming a critical building block for a wide range of emerging applications. To satisfy the rising demands for information extraction in real-world applications, it is crucial to lower the barrier to entry for IE development and enable users with general computer science background to develop higher quality extractors. In this demonstration 1 , we present VINERy, an intuitive yet expressive visual IDE for information extraction. We show how it supports the full cycle of IE development without requiring a single line of code and enables a wide range of users to develop high quality IE extractors with minimal efforts. The extractors visually built in VINERY are automatically translated into semantically equivalent extractors in a state-of-the-art declarative language for IE. We also demonstrate how the auto-generated extractors can then be imported into a conventional Eclipse-based IDE for further enhancement. The results of our user studies indicate that VINERY is a significant step forward in facilitating extractor development for both expert and novice IE developers.

A real-world information extraction (IE) system for semi-structured document images often involves a long pipeline of multiple modules, whose complexity dramatically increases its development and maintenance cost. One can instead consider an end- to-end model that directly maps the input to the target output and simplify the entire process. However, such generation approach is known to lead to unstable performance if not designed carefully. Here we present our recent effort on transitioning from our existing pipeline-based IE system to an end-to-end system focusing on practical challenges that are associated with replacing and deploying the system in real, large-scale production. By carefully formulating document IE as a sequence generation task, we show that a single end-to-end IE system can be built and still achieve competent performance.

RINX: A system for information and knowledge extraction from resumes

A massive amount of data/information that is buried in unstructured data sources such as bridge rehabilitation reports offer opportunities to complement other data sources such as structural health monitoring data for improved prediction of future bridge conditions. Extracting these data/information from their sources is, however, challenging for two reasons: (1) utilizing textual data for analytics remains to be a challenge due to the inherently unstructured nature of these data; and (2) existing information extraction methods, which aim to extract structured data/information from text, are limited in their capabilities and generalizability (e.g., a method developed to extract certain information from one type of document may not perform well on another type). To address these gaps, this paper proposes a deep learning-based information extraction (IE) model that automatically recognizes and extracts bridge-related data/information (e.g., bridge deficiencies) from multiple types of unstructured textual sources and represents the extracted data/information in a structured format to support further data analytics. The proposed IE model utilizes deep learning-based BiLSTM; begin, inside, and outside (BIO) encoding for the phrase-level segmentation; and conditional random fields (CRF) for both word-level and phrase-level labeling for automatic sequence labeling. The paper discusses the proposed model and its performance results.

This study compares the effectiveness of three methods-Fréchet Distance, Dynamic Time Warping (DTW), and Convolutional Neural Networks (CNNs)-in detecting similarities and pattern recognition in time series. It proposes a Pattern-Driven Case Generator (PDCG) framework to automate the creation of labeled time series data for training CNN models, addressing the challenge of manual dataset curation. By injecting controlled noise and interpolating diverse shapes (e.g., W/M/nAn/vVv), a PDCG synthesizes realistic training data that enhances model robustness. Experimental results demonstrate that the CNN model, trained with 10,000 PDCG-generated cases, achieves 86-98% accuracy in pattern recognition, outperforming traditional methods (Fréchet and DTW) for complex, misaligned, and variable-length sequences. The PDCG-enhanced CNN's scalability and adaptability improve with larger datasets, validating the PDCG's efficacy in bridging simulation and real-world applications.

The tremendous advances in deep neural networks have demonstrated the superiority of deep learning techniques for applications such as object recognition or image classification. Nevertheless, deep learning-based methods usually require a large amount of training data, which mainly comes from manual annotation and is quite labor-intensive. In order to reduce the amount of manual work required for generating enough training data, we hereby propose to leverage existing labeled data to generate image annotations automatically. Specifically, the pixel labels are firstly transferred from one image modality to another image modality via geometric transformation to create initial image annotations, and then additional information (e.g., height measurements) is incorporated for Bayesian inference to update the labeling beliefs. Finally, the updated label assignments are optimized with a fully connected conditional random field (CRF), yielding refined labeling for all pixels in the image. The proposed approach is tested on two different scenarios, i.e., (1) label propagation from annotated aerial imagery to unmanned aerial vehicle (UAV) imagery and (2) label propagation from map database to aerial imagery. In each scenario, the refined image labels are used as pseudo-ground truth data for training a convolutional neural network (CNN). Results demonstrate that our model is able to produce accurate label assignments even around complex object boundaries; besides, the generated image labels can be effectively leveraged for training CNNs and achieve comparable classification accuracy as manual image annotations, more specifically, the per-class classification accuracy of the networks trained by the manual image annotations and the generated image labels have a difference within ± 5 % .

One promising application of natural language processing (NLP) research is in the area of information extraction (IE). In this paper, we present work flow of our IE system for the extraction of semantically rich information from the unstructured or semi-structured Chinese web pages. Knowledge engineering approach and automatic training approach are used to extract pattern and built knowledge repository. General IE system needs to label the unlabeled training Web pages. A novel methodology that does not need to label text is developed, including hierarchy filtration pattern matching based on syntax in best distance method and maximum forward boundary recognition using organization suffix repository and part of speech tagging method. As for applications of IE, a new application system based on IE is built. It is object-level vertical search system and object here is Chinese people, so IE is concerned with extracting people's related attributes from a collection of web pages about Chinese people. The results are displayed as hierarchy directory tree according to people's attributes. The system makes user find people quickly and easily.

A variety of real-world applications such as complex industry process usually are lack of abundant training samples since the data acquiring process is time and labor consuming. Hence, it is important to utilize the limited training samples to build a sophisticated data-driven model, which may improve industry productivity. Recently, nonlinear learning models such as artificial neural networks and support vector machines have shown to be effective in modeling small-scale data by their strong modeling ability. However, these nonlinear learning models work as a black box and are often not human understandable and are difficult to be interpreted. In addition, in many applications, domain experts could provide us valuable expertise knowledge which may help further improve the modeling process. In this paper, we propose to integrate expertise knowledge to the nonlinear learning model to advance the data-driven modeling process in real-world applications. Experimental results on six benchmark datasets and a real-world industry application validate the effectiveness of the proposed model.

The development of an information extraction (IE) system for Thai documents raises a number of issues which are not important for IE in English and other European languages. We describe the characteristics of written Thai and the problem statements, and our approach to the Thai IE system. The structure of written Thai is highly ambiguous, which requires more sophisticated techniques than are necessary to perform comparable IE tasks in most European languages, and large amounts of domain knowledge to cope with these ambiguities. The basic characteristic of this system is to provide different natural language components to assess the surface structure of the documents. These components include word segmentation, specific lexical structure terms identification and part-of-speech tagger. Further analysis is to perform a shallow parsing based on the relevant regions that contain the specific trigger terms or patterns specified in the extraction templates. Finally, the information of interest is extracted from the grammar trees in corresponding to predefined concept definitions and returns the users with a list of answers responding to each concept.

Machine learning (ML) is becoming a key enabler in building energy management systems (BEMS), yet most existing reviews focus on simulations and fail to reflect the realities of real-world deployment. In response to this limitation, the present work aims to present a systematic review dedicated entirely to experimental, field-tested applications of ML in BEMS, covering systems such as Heating, Ventilation & Air-conditioning (HVAC), Renewable Energy Systems (RES), Energy Storage Systems (ESS), Ground Heat Pumps (GHP), Domestic Hot Water (DHW), Electric Vehicle Charging (EVCS), and Lighting Systems (LS). A total of 73 real-world deployments are analyzed, featuring techniques like Model Predictive Control (MPC), Artificial Neural Networks (ANNs), Reinforcement Learning (RL), Fuzzy Logic Control (FLC), metaheuristics, and hybrid approaches. In order to cover both methodological and practical aspects, and properly identify trends and potential challenges in the field, current review uses a unified framework: On the methodological side, it examines key-attributes such as algorithm design, agent architectures, data requirements, baselines, and performance metrics. From a practical standpoint, the study focuses on building typologies, deployment architectures, zones scalability, climate, location, and experimental duration. In this context, the current effort offers a holistic overview of the scientific landscape, outlining key trends and challenges in real-world machine learning applications for BEMS research. By focusing exclusively on real-world implementations, this study offers an evidence-based understanding of the strengths, limitations, and future potential of ML in building energy control—providing actionable insights for researchers, practitioners, and policymakers working toward smarter, grid-responsive buildings. Findings reveal a maturing field with clear trends: MPC remains the most deployment-ready, ANNs provide efficient forecasting capabilities, RL is gaining traction through safer offline–online learning strategies, FLC offers simplicity and interpretability, and hybrid methods show strong performance in multi-energy setups.