An Efficient Trie-based Method for Approximate Entity Extraction with Edit-Distance Constraints

Approximate substring matching is a common problem in many applications. In this paper, we study approximate substring matching with edit distance constraints. Existing methods are very sensitive to query strings or query parameters like query length and edit distance. To address the problem, we propose a new approach using partition scheme. It first partitions a query into several segments, and finds matching substrings of these segments as candidates, then performs a bidirectional verification on these candidates to get final results. We devise an even partition scheme to efficiently find candidates, and a best partition scheme to find high quality candidates. Furthermore, through theoretical analysis, we find that the best partition scheme cannot always outperform the even partition scheme. Thus we propose an adaptive approach for selectively choosing scheme using statistic knowledge. We conduct comprehensive experiments to demonstrate the efficiency and quality of our proposed method.

Graphs are widely used to model complex data in many applications, such as bioinformatics, chemistry, social networks, pattern recognition. A fundamental and critical query primitive is to efficiently search similar structures in a large collection of graphs. This article mainly studies threshold-based graph similarity search with edit distance constraints. Existing solutions to the problem utilize fixed-size overlapping substructures to generate candidates, and thus become susceptible to large vertex degrees and distance thresholds. In this article, we present a partition-based approach to tackle the problem. By dividing data graphs into variable-size non-overlapping partitions, the edit distance constraint is converted to a graph containment constraint for candidate generation. We develop efficient query processing algorithms based on the novel paradigm. Moreover, candidate-pruning techniques and an improved graph edit distance verification algorithm are developed to boost the performance. In addition, a cost-aware graph partitioning method is devised to optimize the index. Extending the partition-based filtering paradigm, we present a solution to the top- $$k$$ graph similarity search problem, where tailored filtering, look-ahead and computation-sharing strategies are exploited. Using both public real-life and synthetic datasets, extensive experiments demonstrate that our approaches significantly outperform the baseline and its alternatives.

Named Entity Recognition (NER) is a vital task in natural language processing (NLP) that involves identifying and classifying named entities in text. This paper focuses on the comparison of four different algorithms of NER techniques in the context of India. The four different algorithms are CRF (Conditional Random Fields), Stanza, Spacy, and Nameparser. The accuracy will be compared with the expected accuracy of all four methods of NER. Accuracy will be obtained after running the method on a dataset or text. To calculate the accuracy of four algorithms a large dataset of Indian names will be used. A comparison graph was obtained to display the accuracy graph of all four algorithms. "NLP-Named Entity Recognition in India" which focuses on detecting person names, states, and districts, and suggesting names based on religion. Named Entity Recognition (NER) is a Natural Language Processing (NLP) task that identifies and classifies named entities in text. Named entities are typically proper nouns, such as person names, organization names, and location names. NER is a challenging task for Indian languages, due to the diversity of Indian languages and the lack of resources for training NER systems. The proposed system can be used to improve the accuracy and efficiency of a variety of NLP applications in India.

Named Entity Recognition (NER) is an Information Extraction (IE) building block. Though the information extraction process has been automated using various techniques to find and extract relevant information from unstructured documents, the discovery of targeted knowledge still poses many research difficulties because of Web data's variability and lack of structure. NER, a subtask of IE, came to exist to smooth such difficulty. It deals with finding the proper names (named entities), such as a person's name, country, location, organization, dates, and event in a document. It categorises them as predetermined labels, an initial step in IE tasks. This survey paper presents the roles and importance of NER to IE from the perspective of different algorithms and application area domains. Additionally, it summarizes how researchers implemented NER in particular application areas like finance, medicine, defense, business, food science, archeology, etc. It also outlines the three NER sequence labeling algorithms types: feature-based, neural network-based, and rule-based. Finally, the state-of-the-art and evaluation metrics of NER were presented.

Named Entity Recognition plays an important role in locating and classifying atomic elements into predefined categories such as person names, locations, organizations, expression of times, temporal expressions etc. Several approaches with rule based and machine learning based techniques have been applied on English and some other Latin languages successfully. Arabic has a complex and rich morphology, which makes the named entities recognition a challenging process. In this paper we propose our hybrid NER system that applies conditional random fields (CRF), bilingual NE lexicon and grammar rules to the task of Named Entity Recognition in Arabic languages. The aim of our system is enhancing the overall performance of NER tasks. The empirical results indicate that the hybrid system outperforms the state-of-the-art of Arabic NER in terms of precision when applied to ANERcorp dataset, with f-measures 83.36 for Person, 89.58for Location, and 72.26 for Organization.

Named Entity Recognition (NER), a core task in Information Extraction (IE) alongside Relation Extraction (RE), identifies and extracts entities like place and person names in various domains. NER has improved business processes in both public and private sectors but remains underutilized in government institutions, especially in developing countries like Indonesia. This study examines which government fields have utilized NER over the past five years, evaluates system performance, identifies common methods, highlights countries with significant adoption, and outlines current challenges. Over 64 international studies from 15 countries were selected using PRISMA 2020 guidelines. The findings are synthesized into a preliminary ontology design for Government NER.

Named entity recognition (NER) is one of the preprocessing stages in natural language processing (NLP), which functions to detect and classify entities in the corpus. NER results are used in various NLP applications, including sentiment analysis, text summarization, chatbot, machine translation, and question answering. Several previous reviews partially discussed NER, for instance, NER reviews in specific domains, NER classification, and NER deep learning. This paper provides a comprehensive and systematic review on NER topic studies published from 2011 to 2020. The main contribution of this review is to present a comprehensive systematic literature review on NER from preprocessing techniques, datasets, application domains, feature extraction techniques, approaches, methods, and evaluation techniques. The result concludes that the deep learning approach and the Bi-directional long short-term memory with a conditional random field (Bi-LSTM-CRF) method are the most interesting methods among NER researchers. At the same time, medical and health are NER researchers' most popular domains. These developments have also led to an increasing number of public datasets in the medical and health fields. At the end of this review, we recommend some opportunities and challenges for NER research going forward.

Named entities (NEs) are textual references via proper names, such as people names, company names, places and so on. The importance of NEs has been observed in intranet search engines, including university web sites. In this paper, a mechanism is built exclusively to recognize the three named entities, which are constantly referenced in the University of Essex domain: names, course codes, and room numbers. While a person name is considered a common named entity, course codes and room numbers are specific to the University domain. We developed a technique specifically to train three different classifiers on electronic corpora, consisting of 16,629 examples in total, which were collected and annotated manually from the University domain. The resulting models were then incorporated into the NER system that was built to use pre-trained classifiers in the detection process, mark these NEs, and cross-reference them to the related documents. The proposed method performed well on a test corpus, with the average precision reaching nearly 0.97. The recall varied, but was lower overall than precision with an average of 0.82. Moreover, in terms of name recognition in the University domain, our system outperformed two other systems: the OpenNLP name finder and ANNIE system.

Automatic translation systems are known to struggle with rare words. Among these, named entities (NEs) and domain-specific terms are crucial, since errors in their translation can lead to severe meaning distortions. Despite their importance, previous speech translation (ST) studies have neglected them, also due to the dearth of publicly available resources tailored to their specific evaluation. To fill this gap, we i) present the first systematic analysis of the behavior of state-of-the-art ST systems in translating NEs and terminology, and ii) release NEuRoparl-ST, a novel benchmark built from European Parliament speeches annotated with NEs and terminology. Our experiments on the three language directions covered by our benchmark (en->es/fr/it) show that ST systems correctly translate 75-80% of terms and 65-70% of NEs, with very low performance (37-40%) on person names.

In this paper we present an approach to person coreference in a large collection of news, based on two main hypothesis: first, coreference is an iterative process, where the easy cases are addressed first and are then made available as an incrementally enriched resource for resolving more difficult cases. Second, at each iteration coreference among two person names is established according to a probabilistic model, where a number of features (e.g. frequency of first and last names) are taken into account. The approach does not assume any prior knowledge about persons mentioned in the collection and requires basic linguistic processing (named entity recognition) and resources (a dictionary of person names). The system parameters have been estimated on a Italian news corpus of 5K and then experimented on a collection containing more than 7 millions person names. Evaluation, over a sample of four days news, shows that the error rate of the system (1.4%) is above a baseline (5.4%) for the task. Finally, we discuss open issues for evaluation.KeywordsCross Document CoreferenceName FrequencyDynamic Cascade ClusteringRandom Interval Coverage

Threat intelligence named entity recognition techniques based on few-shot learning

The increasing use of Arabic and Urdu on social media platforms, particularly Twitter, has created a growing need for robust Named Entity Recognition (NER) systems capable of handling noisy, informal, and code-mixed content. However, both languages remain significantly underrepresented in NER research, especially in social media contexts. To address this gap, this study makes four key contributions: (1) We introduced a manual entity consolidation step to enhance the consistency and accuracy of named entity annotations. In the original datasets, entities such as person names and organization names were often split into multiple tokens (e.g., first name and last name labeled separately). We manually refined the annotations to merge these segments into unified entities, ensuring improved coherence for both training and evaluation. (2) We selected two publicly available datasets from GitHub—one in Arabic and one in Urdu—and applied two novel strategies to tackle low-resource challenges: a joint multilingual approach and a translation-based approach. The joint approach involved merging both datasets to create a unified multilingual corpus, while the translation-based approach utilized automatic translation to generate cross-lingual datasets, enhancing linguistic diversity and model generalizability. (3) We presented a comprehensive and reproducible pseudocode-driven framework that integrates translation, manual refinement, dataset merging, preprocessing, and multilingual model fine-tuning. (4) We designed, implemented, and evaluated a customized XLM-RoBERTa model integrated with a novel attention mechanism, specifically optimized for the morphological and syntactic complexities of Arabic and Urdu. Based on the experiments, our proposed model (XLM-RoBERTa) achieves 0.98 accuracy across Arabic, Urdu, and multilingual datasets. While it shows a 7–8% improvement over traditional baselines (RF), it also achieves a 2.08% improvement over a deep learning (BiLSTM = 0.96), highlighting the effectiveness of our cross-lingual, resource-efficient approach for NER in low-resource, code-mixed social media text.

This paper reports about a multi-engine approach for the development of a Named Entity Recognition (NER) system in Bengali by combining the classifiers such as Maximum Entropy (ME), Conditional Random Field (CRF) and Support Vector Machine (SVM) with the help of weighted voting techniques. The training set consists of approximately 272K wordforms, out of which 150K wordforms have been manually annotated with the four major named entity (NE) tags, namely Person name, Location name, Organization name and Miscellaneous name. An appropriate tag conversion routine has been defined in order to convert the 122K wordforms of the IJCNLP-08 NER Shared Task on South and South East Asian Languages (NERSSEAL)1 data into the desired forms. The individual classifiers make use of the different contextual information of the words along with the variety of features that are helpful to predict the various NE classes. Lexical context patterns, generated from an unlabeled corpus of 3 million wordforms in a semi-automatic way, have been used as the features of the classifiers in order to improve their performance. In addition, we propose a number of techniques to post-process the output of each classifier in order to reduce the errors and to improve the performance further. Finally, we use three weighted voting techniques to combine the individual models. Experimental results show the effectiveness of the proposed multi-engine approach with the overall Recall, Precision and F-Score values of 93.98%, 90.63% and 92.28%, respectively, which shows an improvement of 14.92% in F-Score over the best performing baseline SVM based system and an improvement of 18.36% in F-Score over the least performing baseline ME based system. Comparative evaluation results also show that the proposed system outperforms the three other existing Bengali NER systems.

The bioproduction of astaxanthin: A comprehensive review on the microbial synthesis and downstream extraction

Parallel data processing complicates the completion of string similarity joins because parallel data processing requires the use of a well designed data partitioning scheme. Moreover, efficient verification of string pairs is needed to speed up the entire string similarity join process. We propose a novel framework that addresses these requirements through the use of edit distance constraints. The Landmark-Join framework has two functions that reduce two kinds of search spaces. The first, q-bucket partitioning, reduces the number of verifications of dissimilar string pairs and lowers skewness among buckets. The second, local upper bound calculation, prunes the search space of edit distance to speed up each verification. Experimental results show that Landmark-Join has good parallel scalability and that the two proposed functions speed up the entire string similarity join process.