Information Retrieval with Reinforced Word Classes

Word classes (‘parts-of-speech’, ‘syntactic categories’, ‘lexical categories’) are the fundamental building blocks of linguistic expressions in all natural human languages. They have been investigated since antiquity and continue to play a central role in modern linguistics. Today an increasingly important role is assigned to the information that is specified in the lexical entry of a word in the lexicon (including, of course, information about its category membership), both in formal and in functional approaches to grammar. Furthermore, there is a growing awareness that the traditional set of word classes is biased towards the better studied European languages and needs to be revised to accommodate unfamiliar word classes in nonEuropean languages.

The field of information retrieval has witnessed tangible progress over the past decades in response to the expanded usage of the internet and the dire need of users to search for massive amounts of digital information. Given the steady increase of Arabic e-content, excellent information retrieval systems must be devised to suit the nature and requirements of the Arabic language. This paper sheds light on the current progress in the field of Arabic information retrieval, identifies the challenges that hinder the progress of this science, and proposes suggestions for further research. This paper uses the descriptive analytical method to examine the reality of Arabic studies in the field of information retrieval and to study the problems that are being faced in this area. Specifically, the previous literature on information retrieval is reviewed by searching the related databases and websites.

Query expansion is a classical topic in the field of information retrieval, which is proposed to bridge the gap between searchers' information intents and their queries. Previous researches usually expand queries based on document collections, or some external resources such as WordNet and Wikipedia [1, 2, 3, 4, 5]. However, it seems that independently using one of these resources has some defects, document collections lack semantic information of words, while WordNet and Wikipedia may not include domain-specific knowledge in certain document collection. Our work aims to combine these two kinds of resources to establish an expansion model which represents not only domain-specific information but also semantic information. In our preliminary experiments, we construct a two-layer word graph and use Random-Walk algorithm to calculate the weights of each term in pseudo-relevance feedback documents, then select the highest weighted term to expand original query. The first layer of the word graph contains terms in related documents, while the second layer contains semantic senses corresponding to these terms. These terms and semantic senses are treated as vertices of the graph and connected with each other by all possible relationships, such as mutual information and semantic similarities. We utilized mutual information, semantic similarity and uniform distribution as the weight of term-term relation, sense-sense relation and word-sense relation respectively. Though these experiments show that our expansion outperform original queries, we are troubled with some difficult problems. Given the framework of semantic graph model, we need more effort to find out an optimal graph to represent the relationships between terms and their semantic senses. We utilized a two-layer graph model in our preliminary research, where terms from different documents are treated equally. Maybe we can introduce the document as a third layer in future work, where we can differ the same terms in different documents according to document relevance and context. Then we need appropriately represent initial weights of this words, senses and relationships. Various measures for weights of terms and term relations have been proved effective in other information retrieval tasks, such as TFIDF, mutual information (MI), but there is little research on weights for semantic senses and their relations. For polysemous words, we add all of their semantic senses to the graph and assume that these senses are uniformly distributed. Actually, it is not precise for a word in a special document and query. As we know, a polysemous word may have only one or two senses in a document, and they are not uniformly distributed. Give a word, what we should do is to determine its word senses in a relevant document and estimate the distribution of these senses. Word sense disambiguation may help us in this problem. Then, there are many methods to compute word similarity according to WordNet, which we use to represent the weights of relationships between word senses. Varelas et al implemented some popular methods to compute semantic similarity by mapping terms to an ontology and examining their relationships in that ontology [4]. We also need to know which algorithm for semantic similarity is most suitable for our model. Additional, WordNet is suitable to calculate word similarity but not suitable to measure word relevance. The inner hyperlinks of Wikipedia could help us to calculate word relevance. We wish to find an effective way to combine the similarity measure from WordNet and relevance measure from Wikipedia, which may completely reflect word relationships.

The document-length normalization problem has been widely studied in the field of information retrieval. The cosine normalization (Baeza-Yates and Ribeiro-Neto, 1999), the maximum if normalization (Allan et al., 1997) and the byte length normalization (Robertson et al., 1992) are the most commonly used normalization techniques. In (Singhal et al., 1996), authors studied the retrieval probability of documents w.r.t. their size, using different similarity measures. They have shown that none of existing measures retrieve the documents of different lengths with the same probability. We first show here that the document and query sizes are indeed very influent on the similarity score expectation. Therefore, we propose to realize a statistical regression of the similarity scores distribution w. r. t. document and query sizes in order to normalize them. Experimental results appear to indicate that our approach, as well in the field of classical Information Retrieval as when applied to a document clustering process, allows to judge similarities really more fairly.

Learning a high-dimensional dense representation for vocabulary terms, also known as a word embedding, has recently attracted much attention in natural language processing and information retrieval tasks. The embedding vectors are typically learned based on term proximity in a large corpus. This means that the objective in well-known word embedding algorithms, e.g., word2vec, is to accurately predict adjacent word(s) for a given word or context. However, this objective is not necessarily equivalent to the goal of many information retrieval (IR) tasks. The primary objective in various IR tasks is to capture relevance instead of term proximity, syntactic, or even semantic similarity. This is the motivation for developing unsupervised relevance-based word embedding models that learn word representations based on query-document relevance information. In this paper, we propose two learning models with different objective functions; one learns a relevance distribution over the vocabulary set for each query, and the other classifies each term as belonging to the relevant or non-relevant class for each query. To train our models, we used over six million unique queries and the top ranked documents retrieved in response to each query, which are assumed to be relevant to the query. We extrinsically evaluate our learned word representation models using two IR tasks: query expansion and query classification. Both query expansion experiments on four TREC collections and query classification experiments on the KDD Cup 2005 dataset suggest that the relevance-based word embedding models significantly outperform state-of-the-art proximity-based embedding models, such as word2vec and GloVe.

Paraphrase detection allows determining how original and suspect documents convey the same meaning. It has attracted attention from researchers in many Natural Language Processing (NLP) tasks such as plagiarism detection, question answering, information retrieval, etc., Traditional methods (e.g., Term Frequency-Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), and Latent Semantic Analysis (LSA)) cannot capture efficiently hidden semantic relations when sentences may not contain any common words or the co-occurrence of words is rarely present. Therefore, we proposed a deep learning model based on Global Word embedding (GloVe) and Recurrent Convolutional Neural Network (RCNN). It was efficient for capturing more contextual dependencies between words vectors with precise semantic meanings. Seeing the lack of resources in Arabic language publicly available, we developed a paraphrased corpus automatically. It preserved syntactic and semantic structures of Arabic sentences using word2vec model and Part-Of-Speech (POS) annotation. Overall experiments shown that our proposed model outperformed the state-of-the-art methods in terms of precision and recall

The document retrieval is one of the fast growing and complex research area in the field of information retrieval. Concept-based information retrieval is more beneficial than keyword-based access. The most important advantage is the ability to make use of semantic relationships among concepts in finding relevant documents. Removal of irrelevant documents by identifying conceptual mismatches is another benefit. The linguistic representation of concepts is words and phrases. These concepts are mental structures. The inherent conciseness of natural language can make words represent multiple concepts. Different words can represent the same or similar concepts. We present a novel method of concept mining methodology, which provides an efficient way of document retrieval. The performance of this proposed retrieval system was analyzed by the existing methods for different medical documents. The proposed retrieval system shows that the documents are retrieved effectively based on concepts and the system performs with higher accuracy value.

Research and experimentation using Arabic WordNet in the field of information retrieval are relatively new. It is limited compared to the research that has been done using Princeton WordNet. This work attempts to study the impact of Arabic WordNet on the performance of Arabic information retrieval. We extend Lucene with Arabic WordNet to expand user's queries. The major contribution of this study is to propose an interactive query expansion (IQE) methodology using the word's part-of-speech, according to the part it plays in a query. First, the user selects the appropriate part of speech for each term in the original query, and then he reselects the appropriate synonyms. Experimental results show that our IQE strategy produces a good Mean Average Precision (MAP), it is able to improve MAP by 12.6%, but no variant of automatic query expansion (AQE) strategies did. Nevertheless, the experiments allow us to conclude that with an appropriate use of Arabic WordNet as a source of linguistic information for AQE can improve effectiveness for Arabic information retrieval.

This article presents a literature review of computer-science-related research applied on hadith, a kind of Arabic narration which appeared in the 7th century. We study and compare existent works in several fields of Natural Language Processing (NLP), Information Retrieval (IR), and Knowledge Extraction (KE). Thus, we illicit their main drawbacks and identify some perspectives, which may be considered by the research community. We also study the characteristics of these types of documents, by enumerating the advantages/limits of using hadith as a language resource. Moreover, our study shows that previous studies used different collections of hadiths, thus making it hard to compare their results objectively. Besides, many preprocessing steps are recurrent through these applications, thus wasting a lot of time. Consequently, the key issues for building generic language resources from hadiths are discussed, taking into account the relevance of related literature and the wide community of researchers that are interested in these narrations. The ultimate goal is to structure hadith books for multiple usages, thus building common collections which may be exploited in future applications.

Road traffic crashes (RTCs) are a global public health issue, with traditional analysis methods often hindered by delays and incomplete data. Leveraging social media for real-time traffic safety analysis offers a promising alternative, yet effective frameworks for this integration are scarce. This study introduces a novel multitask learning (MTL) framework utilizing large language models (LLMs) to analyze RTC-related tweets from Australia. We collected 26,226 traffic-related tweets from May 2022 to May 2023. Using GPT-3.5, we extracted fifteen distinct features categorized into six classification tasks and nine information retrieval tasks. These features were then used to fine-tune GPT-2 for language modeling, which outperformed baseline models, including GPT-4o mini in zero-shot mode and XGBoost, across most tasks. Unlike traditional single-task classifiers that may miss critical details, our MTL approach simultaneously classifies RTC-related tweets and extracts detailed information in natural language. Our fine-tunedGPT-2 model achieved an average accuracy of 85% across the six classification tasks, surpassing the baseline GPT-4o mini model’s 64% and XGBoost’s 83.5%. In information retrieval tasks, our fine-tuned GPT-2 model achieved a BLEU-4 score of 0.22, a ROUGE-I score of 0.78, and a WER of 0.30, significantly outperforming the baseline GPT-4 mini model’s BLEU-4 score of 0.0674, ROUGE-I score of 0.2992, and WER of 2.0715. These results demonstrate the efficacy of our fine-tuned GPT-2 model in enhancing both classification and information retrieval, offering valuable insights for data-driven decision-making to improve road safety. This study is the first to explicitly apply social media data and LLMs within an MTL framework to enhance traffic safety.

Analyzing and modeling user behavior in the Information Retrieval (IR) process is important for IR research. In this paper, we propose a new research schema in IR research. In this schema, we first investigate humans' cognitive process when completing a specific IR task and build cognitive models for that task. The findings in the investigation and the proposed cognitive models are then utilized to improve machines' performance in the IR task. Through this research schema, we revisit three IR tasks. In the first study, we carefully analyze users' clicking behavior in mobile search and propose a mobile click model to extract relevance feedback from the mobile search logs. In the second study, we conduct an eye-tracking study to investigate how human assessors read a document during relevance judgment task and adopt the findings in building a novel retrieval model that can better approximate humans' relevance judgment. In the last study, we conduct another eye-tracking study to investigate humans' reading behavior when completing the reading comprehension task. We build a prediction model for user attention and leverage the predicted attention signals to improve the machine reading comprehension model. By successfully adopting this research schema to three different IR tasks, we demonstrate its effectiveness and generalizability.

Previous research especially based on the concept of intertextuality has denoted an important shift in perspective considering the relationship between text and narrator. The distinction between narrator and voice or, as Roland Barthes formulated in the title of his famous essay ‘The Death of the Author’ (1967), likewise Foucault, dispensed with the idea of authorship. This approach may also provide a deeper insight in the construction of fragments, that – as has been rightly remarked – are made rather than born (Dionisotti, above). However, although prior work has already been focused on the possibilities of methods from the field of Information Retrieval especially concerning the digital editing of quotations and fragments, this study argues the importance of combining textmining methods with approaches from narratology. Within the eAQUA project we adapt and use tools from the field of NLP (Natural Language Processing) to support our scientific work. In this paper we describe the extent to which the complex relationship between quotations as fragmentary texts and their surrounding context benefits from algorithm-based methods like word level n-gams as features in classification and referencing of fragments. The examples described in this paper using eAQUA’s Citatongraph demonstrate that the Citationgraph gives an excellent estimate of quotations and text reuse. Concerning the question of fragments this tool remarkably broadens our possibilities helping to ‘unearth’ fragments lost in the process of editorial decisions. Representing a kind of ‘unediting’ whereby the various layers of editorial decisions can be decoded this new approach yields a promising new avenue of research.

Signal detection theory--as developed in electrical engineering and based on statistical decision theory--was first applied to human sensory discrimination 40 years ago. The theoretical intent was to provide a valid model of the discrimination process; the methodological intent was to provide reliable measures of discrimination acuity in specific sensory tasks. An analytic method of detection theory, called the relative operating characteristic (ROC), can isolate the effect of the placement of the decision criterion, which may be variable and idiosyncratic, so that a pure measure of intrinsic discrimination acuity is obtained. For the past 20 years, ROC analysis has also been used to measure the discrimination acuity or inherent accuracy of a broad range of practical diagnostic systems. It was widely adopted by methodologists in the field of information retrieval, is increasingly used in weather forecasting, and is the generally preferred method in clinical medicine, primarily in radiology. This book attends to both themes, ROC analysis in the psychology laboratory and in practical diagnostic settings, and to their essential unity. The focus of this book is on detection and recognition as fundamental tasks that underlie most complex behaviors. As defined here, they serve to distinguish between two alternative, confusable stimulus categories, which may be perceptual or cognitive categories in the psychology laboratory, or different states of the world in practical diagnostic tasks. This book on signal detection theory in psychology was written by one of the developers of the theory, who co-authored with D.M. Green the classic work published in this area in 1966 (reprinted in 1974 and 1988). This volume reviews the history of the theory in engineering, statistics, and psychology, leading to the separate measurement of the two independent factors in all discrimination tasks, discrimination acuity and decision criterion. It extends the previous book to show how in several areas of psychology--in vigilance and memory--what had been thought to be discrimination effects were, in reality, effects of a changing criterion. The book shows that data plotted in terms of the relative operating characteristic have essentially the same form across the wide range of discrimination tasks in psychology. It develops the implications of this ROC form for measures of discrimination acuity, pointing up the valid ones and identifying several common, but invalid, ones. The area under the binormal ROC is seen to be supported by the data; the popular measures d' and percent correct are not. An appendix describes the best, current programs for fitting ROCs and estimating their parameters, indices, and standard errors. The application of ROC analysis to diagnostic tasks is also described. Diagnostic accuracy in a wide range of tasks can be expressed in terms of the ROC area index. Choosing the appropriate decision criterion for a given diagnostic setting--rather than considering some single criterion to be natural and fixed--has a major impact on the efficacy of a diagnostic process or system. Illustrated here by separate chapters are diagnostic systems in radiology, information retrieval, aptitude testing, survey research, and environments in which imminent dangerous conditions must be detected. Data from weather forecasting, blood testing, and polygraph lie detection are also reported. One of these chapters describes a general approach to enhancing the accuracy of diagnostic systems.

Stemming is a mechanism of word form normalization that transforms the variant word forms to their common root. In an Information Retrieval system, it is used to increase the system’s performance, specifically the recall and desirably the precision. Although its usefulness is shown to be mixed in languages such as English, because morphologically complex languages stemming produces a significant performance improvement. A number of linguistic rule-based stemmers are available for most European languages which employ a set of rules to get back the root word from its variants. But for Indian languages which are highly inflectional in nature, devising a linguistic rule-based stemmer needs some additional resources which are not available. We present an approach which is purely corpus based and finds the equivalence classes of variant words in an unsupervised manner. A set of experiments on four languages using FIRE, CLEF, and TREC test collections shows that our approach provides comparable results with linguistic rule-based stemmers for some languages and gives significant performance improvement for resource constrained languages such as Bengali and Marathi.

In information retrieval, vocabulary mismatch between the search vocabulary and documents vocabulary has been a common challenge for web users, hindering information access and retrieval. This issue is often attributed to the ambiguous representation of user information needs as queries, leading to the retrieval of many irrelevant documents, particularly for non-skilled web users. This paper aims to improve user query representation for effective retrieval of relevant documents from the web. To achieve this, a query expansion strategy was employed to identify terms with similar meanings to the user’s initial query term. The similarities between the expanded terms and the initial user query term were determined by calculating the cosine angle between the vector resenting document vocabulary and the vector representing query term. Thereafter, Formal Concept Analysis (FCA) was employed to analyze and present the results. Findings from the analysis revealed that concatenating similar terms with the initial user query terms resulted in a 0.16% improvement, as evident in the retrieval precision of 0.64% with the initial query and 0.80% with the expanded query terms.