A hybrid semantic query expansion approach for Arabic information retrieval

A new approach for query expansion using Wikipedia and WordNet

Query-document vocabulary mismatch, the lack of query expressiveness for user needs and the phenomenon of short queries are the main issues associated with information retrieval systems. Query Expansion (QE) is one of the well-known alternative for overcoming these problems. It mainly involves finding synonyms or related words for the query terms. There are several approaches in the query expansion field such as statistical and semantic approaches; they focus on expanding the individual query terms rather than the entire query during the expansion process. An other category of approaches deals with the whole query by using a neural approach based on Pseudo Relevance feedback (PRF) documents. In this work, we carried out an ablation study to measure the impact of the classical and semantic (word embedding, order, context) based query expansion on the retrieval performance. The experiments conducted on the Arabic EveTAR dataset reveal that our hybrid proposed approach combining classical (PRF) and transformer (AraBERT) is competitive with the state-of-the-art methods. In fact, the obtained result in terms of the Mean Average Precision (MAP) is up to 0.72.

The internet has been inundated with an ocean of information, and hence, information retrieval systems are failing to provide optimal results to the user. In order to meet the challenge, query expansion techniques have emerged as a game-changer and are improving the results of information retrieval significantly. Of late, semantic query expansion techniques have attracted increased interest among researchers since these techniques offer more pertinent and practical results to the users. These allow the user to retrieve more meaningful and useful information from the web. Currently, few research works provide a comprehensive review on semantic query expansion; usually, they cannot provide a full view on recent advances, diversified data application, and practical challenges. Therefore, it is imperative to go deep in review in order to explain these advances and assist researchers with concrete insights for future development. This article represents the comprehensive review of the query expansion methods, with a particular emphasis on semantic approaches. It overviews the recent frameworks that have been developed within a period of 2015-2024 and reviews the limitations of each approach. Further, it discusses challenges that are inherent in the semantic query expansion field and identifies some future research directions. This article emphasizes that the linguistic approach is the most effective and flexible direction for researchers to follow, while the ontology approach better suits domain-specific search applications. This, in turn, means that development of the ontology field may further open new perspectives for semantic query expansion. Moreover, by employing artificial intelligence (AI) and making most of the query context without relying on user intervention, improvements toward the optimal expanded query can be achieved.

This thesis investigates the question whether and how ontologies such as the ones currently evolving in the Semantic Web can serve as knowledge structures for the generation of query expansion terms in information retrieval systems. This issue is examined using a specific example domain, namely educational research. Initial results support the already well-researched finding that query expansion can increase recall. Subsequent experiments will focus on comparing the effectiveness of thesaurus and ontology-based query expansion, on identifying ontological relationships which are especially useful for the generation of query expansion terms in the domain of educational research, and on evaluating the usefulness of different ontological relationships as expansion terms for different types of queries, as well as for different query expansion modes.

Conventional approaches to query expansion (QE) rely on the integration of an unstructured corpus and probabilistic rules for the extraction of candidate expansion terms. These methods do not consider search query semantics, thereby resulting in ineffective retrieval of information. The semantic approaches for QE overcome this limitation, whereby a search query is expanded with meaningful terms that accord with user information needs. This paper surveys recent approaches to semantic QE that employ different models and strategies and leverages various knowledge structures. We organize these approaches into a taxonomy that includes linguistic methods, ontology-based methods, and mixed-mode methods. We also discuss the strengths and limitations of each type of semantic QE method. In addition, we evaluate various semantic QE approaches in terms of knowledge structure utilization, corpus collection, baseline model adaptation, and retrieval performance. Finally, future directions in exploiting personalized social information and multiple ontologies for semantic QE are suggested.

The effects of query structures and query expansion (QE) on retrieval performance were tested with a best match retrieval system (InQuery^1). Query structure means the use of operators to express the relations between search keys. Six different structures were tested, representing strong structures (e.g., queries with facets or concepts identified) and weak structures (no concepts identified, a query is ‘a bag of search keys’). QE was based on concepts, which were first selected from a searching thesaurus, and then expanded by semantic relationships given in the thesaurus. The expansion levels were (a) no expansion, (b) a synonym expansion, (c) a narrower concept expansion, (d) an associative concept expansion, and (e) a cumulative expansion of all other expansions. With weak structures and Boolean structured queries, QE was not very effective. The best performance was achieved with a combination of a facet structure, where search keys within a facet were treated as instances of one search key (the SYN operator), and the largest expansion.

The techniques of information retrieval eliminate the difficulty of finding specific information in electronic medical records (EMR), and the methods of query expansion (QE) improve the recall of EMR retrieval. However, most existing QE methods of EMR retrieval can’t get high-quality expansion terms and corresponding weights for Chinese EMR retrieval because of low-quality sources and improper calculation methods. In this paper, we propose a novel algorithm of expansion term selection and weight assignment for QE of Chinese EMR retrieval based on the clinical needs and unique characteristics of Chinese medical terms. The algorithm first selects expansion terms from a high quality Chinese medical knowledge graph and standard medical term sets, which can ensure the quality of expansion terms. Then it assigns weights to the selected expansion terms based on semantic similarities and manually designed expansion categories that reflect the opinions of medical experts. Experiment results show that our algorithm gets higher-quality expansion terms and more rational weights compared with four benchmark algorithms, using Precision at 10, Recall, Mean Average Precision and Binary Preference-based Measure as evaluation metrics, which implies that our algorithm can significantly improve the effectiveness of QE.

The aim of this paper is to retrieve the most relevant expansion words for expanding the initial query of the user in order to enhance the outcomes of web search results. Query expansion plays a major role in reformulating a user’s initial query to a one more pertinent to the user’s intended meaning. The reformulated query is then used to obtain more appropriate outcomes from a large amount of information on the web. The proposed semantic query expansion technique uses Wikipedia and WordNet as data sources. Wikipedia is taken as a base for all query expansions because it is one of the most diversified and relevant databases available on the web. To further improve the proposed query expansion technique,WordNet—a lexical database—is used as the as another data source because the synonyms (synsets) of the query term provided by it can be quite useful for query expansion. The proposed expansion technique successfully combines the two data sources to retrieve the most relevant expansion terms from the data sources in response to the user’s original query. The proposed work has been divided into four phases: (1) extraction of relevant words from Wikipedia (2) extraction of relevant words from WordNet (3) merging of the expansion terms obtained from Wikipedia and WordNet, and (4) query formulation by combining the expansion terms using Boolean operators. This reformulated query is then fired on the web to find the desired result. The Experimental result shows a significant improvement in information retrieval using query expansion.

With the development of the web 2.0 communities, information retrieval has been widely applied based on the collaborative tagging system. However, a user issues a query that is often a brief query with only one or two keywords, which leads to a series of problems like inaccurate query words, information overload and information disorientation. The query expansion addresses this issue by reformulating each search query with additional words. By analyzing the limitation of existing query expansion methods in folksonomy, this paper proposes a novel query expansion method, based on user profile and topic model, for search in folksonomy. In detail, topic model is constructed by variational antoencoder with Word2Vec firstly. Then, query expansion is conducted by user profile and topic model. Finally, the proposed method is evaluated by a real dataset. Evaluation results show that the proposed method outperforms the baseline methods.

Query Expansion is commonly used in Information Retrieval to overcome vocabulary mismatch issues, such as synonymy between the original query terms and a relevant document. In general, query expansion experiments exhibit mixed results. Overall TREC Genomics Track results are also mixed; however, results from the top performing systems provide strong evidence supporting the need for expansion. In this paper, we examine the conditions necessary for optimal query expansion performance with respect to two system design issues: IR framework and knowledge source used for expansion. We present a query expansion framework that improves Okapi baseline passage MAP performance by 185%. Using this framework, we compare and contrast the effectiveness of a variety of biomedical knowledge sources used by TREC 2006 Genomics Track participants for expansion. Based on the outcome of these experiments, we discuss the success factors required for effective query expansion with respect to various sources of term expansion, such as corpus-based cooccurrence statistics, pseudo-relevance feedback methods, and domain-specific and domain-independent ontologies and databases. Our results show that choice of document ranking algorithm is the most important factor affecting retrieval performance on this dataset. In addition, when an appropriate ranking algorithm is used, we find that query expansion with domain-specific knowledge sources provides an equally substantive gain in performance over a baseline system.

This paper presents an ongoing research on the comparison of ontological query expansion methods. Query Expansion is a technique that aims to enhance the results of a search by adding terms to the search query; today, it is a very important research topic in the semantic web and information retrieval areas. Although many efforts have been form the theoretical point of view to implements effective and general methods for expanding queries, based on both statistical and ontological approaches, the practical applicability of is nowadays restricted to few and very specific domains. The aim of this paper is the definition of a platform for the implementation of a subset of such methods, in order to make comparisons among them and try to define how and when use ontological QE. This work is part of JUMAS, a research project funded by European Community where query expansion is used to support the retrieval of signifiant information from audio–video transcriptions in the legal domain.

Query Expansion using Pseudo Relevance Feedback is a useful and a popular technique for reformulating the query. In our proposed query expansion method, we assume that relevant information can be found within a document near the central idea. The document is normally divided into sections, paragraphs and lines. The proposed method tries to extract keywords that are closer to the central theme of the document. The expansion terms are obtained by equi-frequency partition of the documents obtained from pseudo relevance feedback and by using tf-idf scores. The idf factor is calculated for number of partitions in documents. The group of words for query expansion is selected using the following approaches: the highest score, average score and a group of words that has maximum number of keywords. As each query behaved differently for different methods, the effect of these methods in selecting the words for query expansion is investigated. From this initial study, we extend the experiment to develop a rule-based statistical model that automatically selects the best group of words incorporating the tf-idf scoring and the 3 approaches explained here, in the future. The experiments were performed on FIRE 2011 Adhoc Hindi and English test collections on 50 queries each, using Terrier as retrieval engine.

Queries to search engines on the Web are usually short. They do not provide sufficient information for an effective selection of relevant documents. Previous research has proposed the utilization of query expansion to deal with this problem. However, expansion terms are usually determined on term co-occurrences within documents. In this study, we propose a new method for query expansion based on user interactions recorded in user logs. The central idea is to extract correlations between query terms and document terms by analyzing user logs. These correlations are then used to select high-quality expansion terms for new queries. Compared to previous query expansion methods, ours takes advantage of the user judgments implied in user logs. The experimental results show that the log-based query expansion method can produce much better results than both the classical search method and the other query expansion methods.

In this paper, we address there important issues with query expansion using a thesaurus; how to give weights to the terms in expanded queries, how to select additional search terms in the thesaurus, and how to enrich the terms in the manual thesaurus (namely, thesaurus reconstruction). To weight the terms in expanded queries, we construct the weighted thesaurus that has a similarity value between the terms in the thesaurus, using statistical co-occurrence in a corpus. To enrich the terms in the manual thesaurus, domain dependent terms which occur in a corpus are inserted into the weighted thesaurus using the co-occurrence information. In this paper, the reconstructed thesaurus with weights is defined as a domain-adapted, weighted thesaurus. Then we explain query expansion using the domain-adapted, weighted thesaurus in an extended Boolean retrieval model. To select additional search terms during query expansion, our model uses semi-automatic query expansion and a restriction method. In the experiments, our system had almost twice the recall of the boolean retrieval system not using the thesaurus or the query expansion retrieval system using the original thesaurus. And also, the precision of our system was almost the same precision as the other systems.

Most effective particular object and image retrieval approaches are based on the bag-of-words (BoW) model, and all state-of-the-art performance mainly involves a query expansion procedure, which is able to significantly improve retrieval results. Nowadays, Convolutional Neural Network(CNN) is widely applied in computer vision field, including image classification, caption, recognition and retrieval, etc. We introduce an extension to query expansion: an automatic method to select good candidate samples for interactive annotation which is used in query expansion using both BoW method and CNN feature. In this work, we address the query expansion framework using active learning, where the main focus is on the sample selection step in the process of query expansion. More specifically, we propose an active sample selection algorithm based on binary relevance classification, based on the assumption that most confusing samples of the classifiers have high probability to contain helpful true positives for query expansion, which significantly improves the retrieval performance. It takes full use of the multimodal information of the shortlist obtained from the basic retrieval to train a binary relevance classifier, which is used to pick up the most confusing samples for human annotation, with top list as unlabeled data and bottom list as fake negatives. And it can achieve a faster and better retrieval than naive top sample selection method. We also fuse BoW vector and CNN prediction in the retrieval system for a better performance. To evaluate the performance of our proposed method, experiments are conducted on Standard Oxford (5K and 105K) and Paris (6K) datasets, and experimental results and comparison with the state-of-the-art methods demonstrate the effectiveness of the proposed method.