A Comparison of Deep Learning Based Query Expansion with Pseudo-Relevance Feedback and Mutual Information

We systematically investigate a new approach to estimating the parameters of language models for information retrieval, called parsimonious language models. Parsimonious language models explicitly address the relation between levels of language models that are typically used for smoothing. As such, they need fewer (non-zero) parameters to describe the data. We apply parsimonious models at three stages of the retrieval process: 1) at indexing time; 2) at search time; 3) at feedback time. Experimental results show that we are able to build models that are significantly smaller than standard models, but that still perform at least as well as the standard approaches.

Pseudo-relevance feedback (PRF) is a very effective query expansion approach, which reformulates queries by selecting expansion terms from top k pseudo-relevant documents. Although standard PRF models have been proven effective to deal with vocabulary mismatch between users’ queries and relevant documents, expansion terms are selected without considering their similarity to the original query terms. In this article, we propose a method to incorporate word embedding (WE) similarity into PRF models for Arabic information retrieval (IR). The main idea is to select expansion terms using their distribution in the set of top pseudo-relevant documents along with their similarity to the original query terms. Experiments are conducted on the standard Arabic TREC 2001/2002 collection using three neural WE models. The obtained results show that our PRF extensions significantly outperform their baseline PRF models. Moreover, they enhanced the baseline IR model by 22% and 68% for the mean average precision (MAP) and the robustness index (RI), respectively.

Query Expansion using Pseudo Relevance Feedback is a useful and a popular technique for reformulating the query. In our proposed query expansion method, we assume that relevant information can be found within a document near the central idea. The document is normally divided into sections, paragraphs and lines. The proposed method tries to extract keywords that are closer to the central theme of the document. The expansion terms are obtained by equi-frequency partition of the documents obtained from pseudo relevance feedback and by using tf-idf scores. The idf factor is calculated for number of partitions in documents. The group of words for query expansion is selected using the following approaches: the highest score, average score and a group of words that has maximum number of keywords. As each query behaved differently for different methods, the effect of these methods in selecting the words for query expansion is investigated. From this initial study, we extend the experiment to develop a rule-based statistical model that automatically selects the best group of words incorporating the tf-idf scoring and the 3 approaches explained here, in the future. The experiments were performed on FIRE 2011 Adhoc Hindi and English test collections on 50 queries each, using Terrier as retrieval engine.

The language modeling approach centers on the issue of estimating an accurate model by choosing appropriate language models as well as smoothing techniques. In the thesis, we propose a novel context-sensitive semantic smoothing method referred to as a topic signature language model. It extracts explicit topic signatures from a document and then statistically maps them into individual words in the vocabulary. In order to support the new language model, we developed two automated algorithms to extract multiword phrases and ontological concepts, respectively, and an EM-based algorithm to learn semantic mapping knowledge from co-occurrence data. The topic signature language model is applied to three applications: information retrieval, text classification, and text clustering. The evaluations on news collection and biomedical literature prove the effectiveness of the topic signature language model. In the experiment of information retrieval, the topic signature language model consistently outperforms the baseline two-stage language model as well as the context-insensitive semantic smoothing method in all configurations. It also beats the state-of-the-art Okapi models in all configurations. In the experiment of text classification, when the size of training documents is small, the Bayesian classifier with semantic smoothing not only outperforms the classifiers with background smoothing and Laplace smoothing, but it also beats the active learning classifiers and SVM classifiers. On the task of clustering, whether or not the dataset to cluster is small, the model-based k-means with semantic smoothing performs significantly better than both the model-based k-means with background smoothing and Laplace smoothing. It is also superior to the spherical k-means in terms of effectiveness. In addition, we empirically prove that, within the framework of topic signature language models, the semantic knowledge learned from one collection could be effectively applied to other collections. In the thesis, we also compare three types of topic signatures (i.e., words, multiword phrases, and ontological concepts), with respect to their effectiveness and efficiency for semantic smoothing. In general, it is more expensive to extract multiword phrases and ontological concepts than individual words, but semantic mapping based on multiword phrases and ontological concepts are more effective in handling data sparsity than on individual words.

Although many variants of language models have been proposed for information retrieval, there are two related retrieval heuristics remaining external to the language modeling approach: (1) proximity heuristic which rewards a document where the matched query terms occur close to each other; (2) passage retrieval which scores a document mainly based on the best matching passage. Existing studies have only attempted to use a standard language model as a black box to implement these heuristics, making it hard to optimize the combination parameters. In this paper, we propose a novel positional language model (PLM) which implements both heuristics in a unified language model. The key idea is to define a language model for each position of a document, and score a document based on the scores of its PLMs. The PLM is estimated based on propagated counts of words within a document through a proximity-based density function, which both captures proximity heuristics and achieves an effect of soft passage retrieval. We propose and study several representative density functions and several different PLM-based document ranking strategies. Experiment results on standard TREC test collections show that the PLM is effective for passage retrieval and performs better than a state-of-the-art proximity-based retrieval model.

Vocabulary mismatch in Information retrieval can be solved by Query Expansion (QE) techniques. Relevance feedback is a prominent solution to improve recall of retrieval system. Sometimes user may be reluctant and novice in providing feedback to improve the retrieval performance. Pseudo Relevance Feedback (PRF) automates the process. PRF treats top ranked resultant items are relevant and uses them to expand the query, which is not always correct. PRF by local analysis does not give guarantee to feedback positive terms to the system. Use of global analysis to capture the positive feedback is a regular practice in information retrieval process. This paper addresses the limitations of local analysis and global analysis alone by a novel approach that integrates both PRF and WordNet to select good expansion terms. The proposed solution filters the expansion terms and optimizes the expanded query. The proposed work is carried out on a huge Telugu text corpus collected from Wikipedia and other Telugu daily news portals.

Statistical language modeling (LM) that purports to quantify the acceptability of a given piece of text has long been an interesting yet challenging research area. In particular, language modeling for information retrieval (IR) has enjoyed remarkable empirical success; one emerging stream of the LM approach for IR is to employ the pseudo-relevance feedback process to enhance the representation of an input query so as to improve retrieval effectiveness. This paper presents a continuation of such a general line of research and the main contribution is threefold. First, we propose a principled framework which can unify the relationships among several widely-used query modeling formulations. Second, on top of the successfully developed framework, we propose an extended query modeling formulation by incorporating critical query-specific information cues to guide the model estimation. Third, we further adopt and formalize such a framework to the speech recognition and summarization tasks. A series of empirical experiments reveal the feasibility of such an LM framework and the performance merits of the deduced models on these two tasks.

The techniques of information retrieval eliminate the difficulty of finding specific information in electronic medical records (EMR), and the methods of query expansion (QE) improve the recall of EMR retrieval. However, most existing QE methods of EMR retrieval can’t get high-quality expansion terms and corresponding weights for Chinese EMR retrieval because of low-quality sources and improper calculation methods. In this paper, we propose a novel algorithm of expansion term selection and weight assignment for QE of Chinese EMR retrieval based on the clinical needs and unique characteristics of Chinese medical terms. The algorithm first selects expansion terms from a high quality Chinese medical knowledge graph and standard medical term sets, which can ensure the quality of expansion terms. Then it assigns weights to the selected expansion terms based on semantic similarities and manually designed expansion categories that reflect the opinions of medical experts. Experiment results show that our algorithm gets higher-quality expansion terms and more rational weights compared with four benchmark algorithms, using Precision at 10, Recall, Mean Average Precision and Binary Preference-based Measure as evaluation metrics, which implies that our algorithm can significantly improve the effectiveness of QE.

Cyberbullying has been a significant issue in online communities, posing threats to individuals' mental and social health. This paper examines the use of deep learning and natural language processing (NLP) methods for automated cyberbullying detection in text data. A complete preprocessing pipeline is applied, involving URL and mention removal, stop-word removal, and text normalization, to improve data quality for model training. For the detection issue, various architectures of deep learning are used, i.e., Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), and hybrid models like CNN-LSTM and CNN-GRU. Tokenization and padding on the dataset ensure equal input length before training on the model. Evaluation on performance is done on accuracy, precision, recall, and F1-score with comparative analysis on all models. Experimental evidence reveals that hybrid models are more accurate than independent CNN and RNN architectures, with better classification accuracy and generalization. This highlights the potential of combining deep learning and NLP methods in building automated systems for cyberbullying identification. This work's findings can inform the development of AI-powered content moderation tools, leading to safer, more inclusive, and robust digital spaces.

Microblog retrieval has received much attention in recent years due to the wide spread of social microblogging platforms such as Twitter. Many research studies investigated different approaches for microblog retrieval. Query expansion is one of the approaches that showed stable performance for improving microblog retrieval effectiveness. In this paper we comprehensively investigate query expansion for microblog retrieval using pseudo relevance feedback (PRF) with different configurations. In addition, we propose a hyperlink-extended PRF by expanding queries using embedded hyperlinks in retrieved microblogs. Our experimental results on TREC microblog data showed that PRF alone could outperform many retrieval approaches if configured properly. Results also showed that utilizing hyperlinked documents in tweets for query expansion improves results over traditional PRF. Our best results achieved using hyperlink-extended PRF outperformed most of the reported results using other approaches.

Query Expansion (QE) is widely applied to improve the retrieval performance of ad-hoc search, using different techniques and several data sources to find expansion terms. In Information Retrieval literature, selecting expansion terms remains a challenging task that relies on the extraction of term relationships. In this paper, we propose a new learning to rank-based query expansion model. The main idea behind is that, given a query and the set of its related ARs, our model ranks these ARs according to their relevance score regarding to this query and then selects the most suitable ones to be used in the QE process. Experiments are conducted on three test collections, namely: CLEF2003, TREC-Robust and TREC-Microblog, including long, hard and short queries. Results showed that the retrieval performance can be significantly improved when the ARs ranking method is used compared to other state of the art expansion models, especially for hard and long queries.

In this paper we study cross-language information retrieval using a bilingual topic model trained on comparable corpora such as Wikipedia articles. The bilingual Latent Dirichlet Allocation model (BiLDA) creates an interlingual representation, which can be used as a translation resource in many different multilingual settings as comparable corpora are available for many language pairs. The probabilistic interlingual representation is incorporated in a statistical language model for information retrieval. Experiments performed on the English and Dutch test datasets of the CLEF 2001-2003 CLIR campaigns show the competitive performance of our approach compared to cross-language retrieval methods that rely on pre-existing translation dictionaries that are hand-built or constructed based on parallel corpora.

The optimal settings of retrieval parameters often depend on both the document collection and the query, and are usually found through empirical tuning. In this paper, we propose a family of two-stage language models for information retrieval that explicitly captures the different influences of the query and document collection on the optimal settings of retrieval parameters. As a special case, we present a two-stage smoothing method that allows us to estimate the smoothing parameters completely automatically. In the first stage, the document language model is smoothed using a Dirichlet prior with the collection language model as the reference model. In the second stage, the smoothed document language model is further interpolated with a query background language model. We propose a leave-one-out method for estimating the Dirichlet parameter of the first stage, and the use of document mixture models for estimating the interpolation parameter of the second stage. Evaluation on five different databases and four types of queries indicates that the two-stage smoothing method with the proposed parameter estimation methods consistently gives retrieval performance that is close to---or better than---the best results achieved using a single smoothing method and exhaustive parameter search on the test data.

To support the practice of evidence-based medicine (EBM), clinically relevant and scientifically sound articles should be easily accessible. Due to the huge volume of medical literature and the low performance of present retrieval models, clinicians could only get relevant documents in the order of publication time. This study propose a new clinical task-specific retrieval technique that improves retrieval accuracy by exploiting clinical task-specific EBM terms to query expansion using co-occurrence analysis technique. The idea is aimed at selecting query expansion terms that are relevant to a specific clinical-task using task-specific EBM terms. Focusing on treatment and diagnosis tasks, the new method which was performed on the OHSUMED collection showed a further improved result than the existing method.

In microblog retrieval, query expansion can be essential to obtain good search results due to the short size of queries and posts. Since information in microblogs is highly dynamic, an up-to-date index coupled with pseudo-relevance feedback (PRF) with an external corpus has a higher chance of retrieving more relevant documents and improving ranking. In this paper, we focus on the research question:how can we reduce the query expansion computational cost while maintaining the same retrieval precision as standard PRF? Therefore, we propose to accelerate the query expansion step of pseudo-relevance feedback. The hypothesis is that using an expansion corpus organized into verticals for expanding the query, will lead to a more efficient query expansion process and improved retrieval effectiveness. Thus, the proposed query expansion method uses a distributed search architecture and resource selection algorithms to provide an efficient query expansion process. Experiments on the TREC Microblog datasets show that the proposed approach can match or outperform standard PRF in MAP and NDCG@30, with a computational cost that is three orders of magnitude lower.