Optimizing Approximate Nearest Neighbour-Based Dense Retrieval for Efficient Passage Ranking with Query Expansion

An effective exploitation of distributed computing environments to improve the performance of large advanced database applications requires parallel query processing techniques. This is because parallel query execution can be achieved by operations of pipelining and data partitioning, as well as independent subqueries. In order to develop efficient parallel query services in distributed environments, we present a query optimization, a dynamic parallel query scheduling algorithm, and an implementation mechanism for querying distributed objects. Our proposal has four important features. First, the query schedule dynamically schedules independent subqueries in parallel. Second, distributed processors are allowed to execute independent subqueries in random. Third, it doesn't limit the number of processors to deal with parallel query processing in a network based computing environment. Fourth, it provides simple operations to synchronize parallel query programs. We give the query optimization for parallel processing, show an algorithm of dynamic parallel query scheduling, describe an implementation method for the parallel query processing in a network-based distributed computing environment, and present the design of communication and synchronization for the parallel query processing.

The effects of query structures and query expansion (QE) on retrieval performance were tested with a best match retrieval system (InQuery^1). Query structure means the use of operators to express the relations between search keys. Six different structures were tested, representing strong structures (e.g., queries with facets or concepts identified) and weak structures (no concepts identified, a query is ‘a bag of search keys’). QE was based on concepts, which were first selected from a searching thesaurus, and then expanded by semantic relationships given in the thesaurus. The expansion levels were (a) no expansion, (b) a synonym expansion, (c) a narrower concept expansion, (d) an associative concept expansion, and (e) a cumulative expansion of all other expansions. With weak structures and Boolean structured queries, QE was not very effective. The best performance was achieved with a combination of a facet structure, where search keys within a facet were treated as instances of one search key (the SYN operator), and the largest expansion.

This paper presents a framework which combines data and text retrieval techniques to exercise and evaluate ontology based query expansions. We prepare by using linguistic techniques to identify query and document concepts, locating them in a ontologically defined semantic space. Expansions originate from the identified query concepts, with success determined by matching in the relevant document set. We identify three orthogonal dimensions that can affect query expansion success; relationship source, success measure technique, and query expansion technique. Expansion technique is further divided into six different categories: simple pruning, complex probability, voting, directional, semantic propagation, and multiple source concept. We describe each technique and show examples where they would be useful. The system architecture used facilitates plugging in of various expansion and evaluation routines, and flowing results from one method to the next. The system is useful for microanalysis of query expansion, discovering which components of ontological derived knowledge most influence query expansion success. In this work, we apply our framework to the medical domain.

Hundreds of millions of users each day use web search engines to meet their information needs. Advances in web search effectiveness are therefore perhaps the most significant public outcomes of IR research. Query expansion is one such method for improving the effectiveness of ranked retrieval by adding additional terms to a query. In previous approaches to query expansion, the additional terms are selected from highly ranked documents returned from an initial retrieval run. We propose a new method of obtaining expansion terms, based on selecting terms from past user queries that are associated with documents in the collection. Our scheme is effective for query expansion for web retrieval: our results show relative improvements over unexpanded full text retrieval of 26%--29%, and 18%--20% over an optimised, conventional expansion approach.

Unlike other areas of Exadata, parallel query execution uses the same Oracle Release 11.2 functionality as non-Exadata systems. Because Exadata is, at its heart, a data-warehousing system, and efficient handling and processing of data-warehouse workloads was a primary design goal, parallel query processing is an important feature. Remember that Smart Scans and offloading rely on direct-path reads, the same read functionality that is used by parallel query slaves. In Release 11.2, Oracle has provided three improvements to the earlier attempts at controlling parallel execution. These improvements, along with the processing power of Exadata, make this feature more manageable, more scalable, and less likely to saturate server resources, such as memory and CPU, than earlier releases of the database. The first of those improvements that we will discuss is parallel statement queuing.

The key element of many query processing tasks can be formalized as calculation of similarities between queries. These include query suggestion, query reformulation, and query expansion. Although many methods have been proposed for query similarity calculation, they could perform poorly on rare queries. As far as we know, there was no previous work particularly about rare query similarity calculation, and this paper tries to study this problem. Specifically, we address three problems. Firstly, we define an n-gram space to represent queries with their own content and a similarity function to measure the similarities between queries. Secondly, we propose learning the similarity function by leveraging the training data derived from user behavior data. This is formalized as an optimization problem and a metric learning approach is employed to solve it efficiently. Finally, we exploit locality sensitive hashing for efficient retrieval of similar queries from a large query repository. We experimentally verified the effectiveness of the proposed approach by showing that our method can indeed enhance the accuracy of query similarity calculation for rare queries and efficiently retrieve similar queries. As an application, we also experimentally demonstrated that the similar queries found by our method can significantly improve search relevance.

Contextualized query expansion via unsupervised chunk selection for text retrieval

Systems for retrieving and managing content-based medical images are becoming more important, especially as medical imaging technology advances and the medical image database grows. In addition, these systems can also use medical images to better grasp and gain a deeper understanding of the causes and treatments of different diseases, not just for diagnostic purposes. For achieving all these purposes, there is a critical need for an efficient and accurate content-based medical image retrieval (CBMIR) method. This paper proposes an efficient method (RbQE) for the retrieval of computed tomography (CT) and magnetic resonance (MR) images. RbQE is based on expanding the features of querying and exploiting the pre-trained learning models AlexNet and VGG-19 to extract compact, deep, and high-level features from medical images. There are two searching procedures in RbQE: a rapid search and a final search. In the rapid search, the original query is expanded by retrieving the top-ranked images from each class and is used to reformulate the query by calculating the mean values for deep features of the top-ranked images, resulting in a new query for each class. In the final search, the new query that is most similar to the original query will be used for retrieval from the database. The performance of the proposed method has been compared to state-of-the-art methods on four publicly available standard databases, namely, TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI. Experimental results show that the proposed method exceeds the compared methods by 0.84%, 4.86%, 1.24%, and 14.34% in average retrieval precision (ARP) for the TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI databases, respectively.

With the explosive growth of big data, data-intensive analysis services need a large data storage and high efficient parallel query processing techniques in cloud using lower-end machines, because most organizations adopt inexpensive low-end clusters. A key challenge is to optimize query processing in such a cloud environment. In this paper, we present a new hybrid data access architecture, called HyDB, for providing data-intensive analysis services, which is featured with a distributed data storage, parallel data access, and query optimization methods. First, we propose a new data partitioning method based on both workloads and co-located resources. The data partitioning method achieves higher consolidation and outperforms the existing approaches. Second, we provide a new parallel access method which includes parallel query processing, optimal query plan generation, and optimal path selection by using a plan tree pruning technique. We have implemented HyDB. Finally, we conduct extensive experimental studies and confirm the efficiency of our HyDB.

This paper summarises the results of ITC-irst in the Cross-Language Spoken Document Retrieval track at the Cross Language Evaluation Forum 2003. The target collection consisted of automatic transcriptions of American Broadcast News manually segmented into stories. Topics consisted of 50 short queries in English, for which human-made translations into several languages are available: Dutch, French, German, Italian, and Spanish. Experiments were carried out by applying a bilingual document retrieval system which has been slightly modified in the query expansion module. Significant performance improvements were achieved by applying query expansion both on the target collection and on a larger corpus of written news.

This paper presents a method for resolution of ambiguity of place names and its evaluation over the Geological Corpus. We discussed the methods for generating co-occurrence models (co-occurrence matrix and cooccurrence graph) for the purpose of place name disambiguation. A graph based disambiguation algorithm is introduced to resolve the ambiguities such as synonyms and polynyms in place names. Using the neighborhoods in the cooccurrence graph, the query can be expanded with related place names after disambiguation. Experimental results on a Geological corpus (IITKGPGEOCORP) show that query expansion with place disambiguation outperforms the baseline method.

Our objective is to enhance the effectiveness and efficiency of ad hoc and routing retrieval for large scale textbases. Effective retrieval means ranking relevant answer documents of a user's information need high on the output list. Our text processing and retrieval system PIRCS (Probabilisitc Indexing and Retrieval -Components- System) handles English text in a domain independent fashion, and is based on a probabilistic model but extended with the concept of document components as discussed in our last year's site report. Our focus for enhancing effectiveness remains on three areas: 1) improvements on document representation; 2) combination of retrieval algorithms; and 3) network implementation with learning capabilities. Using representation with more restricted contexts such as phrases or subdocument units help to decrease ambiguity in both retrieval and learning. Combining evidences from different retrieval algorithms is known to improve results. Viewing retrieval in a network helps to implement query-focused and document-focused retrieval and feedback, as well as query expansion.

How to build up an efficient simulation tool for complex parallel relational query processing based on High-level Petri nets

A semantic framework for enhancing pseudo-relevance feedback with soft negative sampling and contrastive learning

An evaluation of query expansion by the addition of clustered terms for a document retrieval system