Parallel Field Ranking

Research Context: Nearest Neighbor Search (NNS) is a fundamental problem in computing, with applications in recommendation systems, semantic search, information retrieval, and Retrieval-Augmented Generation (RAG) in Large Language Models (LLMs). High-dimensional datasets with millions of vectors make exact search computationally intractable, demanding efficient approximations. Practical Problem: Approximate Nearest Neighbor Search (ANNS) has become the standard approach, with graph-based methods standing out. However, structures such as Hierarchical Navigable Small World (HNSW) graphs impose high construction and update costs, which limits their scalability in real-world scenarios. Proposed Solution: This paper proposes the integration of Canopy Clustering as a divide-and-conquer heuristic to mitigate the computational burden of HNSW graph construction under cosine similarity, enabling more cost-effective ANNS pipelines in information retrieval systems. Related IS Theory: The research aligns with Information Systems theories on efficiency of knowledge retrieval, and the optimization of data-intensive processes in decision-support environments. It builds upon principles of data organization and access structures in IS, particularly the trade-offs between accuracy and computational feasibility in large-scale information retrieval. Research Method: Controlled experiments were conducted to evaluate the applicability of divide-and-conquer and Canopy Clustering prior to HNSW construction. Comparative analyses measured build time, query latency, and recall performance. Summary of Results: Preliminary findings suggest that Canopy Clustering can reduce construction costs while sustaining recall levels adequate for IS applications. Contributions and Impact to IS area: The work introduces a novel heuristic combination to enhance the scalability of graph-based similarity search indexing, providing methodological advances that may extend to other high-dimensional data processing tasks. It offers actionable insights for designing more efficient and viable intelligent information systems.

Semi-supervised ranking is a relatively new and important learning problem. Some semi-supervised learning to rank algorithms have been proposed by adapting existing pair-wise ranking algorithms into the semi-supervised setting. In contrast, there is comparatively little work in developing semi-supervised ranking algorithm based on the list-wise approach. In this paper, we proposed a semi-supervised extension of one of the state-of-the art list-wise ranking algorithm--AdaRank, for the task of learning to rank with partially labeled data. The proposed algorithm is a two-staged approach that combines the process of label propagation and a regularized version of AdaRank. The label propagation process is used to propagate the relevance labels from labeled instances to other unlabeled ones, so that more training data are available to learn the ranking function. The regularized version of AdaRank optimizes a novel performance measure, which incorporates both the usual Information Retrieval IR metrics and the smoothness of the ranking function. Experimental results on benchmark datasets show the advantage of our methods over existing pair-wise algorithms for the Information Retrieval measures.

Plithogenic cubic sets extend the cubic set framework into new mathematical territory that enables superior handling of decision-making and pattern recognition when dealing with uncertainty. Through Plithogenic logic researchers could develop advanced methods to represent complex patterns by supporting inconsistent and self-contradictory or combined features. This research paper explores the schematic explanation of Plithogenic cubic sets and their use in pattern recognition applications. The strong nature of pattern recognition emerge from Plithogenic logic's dimensionality flexibility when combined with a two-member structure in cubic sets while resolving imprecision and vagueness and the complexity of information aspects. Through theoretical foundations and illustrative examples the author presented their capability to employ Plithogenic cubic sets successfully in pattern recognition and classification applications. The schematic method delivers a complete grasp of Plithogenic cubic sets as an enabling instrument for pattern recognition research.

Retrieval of reliable relevant information is the major concern in medical information retrieval. Among all factors that affect the performance of retrieval system, ranking function is the most important factor. The retrieved documents from large document collection are arranged in the decreasing order of their relevance score by ranking function. A neuro-fuzzy system-based hybrid ranking function (HRF) is proposed in this paper. The proposed ranking function considers weight of document and query with respect to keyword as input features and gives relevance score between document and query as output. Experiments are performed on OHSUMED and PMC benchmark medical document corpus by using 15 experimental queries. The experimental results prove that the proposed HRF performs better when compared with fuzzy logic-based ranking function (FRF) and conventional statistical Euclidean distance-based ranking function (ERF) and cosine similarity-based ranking function (CRF) in terms of precision, recall and F-measure.

Queries describe the users' search intent and therefore they play an essential role in the context of ranking for information retrieval and Web search. However, most of existing approaches for ranking do not explicitly take into consideration the fact that queries vary significantly along several dimensions and entail different treatments regarding the ranking models. In this paper, we propose to incorporate query difference into ranking by introducing query-dependent loss functions. In the context of Web search, query difference is usually represented as different query categories; and, queries are usually classified according to search intent such as navigational, informational and transactional queries. Based on the observation that such kind of query categorization has high correlation with the user's different expectation on the result accuracy on different rank positions, we develop position-sensitive query-dependent loss functions exploring such kind of query categorization. Beyond the simple learning method that builds ranking functions with pre-defined query categorization, we further propose a new method that learns both ranking functions and query categorization simultaneously. We apply the query-dependent loss functions to two particular ranking algorithms, RankNet and ListMLE. Experimental results demonstrate that query-dependent loss functions can be exploited to significantly improve the accuracy of learned ranking functions. We also show that the ranking function jointly learned with query categorization can achieve better performance than that learned with pre-defined query categorization. Finally, we provide analysis and conduct additional experiments to gain deeper understanding on the advantages of ranking with query-dependent loss functions over other query-dependent ranking approaches and query-independent approaches.

A generic ranking function discovery framework by genetic programming for information retrieval

In this paper, we introduce “task trail” to understand user search behaviors. We define a task to be an atomic user information need, whereas a task trail represents all user activities within that particular task, such as query reformulations, URL clicks. Previously, web search logs have been studied mainly at session or query level where users may submit several queries within one task and handle several tasks within one session. Although previous studies have addressed the problem of task identification, little is known about the advantage of using task over session or query for search applications. In this paper, we conduct extensive analyses and comparisons to evaluate the effectiveness of task trails in several search applications: determining user satisfaction, predicting user search interests, and suggesting related queries. Experiments on large scale data sets of a commercial search engine show that: (1) Task trail performs better than session and query trails in determining user satisfaction; (2) Task trail increases webpage utilities of end users comparing to session and query trails; (3) Task trails are comparable to query trails but more sensitive than session trails in measuring different ranking functions; (4) Query terms from the same task are more topically consistent to each other than query terms from different tasks; (5) Query suggestion based on task trail is a good complement of query suggestions based on session trail and click-through bipartite. The findings in this paper verify the need of extracting task trails from web search logs and enhance applications in search and recommendation systems.

One crucial task of learning to rank in the field of information retrieval (IR) is to determine an ordering of documents according to their degree of relevance to the user given query. In this paper, a learning method is proposed named AdaGP-Rank by applying boosting techniques to genetic programming. This approach uses genetic programming to evolve ranking functions while a process inspired from AdaBoost technique helps the evolved ranking functions concentrate on the ranking of those documents associating those 'hard' queries. Based on the confidence coefficients, the ranking functions obtained at each boosting round are then combined into a final strong ranker. Experiments conform that AdaGP-Rank has general better performance than several state-of-the-art ranking algorithms on the benchmark data sets.

The diffusion of automated information retrieval (IR) ap plications and of applications which share many of the peculiarities of IR applications is increasing. The demand for these types of applications draws attention to the lack of methodologies available for IR data design. The data modell ing of an IR application, that is, the data modelling of the IR part of a complete information system, has not yet been approached and studied as a complete process and in a struc tured way. The first part of this paper introduces the motivations and the scope of the DIRD (Design of Information Retrieval Data) project, which is devoted to the development of an environ ment for the design of advanced applications of IR. The second part of the paper addresses the design of the IR part (or IR application) of an information system. The concep tual paradigm necessary for the design of advanced IR applica tions is investigated. The Entity Relationship (ER) approach is compared with that conceptual modelling paradigm and is examined as a candidate data model for the conceptual design of IR data. A new ER approach is then introduced: this new approach extends the constructs of the ER model to manage the complexity of IR data. Two design examples are given to present the use of the new ER approach in designing IR data.

This article presents a comprehensive and objective systematic review of existing research on recommendation systems with regards to core theory, latest studies, various applications, current attitudes, and potential future applications. The research is mainly based on exploring professional peer-reviewed studies and articles and using their abstracts to create a comprehensive and unbiased review of existing research. The following search terms were used to identify articles and studies for the research: recommendation systems; recommender systems; core theory of recommender systems; current attitudes towards recommendation systems; latest studies on recommendation systems; applications of recommendation systems; potential studies on recommendation systems; and future potential applications of recommendation systems. The research also used the advanced search filter to locate recent studies for comparison by limiting the search by year to find studies published from 2021 onwards. Most literature on this area highlights the importance of recommendation systems in almost all aspects of modern life. Specifically, recommendation systems have become critical components in business, health care, education, marketing, and social networking domains. Additionally, most studies identified reinforcement of learning and deep learning techniques as significant developments in the field. These techniques form the backbone of most modern recommendation systems. The primary concern that could hinder further evolution systems is their consequent filter bubble effects which many studies showed to be problematic. Healthcare is a central area that shows tremendous potential for these systems. Although recommender systems have been implemented in this domain, there remains a lot of untapped potential that, if unleashed, could revolutionize medicine and healthcare. But the problems facing these systems have to be tackled first to establish trust. Keywords: Recommendation systems, Recommender systems, Deep learning, Reinforcement learning DOI: 10.7176/CEIS/13-4-04 Publication date: August 31 st 2022

Low-rank matrix completion is a hot topic in the field of machine learning. It is widely used in image processing, recommendation systems and subspace clustering. However, the traditional method uses the nuclear norm to approximate the rank function, which leads to only the suboptimal solution. Inspired by the closed-form formulation of L_{2/3} regularization, we propose a new truncated schatten 2/3-norm to approximate the rank function. Our proposed regularizer takes full account of the prior rank information and achieves a more accurate approximation of the rank function. Based on this regularizer, we propose a new low-rank matrix completion model. Meanwhile, a fast and efficient algorithm are designed to solve the proposed model. In addition, a rigorous mathematical analysis of the convergence of the proposed algorithm is provided. Finally, the superiority of our proposed model and method is investigated on synthetic data and recommender system datasets. All results show that our proposed algorithm is able to achieve comparable recovery performance while being faster and more efficient than state-of-the-art methods.

Natural Language Processing (NLP) techniques are believed to hold the potential to assist bag-of-words Information Retrieval (IR) in terms of retrieval accuracy. In this paper, we report a natural language based IR approach where the common syntactic structures between documents and the query is regarded to as a query-dependent feature for documents. Specifically, a is proposed for query terms, which can be seen as a weight to model the degree of term's involvement in the common syntactic structures. This structural weight is used together with the TF-IDF weighting scheme, which results in a new ranking function. The accumulation of this structural weight of all the query terms in the new ranking function will be seen as a measure of how much a document and a query share the common syntactic structures. The experimental results show that by using this ranking function, significant improvements in the retrieval performance are achieved.

Ranking is likely the most important process of an information retrieval (IR) system that will be used to evaluate and measure the effectiveness of an IR system. This paper aims to produce the implementation of fuzzy logic controller of Mamdani-type fuzzy inference system for defining the ranking function by using the BM25 model in the Malay IR system that also includes the Malay Stemmer. The result of the ranking function then will be compared to the result of vector space model that is also applied in Malay IR system and be evaluated using relevant document by the hadith expert. The results showed that FBMIR has slightly outperformed vector space model on three topic set of query results such as 'Iman', 'Ilmu' and 'Wuduk' on the precision at rank 10 and the percentage of no relevant document in the top ten retrieved measures.

A vector field X on a Riemannian manifold determines a submanifold in the tangent bundle. The volume of X is the volume of this submanifold for the induced Sasaki metric. When M is compact, the volume is well defined and, usually, this functional is studied for unit fields. Parallel vector fields are trivial minima of this functional. For manifolds of dimension 5, we obtain an explicit result showing how the topology of a vector field with constant length influences its volume. We apply this result to the case of vector fields that define Riemannian foliations with all leaves compact.

Accurate ordering or ranking over instances is of paramount importance for several applications (Faria et al. Learning to rank for content-based image retrieval. In: Proceedings of the Multimedia Information Retrieval Conference, pp. 285–294, 2010; Veloso et al. Learning to rank at query-time using association rules. In: Proceedings of the Conference on Research and Development in Information Retrieval (SIGIR), pp. 267–274, 2008; Veloso et al. J Inf Data Manag 1(3): 567–582, 2010; Veloso and Meira, Efficient on-demand opinion mining. In: Proceedings of the Brazilian Symposium on Databases (SBBD), pp. 332–346, 2007; Veloso et al. Automatic moderation of comments in a large on-line journalistic environment. In: Proceedings of the International AAAI Conference on Weblogs and Social Media (ICWSM), pp. 234–237, AAAI, 2007).One clear application is Information Retrieval, where documents retrieved by search engines must be ranked according to the corresponding relevance to the query (Trotman, Inf Ret 8(3):359–381, 2005).Many features may affect the relevance of such documents, and, thus, it is difficult to adapt ranking functions manually. Recently, a body of empirical evidence has emerged suggesting that methods that automatically learn ranking functions offer substantial improvements in enough situations to be regarded as a relevant advance for applications that depend on ranking . Hence, learning ranking functions has attracted significant interest from the machine learning community. In the context of Information Retrieval, the conventional approach to this learning task is to assume the availability of examples (i.e., a training data, \({S},\) which typically consists of document features and the corresponding relevance to specific queries), from which a learning function can be learned. When a new query is given, the documents associated with this query are ranked according to the learned function (i.e., this function gives a score to a document indicating its relevance with regard to the query). In this chapter we present ranking algorithms based on demand-driven associative classification .