An Extended Vector Space Model for XML Information Retrieval

Not many XML information retrieval (IR) systems exist that allow easy addition of and switching between different IR models. Especially for the scientific environment where building a system takes a lot of time and keeps researchers away from the real work, i.e., investigating what is the most effective IR model, a platform that would provide this functionality would be ideal. For this reason we developed such an XML IR system. It is centered around a logical algebra, named score region algebra (SRA), that enables transparent specification of IR models for XML databases (see [1] for more details). The transparency is achieved by a possibility to instantiate various retrieval models, using abstract score functions within algebra operators, while logical query plan and operator definitions remain unchanged. Our algebra operators model three important aspects of XML IR: element relevance score computation, element score propagation, and element score combination. To implement a new IR model, one only needs to provide definitions for these abstract function classes. To illustrate the usefulness of our algebra our demo system supports several, well known IR scoring models (e.g., Language Models, Okapi, and tf.idf), combined with different score propagation and combination functions. The user can select which model to use at run time. Following good practice in database systems design, our prototype system has a typical three-layered architecture. (1) The conceptual layer takes a NEXI [3] query expression as input, e.g.,

The vector space model of information retrieval is one of the classical and widely applied retrieval models. Paradoxically, it has been characterized by a discrepancy between its formal framework and implementable form. The underlying concepts of the vector space model are mathematical terms: linear space, vector, and inner product. However, in the vector space model, the mathematical meaning of these concepts is not preserved. They are used as mere computational constructs or metaphors. Thus, the vector space model actually does not follow formally from the mathematical concepts on which it has been claimed to rest. This problem has been recognized for more than two decades, but no proper solution has emerged so far. The present article proposes a solution to this problem. First, the concept of retrieval is defined based on the mathematical measure theory. Then, retrieval is particularized using fuzzy set theory. As a result, the retrieval function is conceived as the cardinality of the intersection of two fuzzy sets. This view makes it possible to build a connection to linear spaces. It is shown that the classical and the generalized vector space models, as well as the latent semantic indexing model, gain a correct formal background with which they are consistent. At the same time it becomes clear that the inner product is not a necessary ingredient of the vector space model, and hence of Information Retrieval (IR). The Principle of Object Invariance is introduced to handle this situation. Moreover, this view makes it possible to consistently formulate new retrieval methods: in linear space with general basis, entropy‐based, and probability‐based. It is also shown that Information Retrieval may be viewed as integral calculus, and thus it gains a very compact and elegant mathematical way of writing. Also, Information Retrieval may thus be conceived as an application of mathematical measure theory.

Information retrieval has become an important field of study and research under computer science due to explosive growth of information available in the form of full text, hypertext, administrative text, directory, numeric or bibliographic text. The research work is going on various aspects of information retrieval systems so as to improve its efficiency and reliability. This paper presents a comprehensive study, which discusses not only emergence and evolution of information retrieval but also includes different information retrieval models and some important aspects such as document representation, similarity measure and query expansion. Keywords—Information Retrieval, query expansion, similarity measure, query expansion, vector space model.

Term mismatch is a common limitation of traditional information retrieval (IR) models where relevance scores are estimated based on exact matching of documents and queries. Typically, good IR model should consider distinct but semantically similar words in the matching process. In this paper, we propose a method to incorporate word embedding (WE) semantic similarities into existing probabilistic IR models for Arabic in order to deal with term mismatch. Experiments are performed on the standard Arabic TREC collection using three neural word embedding models. The results show that extending the existing IR models improves significantly baseline bag-of-words models. Although the proposed extensions significantly outperform their baseline bag-of-words, the difference between the evaluated neural word embedding models is not statistically significant. Moreover, the overall comparison results show that our extensions significantly improve the Arabic WordNet based semantic indexing approach and three recent WE-based IR language models.

A common limitation of many information retrieval (IR) models is that relevance scores are solely based on exact (i.e., syntactic) matching of words in queries and documents under the simple Bag-of-Words (BoW) representation. This not only leads to the well-known vocabulary mismatch problem, but also does not allow semantically related words to contribute to the relevance score. Recent advances in word embedding have shown that semantic representations for words can be efficiently learned by distributional models. A natural generalization is then to represent both queries and documents as Bag-of-Word-Embeddings (BoWE), which provides a better foundation for semantic matching than BoW. Based on this representation, we introduce a novel retrieval model by viewing the matching between queries and documents as a non-linear word transportation (NWT) problem. With this formulation, we define the capacity and profit of a transportation model designed for the IR task. We show that this transportation problem can be efficiently solved via pruning and indexing strategies. Experimental results on several representative benchmark datasets show that our model can outperform many state-of-the-art retrieval models as well as recently introduced word embedding-based models. We also conducted extensive experiments to analyze the effect of different settings on our semantic matching model.

We introduce a new information retrieval model, the generalized vector space model (GVSM). In the GVSM, Boolean-like queries can be transformed into vectors spanned by the atoms of the free Boolean algebra generated by the index terms. Documents can, therefore, be ranked with respect to a query as in the conventional vector space model. Most significantly, the GVSM provides a unified view of the Boolean retrieval and vector-processing systems.

The Lampung University repository system is a repository of data related to study, community service, and other scientific works, currently has 37242 documents accessible through repository.lppm.unila.ac.id. Despite the amount of data, its optimal use as an information retrieval remains unrealized, hindering the effective promotion of Lampung University's scientific publication excellence. Recognizing the limitations of existing information retrieval systems that are limited to specific methods for topic identification through clustering, this study aims to develop a retrieval system for Lampung University's repository using Vector Space Model (VSM), K-Means and Latent Semantic Analysis (LSA) that generates clusters and study expertise at the level of study program, faculty and Lampung University. The methodology includes data collection, preprocessing, modeling, evaluation and system deployment. The results show that the number of clusters obtained for the university level is 7 clusters, for the faculty level are 6, 7, 8 and 10 clusters, and for the program level are 3 to 5 clusters. In addition, the finding topic identification indicate that the expertise topics at Lampung University, which are agriculture, soil, education, plants, learning, society, Lampung. This study contributes to optimizing the information retrieval system, promoting academic excellence, and advancing the understanding of study expertise at Lampung University.

Ranking is likely the most important process of an information retrieval (IR) system that will be used to evaluate and measure the effectiveness of an IR system. This paper aims to produce the implementation of fuzzy logic controller of Mamdani-type fuzzy inference system for defining the ranking function by using the BM25 model in the Malay IR system that also includes the Malay Stemmer. The result of the ranking function then will be compared to the result of vector space model that is also applied in Malay IR system and be evaluated using relevant document by the hadith expert. The results showed that FBMIR has slightly outperformed vector space model on three topic set of query results such as 'Iman', 'Ilmu' and 'Wuduk' on the precision at rank 10 and the percentage of no relevant document in the top ten retrieved measures.

This study presents an overview of the research aimed at optimizing library information retrieval through the utilization of the Vector Space Model (VSM) method in a computer science context. Libraries, as publicly financed collections, provide extensive knowledge resources, eliminating the need for individual book purchases. However, the challenge lies in efficiently navigating the expanding library collections. To tackle this issue, the study employs information retrieval techniques, particularly the VSM method, which assesses term similarity by assigning weights to terms, enabling document and query representation as vectors. The relevance between documents and queries is measured through vector similarity. This approach, integrated with indexing, streamlines collection retrieval in libraries. Employing the Waterfall model for system development, the research outlines phases like analysis, design, coding, testing, and implementation. While effective, the model's rigidity in accommodating evolving requirements poses limitations. The VSM method's numerical representation of text documents facilitates precise similarity calculations, supported by TF-IDF values indicating term importance in documents relative to the corpus. The study further extends to system design using UML diagrams and a visitor interface, integrating VSM for efficient search functionality. Black-box testing confirms the robustness of the system components and interfaces. Overall, this research presents a systematic approach to enhance information retrieval in libraries, emphasizing the VSM's pivotal role in optimizing document searches within expansive collections.

We report data fusion experiments carried out on the four best‐performing retrieval models from TREC 5. Three were conceptually/algorithmically very different from one another; one was algorithmically similar to one of the former. The objective of the test was to observe the performance of the 11 logical data fusion combinations compared to the performance of the four individual models and their intermediate fusions when following the principle of polyrepresentation. This principle is based on cognitive IR perspective (Ingwersen & Järvelin, 2005) and implies that each retrieval model is regarded as a representation of a unique interpretation of information retrieval (IR). It predicts that only fusions of very different, but equally good, IR models may outperform each constituent as well as their intermediate fusions. Two kinds of experiments were carried out. One tested restricted fusions, which entails that only the inner disjoint overlap documents between fused models are ranked. The second set of experiments was based on traditional data fusion methods. The experiments involved the 30 TREC 5 topics that contain more than 44 relevant documents. In all tests, the Borda and CombSUM scoring methods were used. Performance was measured by precision and recall, with document cutoff values (DCVs) at 100 and 15 documents, respectively. Results show that restricted fusions made of two, three, or four cognitively/algorithmically very different retrieval models perform significantly better than do the individual models at DCV100. At DCV15, however, the results of polyrepresentative fusion were less predictable. The traditional fusion method based on polyrepresentation principles demonstrates a clear picture of performance at both DCV levels and verifies the polyrepresentation predictions for data fusion in IR. Data fusion improves retrieval performance over their constituent IR models only if the models all are quite conceptually/algorithmically dissimilar and equally and well performing, in that order of importance.

Collaboration is perceived as a powerful tool for companies to deal with an increasingly demanding global economic environment. Collaboration impact upon the logistics processes is analyzed in this chapter. Logistics naturally implies the concurrency of a set of companies being part of products and services value chain. However, collaborative logistics imposes new challenges that effectively faced by companies and government institutions will produce the expected return. Such challenges relate to effective decision making process involving logistics activities planning, scheduling, control, and coordination at different companies being part of the logistics network. These decision making processes require the effective capability to capture, process, and analyze information on processes, partners, environment, regulations, etc. That information is, of course, dynamic, and it is available in a scattered way all through a variety of sources, periodicity, and formats. Effective and efficient tools for capturing, classifying, processing, and report useful information for supporting the collaborative logistics decision making processes are required. Large amounts of digital text available on the web contain useful information for enabling collaborative logistics. The amount of digital text it is expected to increase significantly in the near future, making the development of data analysis applications an urgent need. Information gathering has been traditionally faced integrating systems and databases at the various institutions and companies participating in the logistics network. This approach has at least two difficulties that have not been solved properly: 1) It requires the data be available in structured databases and stored in terms of the same attributes, and 2) Organizations need to provide access to their systems for obtaining and processing the data. An alternative approach is using data available in the web as input to automatic text classifiers implemented with machine learning techniques. Automatic text classification (or categorization) is defined as the assignment of a Boolean value to each pair (dj,ci) belonging to the set D×C, where D is the domain of documents and C = {c1,...c|C|} is the set of predefined labels. Binary classification is the most simple and widely studied case, in which a document is classified into one of two mutually exclusive categories or classes. Document representation has a high impact on the task of classification. Some elements used for representing documents are: N-grams, single-word, phrases, or logical terms and statements. The vector space model is one of the most widely used models for ad-hoc information retrieval, mainly because of its conceptual simplicity and the appeal of the

A lot of work has been done on drawing word senses into retrieval to deal with the word sense ambiguity problem, but most of them achieved negative results. In this paper, we first implement a WSD system for nouns and verbs, then the language sense model (LSM) for information retrieval is proposed. The LSM combines the terms and senses of a document seamlessly through an EM algorithm. Retrieval on TREC collections shows that the LSM outperforms both the vector space model (BM25) and the traditional language model significantly for both medium and long queries (7.53%-16.90%). Based on the experiments, we can also empirically draw the conclusion that the fine-grained senses will improve the retrieval performance when they are properly used.

The volume of data in the world is growing very fast and generated from verity of sources like social media, sensors airline industry or scientific data in different formats. Biggest challenge is how to infer meaningful insights from such a varietyful and big data along with concern of data storage and management of fast growing data. The size of the databases used in today’s enterprises has been growing at exponential rates day by day. Hence, industries requirement to quickly process and analyze the big data volumes for business decision making and customer insights has also grown exponentially. Data pouring from various sources may be can be structured or unstructured in nature. Structured data refers to a relatively well-organized information, which can be further inserted into traditional RDBMS. As Traditional RDBMS are efficient and easy queries by simple, straightforward search algorithms or SQL queries. In contrast to structured data, unstructured data can be considered as information, which does not, comes in a pre-defined data format, well organized data storage model, or cannot be stored well into relational tables. It is assumed to be fastest growing type of data, e.g. image, sensors data, web chats, social networking messaging data, video, documents, log files, and email data. There are many techniques and software available, which can process and provide efficient storage of unstructured data and help organization to perform analytics on unstructured data. Unstructured data does not well-organized and not stored in predefined manner e.g. logs, web chats. The variety and on ordered nature of data makes storage methods and structure makes execution a time and resource-consuming affair. Advancement into technology has open floodgates to push huge volume of unstructured type of data. Multimedia data is one of the example of unstructured big data, which spans all over the Internet. This needs high execution capability to extract useful information. Rapid processing of multimedia data such as video is important for e.g. criminal investigations, surveillance monitoring, news analysis, sports analytics domain, emotion extraction, etc. Hence, analysis of multimedia data in minimum timeframe is one of the latest research areas. Therefore, we have researched techniques for analyzing unstructured data to extract meaningful information hidden in the big data. In addition, we will describe about various techniques and software used to Manage, process unstructured big data in efficient manner, and increases the performance of complexity analysis.

Purpose The purpose of this paper is to provide an integrated semantic information retrieval (IR) solution based on an ontology-improved vector space model for situations where a digital collection is established or curated. It aims to create a retrieval approach which could return the results by meanings rather than by keywords. Design/methodology/approach In this paper, the authors propose a semantic term frequency algorithm to create a semantic vector space model (SeVSM) based on ontology. To support the calculation, a multi-branches tree model is created to represent the ontology and a set of algorithms is developed to operate it. Then, a semantic ontology-based IR system based on the SeVSM model is designed and developed to verify the effectiveness of the proposed model. Findings The experimental study using 30 queries from 15 different domains confirms the effectiveness of the SeVSM and the usability of the proposed system. The results demonstrate that the proposed model and system can be a significant exploration to enhance IR in specific domains, such as a digital library and e-commerce. Originality/value This research not only creates a semantic retrieval model, but also provides the application approach via designing and developing a semantic retrieval system based on the model. Comparing with most of the current related research, the proposed research studies the whole process of realizing a semantic retrieval.

The Information Retrieval (IR) problem can be described as the quest to find the set of relevant information objects corresponding to a given information need, which is represented by a request. IR begins with a user who wishes to satisfy an information need. The information need is typically formulated in the form of a request, denoted by q. The intention is that the request q be as good as possible description of the information need N. The information being queried is modelled as a set of information carriers, or documents. An often used premise in IR is the following: if a given document d is about the request q, then there is a high likelihood that d is relevant with respect to the associated information need. Thus, the IR problem is reduced to deciding the aboutness relation between documents and requests. Many IR models have been developed, and there is a wide variation in how they determine aboutness. A common approach is to equate aboutness with overlap, meaning if there is sufficient overlap between the document and the query, the document is deemed to be about the query. The degree of overlap determined by the retrieval model is usually expressed as a real number between zero (no overlap) and unity (complete overlap). In most models, “sufficient” overlap is not prescribed by the model but by experimentation within a given IR setting. Such experimentation may lead to the conclusion, say, that sufficient overlap is apparent when the overlap measure is greater than α, 0 < α ≤ 1. The value α varies from setting to setting.