A Survey of Eigenvector Methods for Web Information Retrieval

The purpose of the Information Retrieval (IR) is to find a set of documents that are relevant for a specific information need of a user. Traditional Information Retrieval model commonly used in commercial search engine is based on keyword indexing system and Boolean logic queries. One big drawback of traditional information retrieval is that they typically retrieve information without an explicitly defined domain of interest to the users so that a lot of no relevance information returns to users, which burden the user to pick up useful answer from these no relevance results. In order to tackle this issue, many semantic web information retrieval models have been proposed recently. The main advantage of Semantic Web is to enhance search mechanisms with the use of Ontology’s mechanisms. In this paper, we present our approach to personalize web search engine based on ontology. In addition, key techniques are also discussed in our paper. Compared to previous research, our works concentrate on the semantic similarity and the whole process including query submission and information annotation.

This chapter aims to master web mining and Information Retrieval (IR) in the digital age, thus describing the overviews of web mining and web usage mining; the significance of web mining in the digital age; the overview of IR; the concept of Collaborative Information Retrieval (CIR); the evaluation of IR systems; and the significance of IR in the digital age. Web mining can contribute to the increase in profits by selling more products and by minimizing costs. Web mining is the application of data mining techniques to discover the interesting patterns from web data in order to better serve the needs of web-based multifaceted applications. Mining web data can improve the personalization, create the selling opportunities, and lead to more profitable relationships with customers in global business. Web mining techniques can be applied with the effective analysis of the clearly understood business needs and requirements. Web mining builds the detailed customer profiles based on the transactional data. Web mining is used to create the personalized search engines which can recognize the individuals' search queries by analyzing and profiling the web user's search behavior. IR is the process of obtaining relevant information from a collection of informational resources. IR has considerably changed with the expansion of the Internet and the advent of modern and inexpensive graphical user interfaces and mass storage devices. The effective IR system, including an active indexing system, not only decreases the chances that information will be misfiled but also expedites the retrieval of information. Regarding IR utilization, the resulting time-saving benefit increases office efficiency and productivity while decreasing stress and anxiety. Most IR systems provide the advanced searching capabilities that allow users to create the sophisticated queries. The chapter argues that applying web mining and IR has the potential to enhance organizational performance and reach strategic goals in the digital age.

Most research oriented search queries consist of multiple topics belonging to one or more domains of knowledge. The objective of such search queries is to find the relationship or impact that each topic has on the other(s). Though web search engines provide easy means to retrieve information off the web, search engines are mainly key word oriented and do not consider the different topics and relationships between such topics present in the results. This project presents a software solution, `SearchDroid' that acts as an intermediate user between a search engine and the end user, refining search results based on different topics identified in the search query and their presence and relationships depicted in search results. A discourse parsing approach has been used to build discourse structures that represent the rhetorical relations of text in search results; this is used to re-rank results based on topics identified in the search query. The project combines linguistics research under discourse parsing with web information retrieval techniques. The lack of literature combining discourse parsing techniques with web information retrieval has been compensated for by introducing a fresh algorithmic approach. An abstract information retrieval mechanism has been created with the use of discourse parsing techniques, and can be integrated into any web information retrieval approach. The system was evaluated by Linguistic Experts, Technical Experts and End users. All experts agreed that a discourse parsing approach was suitable for addressing the problem at hand and that the project had high research value in the aspect of linking linguistics research with web information retrieval research. Hands-on testing of the system by End Users produced high user acceptance of the proposed system.

The Web has become a large repository of documents (or pages) written in many different languages. In this context, traditional information retrieval (IR) techniques cannot be used whenever the user query and the documents being retrieved are in different languages. To address this problem, new cross‐language information retrieval (CLIR) techniques have been proposed. In this work, we describe a method for cross‐language retrieval of medical information. This method combines query terms and related medical concepts obtained automatically through a categorization procedure. The medical concepts are used to create a linguistic abstraction that allows retrieval of information in a language‐independent way, minimizing linguistic problems such as polysemy. To evaluate our method, we carried out experiments using the OHSUMED test collection, whose documents are written in English, with queries expressed in Portuguese, Spanish, and French. The results indicate that our cross‐language retrieval method is as effective as a standard vector space model algorithm operating on queries and documents in the same language. Further, our results are better than previous results in the literature.

Web collection is essential for many Web based researches such as Web Information Retrieval (IR), Web data mining, Corpus linguistics and so on. However, it is usually expensive and time-consuming to collect a large scale of Web pages in lab-based environment and public-available collection becomes a necessity for these researches. In this study, we present a Chinese Web collection, SogouT-16, which is the largest free-of-charge public Chinese Web collection so far. We provide a variety of descriptive characteristics of SogouT-16 and discuss its adoption in a newly-designed ad-hoc retrieval task in NTCIR-13, We Want Web. SogouT-16 also provides online retrieval service and contains a number of auxiliary resources including hyperlink structure graph, query logs, word embedding, and etc. We believe that SogouT-16 will provide new opportunities for novel investigations and applications in IR and other related communities.

AI Voice Assistant Abstract This project aims to develop an AI-based voice assistant capable of understanding andexecuting user commands through natural language processing. The assistant facilitates a hands-free user experience by providing functionalities like web search, weather updates, system operations, and more. The system is built to be interactive, responsive, and user-friendly, supporting real-time voice input and feedback. Introduction An AI Voice Assistant is an intelligent software agent that can interpret voice commands and perform tasks or services for an individual. With the increasing demand for hands-free control, voice assistants have become integral in personal devices, smart homes, and industries. This project is a step towards building a personal assistant using AI/ML and NLP techniques. Objective - To develop a voice- controlled AI assistant that understands and processes human speech. - To implement basic functionalities such as time query, weather forecast, search operations, and system commands. - To provide real-time feedback and ensure a seamless user experience. Functional Modules 1. Speech Recognition Module: Captures and converts voice to text. 2. NLP Module: Understands and interprets commands. 3. Task Execution Module: Executes appropriate system or web-based tasks. 4. Text-to-Speech Module: Converts responses to voice output. AI Voice Assistant System Features - Voice command input - Real-time interaction - Web search and information retrieval - System automation (e.g., open files, shutdown) - Personalized response generation Tools and Technologies - Programming Language: Python - Libraries: SpeechRecognition, pyttsx3, pyaudio, nltk, webbrowser - Platform: Windows/Linux - IDE: VS Code or PyCharm Benefits - Hands- free operation enhances convenience - Time-saving for repetitive tasks - Useful for physically challenged individuals - Can be integrated with IoT and smart devices Conclusion The AI Voice Assistant demonstrates the power of natural language processing and machine learning in creating interactive and intelligent systems. It can be further enhanced by integrating with IoT devices and expanding command sets, making it suitable for a wide range of applications.

Understanding what kinds of Web pages are the most useful for Web search engine users is a critical task in Web information retrieval (IR). Most previous works used hyperlink analysis algorithms to solve this problem. However, little research has been focused on query‐independent Web data cleansing for Web IR. In this paper, we first provide analysis of the differences between retrieval target pages and ordinary ones based on more than 30 million Web pages obtained from both the Text Retrieval Conference (TREC) and a widely used Chinese search engine, SOGOU (www.sogou.com). We further propose a learning‐based data cleansing algorithm for reducing Web pages that are unlikely to be useful for user requests. We found that there exists a large proportion of low‐quality Web pages in both the English and the Chinese Web page corpus, and retrieval target pages can be identified using query‐independent features and cleansing algorithms. The experimental results showed that our algorithm is effective in reducing a large portion of Web pages with a small loss in retrieval target pages. It makes it possible for Web IR tools to meet a large fraction of users' needs with only a small part of pages on the Web. These results may help Web search engines make better use of their limited storage and computation resources to improve search performance.

Web search results enjoy an increasing importance in our daily lives. But what can be said about their quality, especially when querying a controversial issue? The traditional information retrieval metrics of precision and recall do not provide much insight in the case of web information retrieval. In this paper we examine new ways of evaluating quality in search results: coverage and independence. We give examples on how these new metrics can be calculated and what their values reveal regarding the two major search engines, Google and Yahoo. We have found evidence of low coverage for commercial and medical controversial queries, and high coverage for a political query that is highly contested. Given the fact that search engines are unwilling to tune their search results manually, except in a few cases that have become the source of bad publicity, low coverage and independence reveal the efforts of dedicated groups to manipulate the search results.

ABSTRACTThis article presents an algorithm to determine the relevancy of hanging pages in the link-structure-based ranking algorithms. Recently, hanging pages have become major obstacles in web information retrieval (IR). As an increasing number of meaningful hanging pages appear on the Web, their relevancy has to be determined according to the query term in order to make the search engine result pages fair and relevant. Exclusion of these pages in ranking calculation can give biased/inconsistent results, but inclusion of these pages will reduce the speed significantly. Most of the IR ranking algorithms exclude the hanging pages. But there are relevant and important hanging pages on the Web, and they cannot be ignored. In our proposed method, we use anchor text to determine the hanging relevancy, and we use stability analysis to show that rank results are consistent before and after altering the link structure.

Keyword information retrieval is the mainstream way of information retrieval at present. Users need to scan a large amount of text information in the search results to find the information they want, but the process causes inefficiency and poor user experience. In fact, images in the web pages can provide a direct-viewing and fast retrieval way. Through the research on the features of web subject-related image and achieve its automatic identification and extraction, we can display it in the thumbnail together with the page title and summary in the results pages. It can help users filter and browse information in a more convenient way. This paper establishes a web image attribute system from both HTML attributes and external attributes. Then through corresponding automatic extracting algorithms and data analysis it succeeds to gain 16 feature rules of web subject-related image, and finishes building a web subject-related image feature model. The model proves to obtain more than 99% of the extraction rate and filtering rate while applying to the sample data, demonstrating its value in web information retrieval.

The purpose of this research is to discuss some challenges of information retrieval, especially Web information retrieval, in digital ecosystems from a userpsilas perspective. As a dominant search tool, search engines usually return millions of search results in a long flat list in which many or even most of the results can be irrelevant. The long flat list conveys nothing about knowledge structure related to the retrieved results and personal search preferences and interests are not explored. Although some search engines try to cluster the Web results, the automatically formed titles and knowledge hierarchy is prone to mismatching the searcherpsilas human mental model. In digital ecosystems, while many different search tools are available, they are not integrated. To address these issues, a search framework which combines categorization, clustering, ontology, and personalization is proposed, and thus the quality of search results in digital ecosystems is expected to be boosted.

The queries people issue to a search engine and the results clicked following a query change over time. For example, after the earthquake in Japan in March 2011, the query japan spiked in popularity and people issuing the query were more likely to click government-related results than they would prior to the earthquake. We explore the modeling and prediction of such temporal patterns in Web search behavior. We develop a temporal modeling framework adapted from physics and signal processing and harness it to predict temporal patterns in search behavior using smoothing, trends, periodicities, and surprises. Using current and past behavioral data, we develop a learning procedure that can be used to construct models of users' Web search activities. We also develop a novel methodology that learns to select the best prediction model from a family of predictive models for a given query or a class of queries. Experimental results indicate that the predictive models significantly outperform baseline models that weight historical evidence the same for all queries. We present two applications where new methods introduced for the temporal modeling of user behavior significantly improve upon the state of the art. Finally, we discuss opportunities for using models of temporal dynamics to enhance other areas of Web search and information retrieval.

Humans have used electronic information retrieval (IR) systems for more than 50 years as they evolved from experimental systems to full-scale Web search engines and digital libraries. The fields of library and information science (LIS), cognitive science, human factors and computer science have historically been the leading disciplines in conducting research that seeks to model human interaction with IR systems for all kinds of information related behaviors. As technology problems have been mastered, the theoretical and applied framework for studying human interaction with IR systems has evolved from systems-centered to more user-centered, or cognitive-centered approaches. However, cognitive information retrieval (CIR) research that focuses on user interaction with IR systems is still largely under-funded and is often not included at computing and systems design oriented conferences. But CIR-focused research continues, and there are signs that some IR systems designers in academia and the Web search business are realizing that user behavior research can provide valuable insights into systems design and evaluation. The goal of our book is to provide an overview of new CIR research directions. This book does not provide a history of the research field of CIR. Instead, the book confronts new ways of looking at the human information condition with regard to our increasing need to interact with IR systems. The need has grown due to a number of factors, including the increased importance of information to more people in this information age. Also, IR was once considered document-oriented, but has now evolved to include multimedia, text, and other information objects. As a result, IR systems and their complexity have proliferated as users and user purposes for using them have also proliferated. Human interaction with IR systems can often be frustrating as people often lack an understanding of IR system functionality. New more holistic directions in CIR are emerging that conceptualize human-IR system interaction at the human-computer interaction level (taking into account interface issues), the information behavior level (taking into account the role of IR system interaction in the total range of people’s information behaviors), and the

We address the problem of designing, implementing, and experimenting with compressed data structures that support rank and select queries over a dictionary of integers. We shine a new light on this classical problem by showing a connection between the input integers and the geometry of a set of points in a Cartesian plane suitably derived from them. We then build upon some results in computational geometry to introduce the first compressed rank/select dictionary based on the idea of “learning” the distribution of such points via proper linear approximations (LA). We therefore call this novel data structure the la_vector . We prove time and space complexities of the la_vector in several scenarios: in the worst case, in the case of input distributions with finite mean and variance, and taking into account the k th order entropy of some of its building blocks. We also discuss improved hybrid data structures, namely, ones that suitably orchestrate known compressed rank/select dictionaries with the la_vector . We corroborate our theoretical results with a large set of experiments over datasets originating from a variety of applications (Web search, DNA sequencing, information retrieval, and natural language processing) and show that our approach provides new interesting space-time tradeoffs with respect to many well-established compressed rank/select dictionary implementations. In particular, we show that our select is the fastest, and our rank is on the space-time Pareto frontier.

Standard Information Retrieval (IR) metrics are not well suited for new paradigms like XML or Web IR in which retrievable information units are document elements and/or sets of related documents. Part of the problem stems from the classical hypotheses on the user models: They do not take into account the structural or logical context of document elements or the possibility of navigation between units. This article proposes an explicit and formal user model that encompasses a large variety of user behaviors. Based on this model, we extend the probabilistic precision-recall metric to deal with the new IR paradigms.