Generating suggestions for queries in the long tail with an inverted index

Query suggestion or auto-completion mechanisms are widely used by search engines and are increasingly attracting interest from the research community. However, the lack of commonly accepted evaluation methodology and metrics means that it is not possible to compare results and approaches from the literature. Moreover, often the metrics used to evaluate query suggestions tend to be an adaptation from other domains without a proper justification. Hence, it is not necessarily clear if the improvements reported in the literature would result in an actual improvement in the users' experience. Inspired by the cascade user models and state-of-the-art evaluation metrics in the web search domain, we address the query suggestion evaluation, by first studying the users behaviour from a search engine's query log and thereby deriving a new family of user models describing the users interaction with a query suggestion mechanism. Next, assuming a query log-based evaluation approach, we propose two new metrics to evaluate query suggestions, pSaved and eSaved. Both metrics are parameterised by a user model. pSaved is defined as the probability of using the query suggestions while submitting a query. eSaved equates to the expected relative amount of effort (keypresses) a user can avoid due to the deployed query suggestion mechanism. Finally, we experiment with both metrics using four user model instantiations as well as metrics previously used in the literature on a dataset of 6.1M sessions. Our results demonstrate that pSaved and eSaved show the best alignment with the users satisfaction amongst the considered metrics.

Personalized query suggestion based on user behavior

PurposeThe purpose of this study is to generate diversified results for temporally ambiguous queries and the candidate queries are ensured to have a high coverage of subtopics, which are derived from different temporal periods.Design/methodology/approachTwo novel time-aware query suggestion diversification models are developed by integrating semantics and temporality information involved in queries into two state-of-the-art explicit diversification algorithms (i.e. IA-select and xQuaD), respectively, and then specifying the components on which these two models rely on. Most importantly, first explored is how to explicitly determine query subtopics for each unique query from the query log or clicked documents and then modeling the subtopics into query suggestion diversification. The discussion on how to mine temporal intent behind a query from query log is also followed. Finally, to verify the effectiveness of the proposal, experiments on a real-world query log are conducted.FindingsPreliminary experiments demonstrate that the proposed method can significantly outperform the existing state-of-the-art methods in terms of producing the candidate query suggestion for temporally ambiguous queries.Originality/valueThis study reports the first attempt to generate query suggestions indicating diverse interested time points to the temporally ambiguous (input) queries. The research will be useful in enhancing users’ search experience through helping them to formulate accurate queries for their search tasks. In addition, the approaches investigated in the paper are general enough to be used in many domains; that is, experimental information retrieval systems, Web search engines, document archives and digital libraries.

Query suggestions help users of a search engine to refine their queries. Previous work on query suggestion has mainly focused on incorporating directly observable features such as query co-occurrence and semantic similarity. The structure of such features is often set manually, as a result of which hidden dependencies between queries and users may be ignored. We propose an AHNQS model that combines a hierarchical structure with a session-level neural network and a user-level neural network to model the short- and long-term search history of a user. An attention mechanism is used to capture user preferences. We quantify the improvements of AHNQS over state-of-the-art RNN-based query suggestion baselines on the AOL query log dataset, with improvements of up to 21.86% and 22.99% in terms of MRR@10 and Recall@10, respectively, over the state-of-the-art; improvements are especially large for short sessions.

Search has become a ubiquitous, everyday activity, but finding the right information at the right time in an electronic document collection can still be a very challenging process. Significant time is being spent on identifying suitable search terms, exploring matching documents, rephrasing the search request and assessing whether a document contains the information sought. Once another user is faced with a similar information need, the whole process starts again. There is significant potential in cutting down on this activity by taking a user straight to the required information. As well as delivering technical information and vital regulatory information, a knowledge management solution is concerned with capturing valuable insight and experience in order to share it amongst workers. A search engine has been developed and deployed to technical support staff and we were able to assess its impact on mobile workers. The architecture is based on open-source software to satisfy the basic search functionality, such as indexing, search result ranking, faceting and spell checking. The search engine indexes a number of knowledge repositories relevant to the field engineers. On top of that we have developed an adaptive query suggestion mechanism called Sunny Aberdeen. Query suggestions provide an interactive feature that can guide the user through the search process by providing alternative terminology or suggesting ‘best matches’. In our search engine, the query suggestions are generated and adapted over time using state-of-the-art machine learning approaches, which exploit past user interactions with the search engine to derive query suggestions. Apart from continuously updating the suggestions, this framework is also capable of reflecting current search trends as well as forgetting relations that are no longer relevant. Query log analysis of the system running in a real-life context indicates that the system was able to cut down the number of repeat faults and speeds up the decision process for sending out staff to certain jobs.

Personalized search aims to improve the search quality by re-ranking the candidate document list based on user's historical behavior. Existing approaches focus on modeling the order information of user's search history by sequential methods such as Recurrent Neural Network (RNN). However, these methods usually ignore the fine-grained time information associated with user actions. In fact, the time intervals between queries can help to capture the evolution of query intent and document interest of users. Besides, the time intervals between past actions and current query can reflect the re-finding tendency more accurately than discrete steps in RNN. In this paper, we propose PSTIE, a fine-grained Time Information Enhanced model to construct more accurate user interest representations for Personalized Search. To capture the short-term interest of users, we design time-aware LSTM architectures for modeling the subtle interest evolution of users in continuous time. We further leverage time in calculating the re-finding possibility of users to capture the long-term user interest. We propose two methods to utilize the time-enhanced user interest into personalized ranking. Experiments on two datasets show that PSTIE can effectively improve the ranking quality over state-of-the-art models.

In this paper, we model the contents recommender which applies the collaborative filtering and the vector comparison techniques. The system mined the users’ usage history about consuming contents, suggested the user favorable contents. We constructed the usage history data set about 49 users for showing the effectiveness of the proposed algorithms, the results were showed that the collaborative filtering technologies are helpful to resolve the data sparseness problems in the contents recommender.

Search engine users usually strive to reformulate their queries in the search process to gain useful information. It is hard for search engines to understand users’ search intents and return appropriate results if they submit improper or ambiguous queries. Therefore, query reformulation is a bottleneck issue in the usability of search engines. Modern search engines normally provide users with some query suggestions for references. To help users to better learn their information needs, it is of vital importance to investigate users’ reformulation behaviors thoroughly. In this paper, we conduct a detailed investigation of users’ session-level reformulation behavior on a large-scale session dataset and discover some interesting findings that previous work may not notice before: (1) Intent ambiguity may be the direct cause of long sessions rather than the complexity of users’ information needs; (2) Both the added and the deleted terms in a reformulation step can be influenced by the clicked results to a greater extent than the skipped ones; (3) Users’ specification actions are more likely to be inspired by the result snippets or the landing pages, while the generalization behaviors are impacted largely by the result titles. We further discuss some concerns about the existing query suggestion task and give some suggestions on the potential research questions for future work. We hope that this work could provide assistance for the researchers who are interested in the relative domain.

A recent query-log mining approach for query recommendation is based on Query Flow Graphs, a markov-chain representation of the query reformulation process followed by users of Web Search Engines trying to satisfy their information needs. In this paper we aim at extending this model by providing methods for dealing with evolving data. In fact, users' interests change over time, and the knowledge extracted from query logs may suffer an aging effect as new interesting topics appear. Starting from this observation validated experimentally, we introduce a novel algorithm for updating an existing query flow graph. The proposed solution allows the recommendation model to be kept always updated without reconstructing it from scratch every time, by incrementally merging efficiently the past and present data.

In E-commerce recommendation system,Information overload on Internet has brought a tough problem,which is how to precisely position users' interest and provide users with accurate product recommendation.To solve this problem,in this paper,a product characteristic extraction algorithm was developed and a user preference model was constructed by deeply mining large-scale of user reviews in E-commerce community,to improve accuracy of traditional collaborative filtering recommendation algorithm with coordination of historic user rating information;moreover,data sparsity problem was alleviated with similarity propagation technique.Experiment results show that,in condition of sparse data,algorithm in this paper can still fit product preference of users very well,and has significantly improvement in accuracy compared with traditional collaborative filtering algorithm.

Currently, Data Warehouse (DW) analyses are extensively being used not only for strategic business decisions by a few, but also for feedback to a wider audience and into daily operational decisions. As a result, there’s an increase in the number of aggregation star-queries that are being concurrently submitted. Although such queries require similar processing patterns, they are stressing the database engine ability to deliver timely execution, due to the fact that each query executes independently from the others (query-at-time processing model). Recently, there’s an increasing interest in approaches that cooperate to manage large numbers of concurrent aggregation star-queries. We have proposed SPIN in a previous paper [1]. It is a data processing model that shares data and computation in order to handle large concurrent query loads, and its data organization provides almost constant and predictable execution times for all submitted queries. It has a data reader that reads data in circular loop, placing it in a pipeline, before being processed by branches that combine common processing computations. SPIN is IO dependent, i.e. a query is only be answered after a full circular loop, even though tuples and similar predicates have been evaluated in the past. In this paper we propose data processing approach that uses a set of bitsets, built on-the-fly, to significantly reduce the query processing time, the tuple evaluation cost and the number of predicates and tuples evaluated, without sacrificing its predictability features. The data read from storage is reduced to the minimum needed by the current query load.KeywordsExecution TimeData WarehouseSelection OperatorData ReaderAverage Execution TimeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Given a query, there are many different query processing strategies to fulfil user requirements. The selection of tuples are from 0% to 100%. The found set of relations will determine the different query processing strategies being implemented. To implement effective query processing strategies relies on the cost model of query processing. Traditionally, query processing does not handle personalization of user requirements. In the e-commerce environment, in order to achieve a fast response time of a query it requires personalization in the relation. We introduce the concept of personalization at the query processing level. We discuss a cost model for each of the query processing strategies and use a data mining technique such as classification in selecting the most effective query processing strategy for personalization. We introduce the current query processing strategies. We conclude with a data mining technique as an alternative in selecting a query processing strategy.

With the explosion of data in the past decade, big data is becoming a research hotspot in the information field. Many cloud-based distributed data processing platforms have been proposed to provide efficient and cost effective solutions for big data query processing, such as Hadoop, Hive, Pig, etc. However, most of the current research works are focus on improving the performance of query processing based on the view of systematics while without considering the characteristics of queries themselves, such as the query similarity, which will cause large numbers of redundant computation, effect query execution efficiency, thus having an adverse impact on promotion of the multi-queries processing performance. To solve this problem, in this paper, we propose a Multi-queries optimization framework based on MapReduce-oriented cloud environment (Multi-Q), which utilizes the dependence between multiple queries to realize query results reuse. Firstly, a cluster-based partition algorithm called CPA has been exploited to conduct the logic partition of the search range of query workload. Secondly, a multi-queries reuse dependence graph (MRDG) construction method on the basis of the cluster-based partition results has been presented to depict the dependence between the multiple queries. Finally, a Multi-Q processing algorithm based on Multi-Q Reuse Dependence Graph has been put forward to achieve the query results reuse and improve the overall query processing performance. We evaluate our approach by deploying Multi-Q based on Hadoop in a real cloud environment, called SEU-Cloud, and conducting extensive experiments based on the standard TPC-H. The result verifies that compared with Hive, the performance of improvement is approximately 39.3% by using our Multi-Q.

In this paper, we explore the problem of developing personalized chatbots. A personalized chatbot is designed as a digital chatting assistant for a user. The key characteristic of a personalized chatbot is that it should have a consistent personality with the corresponding user. It can talk the same way as the user when it is delegated to respond to others' messages. We present a retrieval-based personalized chatbot model, namely IMPChat, to learn an implicit user profile from the user's dialogue history. We argue that the implicit user profile is superior to the explicit user profile regarding accessibility and flexibility. IMPChat aims to learn an implicit user profile through modeling user's personalized language style and personalized preferences separately. To learn a user's personalized language style, we elaborately build language models from shallow to deep using the user's historical responses; To model a user's personalized preferences, we explore the conditional relations underneath each post-response pair of the user. The personalized preferences are dynamic and context-aware: we assign higher weights to those historical pairs that are topically related to the current query when aggregating the personalized preferences. We match each response candidate with the personalized language style and personalized preference, respectively, and fuse the two matching signals to determine the final ranking score. Comprehensive experiments on two large datasets show that our method outperforms all baseline models.

The precision of Information Retrieval (IR) System is low due to imprecise user queries as well as because of information overload on web. The Fuzzy set infers the user’s information need from vague and imprecise queries and web recommender systems are used to overcome information overload problem. The performance of recommender system is still low due to data sparsity. The concept of trust is used to deal with data sparseness problem and improves the performance of recommender system. Optimization techniques like Genetic Algorithm(GA) have been applied in domain of information retrieval for effective web search. In this research hybrid of Fuzzy set, GA and Trust has been used together in query log mining for personalized web search based on using fuzzy queries for recommendation of optimal set of trusted documents. Thus the use of hybrid of Fuzzy set, trust and GA together infer the user’s information need from vague and imprecise user’s queries and optimize the web page ranking of trusted web pages for effective personalized web search. The experimental results were analyzed statistically as well as compared with GA IR, and Fuzzy Trust based IR. Hence based on comparative analysis of results, thus hybrid of Fuzzy Set, Trust and GA shows the improvement in average precision of search results and confirms the effective personalization of web search.