CrowdReranking

Googling for Health Information

Frequently a user's information needs are stored in the databases of multiple search engines. It is inconvenient and inefficient for an ordinary user to invoke multiple search engines and identify useful documents from the returned results. To support unified access to multiple search engines, a metasearch engine can be constructed. When a metasearch engine receives a query from a user, it invokes the underlying search engines to retrieve useful information for the user. Metasearch engines have other benefits as a search tool such as increasing the search coverage of the Web and improving the scalability of the search. In this article, we survey techniques that have been proposed to tackle several underlying challenges for building a good metasearch engine. Among the main challenges, the database selection problem is to identify search engines that are likely to return useful documents to a given query. The document selection problem is to determine what documents to retrieve from each identified search engine. The result merging problem is to combine the documents returned from multiple search engines. We will also point out some problems that need to be further researched.

Relevance evaluation is an important topic in Web search engine research. Traditional evaluation methods resort to huge amount of human efforts which lead to an extremely time-consuming process in practice. With analysis on large scale user query logs and click-through data, we propose a performance evaluation method that fully automatically generates large scale Web search topics and answer sets under Cranfield framework. These query-to-answer pairs are directly utilized in relevance evaluation with several widely-adopted precision/recall-related retrieval performance metrics. Besides single search engine log analysis, we propose user behavior models on multiple search engines' click-through logs to reduce potential bias among different search engines. Experimental results show that the evaluation results are similar to those gained by traditional human annotation, and our method avoids the propensity and subjectivity of manual judgments by experts in traditional ways.

HyperSSR: A hypergraph based semi-supervised ranking method for visual search reranking

We propose UniScore as a metric for integrating and standardizing the outputs of multiple search engines in the analysis of data-dependent acquisition (DDA) data from LC/MS/MS-based bottom-up proteomics. UniScore is calculated from the annotation information attached to the product ions alone by matching the amino acid sequences of candidate peptides suggested by the search engine with the product ion spectrum. The acceptance criteria are controlled independently of the score values by using the false discovery rate based on the target-decoy approach. Compared to other rescoring methods that use deep learning-based spectral prediction, larger amounts of data can be processed using minimal computing resources. When applied to large-scale global proteome data and phosphoproteome data, the UniScore approach outperformed each of the conventional single search engines examined (Comet, X! Tandem, Mascot, and MaxQuant). Furthermore, UniScore could also be directly applied to peptide matching in chimeric spectra without any additional filters.

When we search information on the Web, we usually enter a query that best represents our information need, and then modify it as the search results are shown. Many thus have studied search queries to understand how users express and find their information need on the Web. For example, studies have shown that query length affects search performance (Belkin et al, 2003). Then, researchers have begun to examine image search queries on the Web. They analyzed search logs from general or commercial search engines or studied specific image database systems based on log analysis (Goodrum & Sprink, 2001; Jorgensen & Jorgensen, 2005; Goodrum, Bejune, & Siochi, 2003; Pu, 2005; Cunningham, Bainbridge, & Masoodian, 2004). While these studies provide basic understanding of image query formulation, further characteristics of user's queries in the interactive searching process have not been explored. This study thus builds on previous studies on image search queries by examining the patterns of query formulation and modification in visual information search. Specifically, first, it investigates the effects of tasks and other contextual factors on query length and query changes; and second, it compares the query length and changes with the user's satisfaction of search results. Participants Twenty-nine college students were recruited for data collection during the spring and summer semesters of 2008, mostly from the Department of Media Studies at the Catholic University of America (CUA). They included 22 from Media Studies and 7 from other fields taking courses in the department; 22 were females and 7 males; and an average age of 21 years. Background information collected were years of computer experience, hours per day using the Web, self-rated online searching expertise, and self-rated digital image searching experience. The seven-point Likert scale was used for self-rating variables. Each participant took part in three different search sessions with his or her own search topic at CUA with the researcher's laptop that had an Internet connection. The university's homepage was set as the default home for the Web browser. Each participant filled out a pre-search questionnaire to collect a search task, topic, self-rated topic familiarity using 7-Likert scale. The participants chose their own preferred browser between Internet Explorer and Firefox. The search session was recorded by Camtasia v5.0 as screen capture. After the session, a post-search questionnaire was given to the participants to assess the level of satisfaction and confidence of their searching using 7-Likert scale. There was no time limit to complete a search session. The participants were free to end a search session at any time. Search Tasks The participants brought their own task and topics to find images. There were three types of search tasks identified: academic task, work-related task, and personal interest (Table 1). Search queries were acquired from screen capture log files. A total of 979 queries were obtained as the entire string of terms submitted by the participants. Table 2 shows a distribution of search queries on a general search engine, a search engine's image search, and other specific sites. Findings showed that Google Images Search was the most frequent source for forming a search query. An average word per query was 3.07 during the first session, 2.95 during the second session, and 2.89 for the last session. This finding of short queries is similar to that of previous studies on Web image queries (Goodrum & Spink, 2001; Jorgensen & Jorgensen, 2005). Analysis of data is in progress. A poster will summarize detail findings and provide answers to the following questions: What are the patterns of query modification? Are there any differences of query length and number of query change based on the task and topic familiarities? Is there any relationship between query length and number of query change and users' satisfaction and confidence with search results? Do different stages of a user's particular task affect query length and number of query change? The author would like to thank OCLC/ALISE for their generous funding for this project.

A metasearch engine is a system that supports uni ed access to multiple local search engines. Database selection is one of the main challenges in building a large-scale metasearch engine. The problem is to eAEciently and accurately determine a small number of potentially useful local search engines to invoke for each user query. In order to enable accurate selection, metadata that re ect the contents of each search engine need to be collected and used. In this paper, we propose a highly scalable and accurate database selection method. This method has several novel features. First, the metadata for representing the contents of all search engines are organized into a single integrated representative. Such a representative yields both computation eAEciency and storage eAEciency. Second, our selection method is based on a theory for ranking search engines optimally. Experimental results indicate that this new method is very e ective. An operational prototype system has been built based on the proposed approach.

Using data from the Pew Internet and American Life Project surveys, this article explores changing trends in reported sophistication and satisfaction with search skills and with search engines. We find that the proportion of Internet users searching online for answers to specific questions—as opposed to casual browsing—has grown significantly. Moreover, as users get more experience online, they increasingly become dependent on search engines, confident in their findings, and savvy about how search engines structure information, privilege paid results, and track users. When other factors are controlled, years of online experience is a strong predictor of the likelihood of a person doing specific searches on a daily basis, and experience can have an even stronger positive effect than education and income. We also find that years of online experience, frequency of use, and sophistication with multiple search engines can overcome socio-economic status in predicting how active a person is in searching across different topics.

Finding relevant data in the biomedical literature can be difficult sometime. To select a few neuroscience examples, suppose you would like to know (a) which serotonin receptor subunits are expressed in dentate gyrus mossy cells; (b) the average volume of the amygdala in adult male chimpanzees; (c) whether there is an EEG signature of CreutzfeldJacob disease; or (d) studies reporting bilateral fMRI activity in Brodmann area 38. These are fairly simple questions, for which standard search engines should fare relatively well. Yet even for these kinds of questions, securing the relevant information takes much longer than googling up the local weather forecast for the week-end or tomorrow’s commuter train schedule. The actual data required to build biologically realistic computational models are often more detailed: what is the time constant of the excitatory synaptic current from a specified pair of neuron types? Finding the answer in this case might require many hours or even days of queries over multiple search engines. Most importantly, the results of these queries must be typically followed by at least cursory reading of dozens of papers. When the graduate student triumphantly brings to the lab the needed reference, the adviser could mumble without lifting the eyes from the keyboard “that’s in young animals, and it was recorded at room temperature”. Another unfortunate major limitation is that, until and unless a definitive answer is found, it is usually impossible to know whether the information is available or not. In other words, existing biomedical search engines are ill-equipped to inform users that something is not yet known. Standard search algorithms such as PubMed are less than ideal to deal with data identification, because they are ultimately based on matching strings or concepts that appear in the title, abstract, and the keywords. These texts, however, are written with narrow scientific agendas in mind. The authors cannot possibly provide a list of keywords that would encompass all research projects for which some data in their articles might be relevant. If the topic of a report is the molecular phenotyping of a new genetic model of schizophrenia, the technical details of the deconvolution algorithm to deblur the optical micrographs would be nearly impossible to pick up through keyword searches. Could we devise a procedure to interrogate the scientific literature so as to extract accurately and efficiently most if not all of the relevant data? Is there a literature mining protocol that could give us the confidence that, if the query returns a blank, it means that the sought data is not yet available? Although common to all of biomedical science, this issue is particularly critical in neuroscience because of its unmatched diversity of dimensions, scales, questions, approaches, and techniques. Thus, effective tagging of publications with relevant metadata remains an outstanding neuroinformatics challenge. Full text searches provide half of the solution, in that they eliminate many of the issues related to false negatives. Many of the helpful terms to identify relevant data, for example, appear in the Materials and Methods sections of published articles rather than in their titles, abstracts, and keywords. Search engines scanning through the entire main text of publications include early visionary projects such as Textpresso, which started within the limited domain of C. elegans, then expanded to a

The quality of search engine service relies on its capability to rank the webpages related to the user query in order of their intrinsic value. PageRank constituted a break-through as a method of webpage evaluation and was in use since then for multiple web search engines. This concept has been subject to attacks by malicious web linkage. As an counteraction, the so-called TrustRank was developed that instead of trusting all webpage developers it trusts only a handful of them that were manually selected. This approach has however a number of deficiencies: (1) it affects mainly the pages close to the manually selected pages, (2) it punishes other worthful webpages, (3) it leaves still a margin of manipulation on webpages with topics distant from those selected manually.Therefore, we developed a new approach to webpage ranking, based on the trustrank concept, which attempts to annihilate the mentioned deficiencies via (1) restricting the number of links that can impact the pagerank (2) allowing for multistep linkage to be taken into account. The preliminary experiments on a large scale search engine data confirm the value of this new approach.

Metasearch engines are developed to overcome the shortcoming of single search engine and try to benefit from cooperative decision by combining the results of multiple independent search engines that make use of different models and configurations. In this work, we study the metasearch problem via voting that facilities multiple agents making cooperative decision. We can deem the source search engines as voters and all ranked documents as candidates, then metaseach problem is actually to find a voting algorithm to obtain group’s preferences on these documents(candidates). In addition to two widely discussed classical voting rules: Borda and Condorcet, we study another two voting algorithms, Black and Kemeny. Since Kemeny ranking problem is NP-hard, a new heuristic algorithm has been proposed for metasearch. Some experiments have been carried out on TREC2001 data for evaluating these metasearch algorithms coming from voting.KeywordsSearch EngineCondorcet WinnerStrongly Connect ComponentBorda RuleVote AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The huge amount of user log data collected by search engine providers creates new opportunities to understand user loyalty and defection behavior at an unprecedented scale. However, this also poses a great challenge to analyze the behavior and glean insights into the complex, large data. In this paper, we introduce LoyalTracker, a visual analytics system to track user loyalty and switching behavior towards multiple search engines from the vast amount of user log data. We propose a new interactive visualization technique (flow view) based on a flow metaphor, which conveys a proper visual summary of the dynamics of user loyalty of thousands of users over time. Two other visualization techniques, a density map and a word cloud, are integrated to enable analysts to gain further insights into the patterns identified by the flow view. Case studies and the interview with domain experts are conducted to demonstrate the usefulness of our technique in understanding user loyalty and switching behavior in search engines.

Priority claims for surgical techniques

The Proteomics Standards Initiative has recently released the mzIdentML data standard for representing peptide and protein identification results, for example, created by a search engine. When a new standard format is produced, it is important that software tools are available that make it straightforward for laboratory scientists to use it routinely and for bioinformaticians to embed support in their own tools. Here we report the release of several open-source Java-based software packages based on mzIdentML: ProteoIDViewer, mzidLibrary, and mzidValidator. The ProteoIDViewer is a desktop application allowing users to visualize mzIdentML-formatted results originating from any appropriate identification software; it supports visualization of all the features of the mzIdentML format. The mzidLibrary is a software library containing routines for importing data from external search engines, post-processing identification data (such as false discovery rate calculations), combining results from multiple search engines, performing protein inference, setting identification thresholds, and exporting results from mzIdentML to plain text files. The mzidValidator is able to process files and report warnings or errors if files are not correctly formatted or contain some semantic error. We anticipate that these developments will simplify adoption of the new standard in proteomics laboratories and the integration of mzIdentML into other software tools. All three tools are freely available in the public domain.

In most major search engines, the interface for image search is the same as traditional Web search: a keyword query followed by a paginated, ranked list of results. Although many image search innovations have appeared in both the literature and on the Web, few have seen widespread use in practice. In this work, we explore the differences between image and general Web search to better support users’ needs. First, we describe some unique characteristics of image search derived through informal interviews with researchers, designers, and managers responsible for building and deploying a major Web search engine. Then, we present results from a large scale analysis of image and Web search logs showing the differences in user behaviour. Grounded in these observations, we present design recommendations for an image search engine supportive of the unique experience of image search. We iterate on a number of designs, and describe a functional prototype that we built.