FindZebra: A search engine for rare diseases

Googling for Health Information

Physicians and the general public are increasingly using web-based tools to find answers to medical questions. The field of rare diseases is especially challenging and important as shown by the long delay and many mistakes associated with diagnoses. In this paper we review recent initiatives on the use of web search, social media and data mining in data repositories for medical diagnosis. We compare the retrieval accuracy on 56 rare disease cases with known diagnosis for the web search tools google.com, pubmed.gov, omim.org and our own search tool findzebra.com. We give a detailed description of IBM's Watson system and make a rough comparison between findzebra.com and Watson on subsets of the Doctor's dilemma dataset. The recall@10 and recall@20 (fraction of cases where the correct result appears in top 10 and top 20) for the 56 cases are found to be be 29%, 16%, 27% and 59% and 32%, 18%, 34% and 64%, respectively. Thus, FindZebra has a significantly (p < 0.01) higher recall than the other 3 search engines. When tested under the same conditions, Watson and FindZebra showed similar recall@10 accuracy. However, the tests were performed on different subsets of Doctors dilemma questions. Advances in technology and access to high quality data have opened new possibilities for aiding the diagnostic process. Specialized search engines, data mining tools and social media are some of the areas that hold promise.

Hand in hand with technological advancements, treatment modalities continue to grow. With the turn of the century, the internet has become the number one source of information for almost every topic. Thus, many patients look toward the internet as their primary source of information to learn about their respective medical conditions. The American Medical Association and National Institute of Health strongly recommend that online medical information be written at the 6th to 8th-grade level to aid comprehension by patients of all literacy backgrounds. To assess the readability of online information regarding Jones fracture. Our hypothesis is that the reading level of medical information published on websites far exceeds the recommended reading level of 6th-8th grade as proposed by the American Medical Associate and National Institute of Health. The result of this study can help us formulate improved recommendations for publishing more comprehensible material and, thus, eventually improve patient compliance and clinical outcomes. The exact phrase "Jones fracture" was queried on the three most common search engines, Google, Yahoo!, and Bing, on December 28, 2022. As of December 2022, Google held 84%, Bing held 9%, and Yahoo! held 2% of the worldwide search engine market share. Web pages uniform resource locator from the first three pages of search results were recorded from each search engine. These web pages were classified according to academic, physician-sponsored, governmental and non-government organizations (NGO), commercial, and unspecified as per formally defined categories. Websites associated with an educational institution or medical organization were classified as academic. Websites with products for sale, corporate sponsorship, or advertisements were classified as commercial. Governmental websites or NGOs comprised those that received government subsidies or grants. Webpages that were independently owned by physicians or physician groups were respectively classed as physician sponsored. The remainder of websites that did not fall under the above categories were classified as unspecified. A total of 93 websites were analyzed for reading assessment. A whopping 44% of websites were commercial, followed by 22% of physician-sponsored websites. Third place belonged to non-government organization websites holding a 15% share. The academic website held a meager 9% portion, while unspecified sites were 3%. The table illustrates mean readability scores, along with average cumulative grade level. The average grade level was 10.95 ± 2.28 for all websites, with a range of 6.18 to 18.90. Since P values were more than 0.05, there was not a significant statistical difference between the first page results and the results of all pages. Thus, we can rationalize that readability scores are consistent throughout all pages of a website. Hand in hand with technological advancements, treatment modalities continue to grow. With the turn of the century, the internet has become the number one source of information for almost every topic. Thus, many patients look towards the internet as the primary source of information to learn about their respective medical conditions. Our study demonstrates that current online medical information regarding Jones fracture is written at an extraordinarily high-grade level, with an average grade level of all websites at 10.95, nearly an 10th-grade educational level. The American Medical Association and National Institute of Health strongly recommend that online medical information should be written at the 6th to 8th-grade level to aid comprehension by patients of all literacy backgrounds. On the contrary, most of the medical information evaluated was at an 10th-grade level, which far exceeds recommendations by AMA and NIH. This is particularly relevant because readability scores are directly proportional to the level of comprehension attained by readers, thus directly impacting patient outcomes. In conclusion, we suggest and encourage that all online reading materials should be re-written at the 6th to 8th-grade level in a public service effort to increase compliance with treatment goals and raise awareness of preventive measures.

In our recent paper, we study web search as an aid in the process of diagnosing rare diseases. To answer the question of how well Google Search and PubMed perform, we created an evaluation framework with 56 diagnostic cases and made our own specialized search engine, FindZebra (findzebra.com). FindZebra uses a set of publicly available curated sources on rare diseases and an open-source information retrieval system, Indri. Our evaluation and the feedback received after the publication of our paper both show that FindZebra outperforms Google Search and PubMed. In this paper, we summarize the original findings and the response to FindZebra, discuss why Google Search is not designed for specialized tasks and outline some of the current trends in using web resources and social media for medical diagnosis.

Searching the Web, Continued

Purpose — The purpose of this chapter is to give an overview of the context of Web search and search engine related research, as well as to introduce the reader to the sections and chapters of the book. Methodology/approach — We review literature dealing with various aspects of search engines, with special emphasis on emerging areas of Web searching, search engine evaluation going beyond traditional methods and new perspectives on Web searching. Findings — The approaches to studying Web search engines are manifold. Given the importance of Web search engines for knowledge acquisition, research from different perspectives needs to be integrated into a more cohesive perspective. Research limitations/implications — The chapter suggests a basis for research in the field and also introduces further research directions. Originality/value of paper — The chapter gives a concise overview of the topics dealt within the book and also shows directions for researchers interested in Web search engines.

Distributed heterogeneous search environments are an emerging phenomenon in Web search, in which topic-specific search engines provide search services, and metasearchers distribute user’s queries to only the most suitable search engines. Previous research has explored the performance of such environments from the user’s perspective (e.g., improved quality of search results). We focus instead on performance from the search service provider’s point of view (e.g, income from queries processed vs. resources used to answer them). We analyse a scenario in which individual search engines compete for queries by choosing which documents to index. We propose the COUGAR algorithm that specialised search engines can use to decide which documents to index on each particular topic. COUGAR is based on a game-theoretic analysis of heterogeneous search environments, and uses reinforcement learning techniques to exploit the sub-optimal behaviour of its competitors.KeywordsNash EquilibriumSearch EngineUser QueryIndex SizeSample TrialThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper presents a new approach to accessing information on the Web. FooCA, an application in the field of Conceptual Knowledge Processing, is introduced to support a holistic representation of today’s standard sequential Web search engine retrieval results. FooCA uses the itemset consisting of the title, a short description, and the URL to build a context and the appropriate concept hierarchy. In order to generate a nicely arranged concept hierarchy using line diagrams to retrieve and analyze the data, the prior context can be iteratively explored and enhanced. The combination of Web Mining techniques and Formal Concept Analysis (FCA) with contextual attribute elicitation gives the user more insight and more options than a traditional search engine interface. Besides serving as a tool for holistic data exploration, FooCA also enables the regular user to learn step by step how to run new, optimized search queries for his personal information need on the Web.

BackgroundAdolescents and young adults are disproportionately affected by HIV, suggesting that HIV prevention methods such as pre-exposure prophylaxis (PrEP) should focus on this group as a priority. As digital natives, youth likely turn to internet resources regarding health topics they may not feel comfortable discussing with their medical providers. To optimize informed decision-making by adolescents and young adults most impacted by HIV, the information from internet searches should be educational, accurate, and readable.ObjectiveThe aims of this study were to compare the accuracy of web-based PrEP information found using web search engines and virtual assistants, and to assess the readability of the resulting information.MethodsAdolescent HIV prevention clinical experts developed a list of 23 prevention-related questions that were posed to search engines (Ask.com, Bing, Google, and Yahoo) and virtual assistants (Amazon Alexa, Microsoft Cortana, Google Assistant, and Apple Siri). The first three results from search engines and virtual assistant web references, as well as virtual assistant verbal responses, were recorded and coded using a six-tier scale to assess the quality of information produced. The results were also entered in a web-based tool determining readability using the Flesch-Kincaid Grade Level scale.ResultsGoogle web search engine and Google Assistant more frequently produced PrEP information of higher quality than the other search engines and virtual assistants with scores ranging from 3.4 to 3.7 and 2.8 to 3.3, respectively. Additionally, the resulting information generally was presented in language at a seventh and 10th grade reading level according to the Flesch-Kincaid Grade Level scale.ConclusionsAdolescents and young adults are large consumers of technology and may experience discomfort discussing their sexual health with providers. It is important that efforts are made to ensure the information they receive about HIV prevention methods, and PrEP in particular, is comprehensive, comprehensible, and widely available.

Internet has become an instant source of informatio n for almost anyone. By viewing the results of sear ch terms displayed by an Internet Search Engine (ISE), a person may decide his next course of action whether to continue using the Internet, abort or to combine it with other sources of information. The results produced by an ISE can be regarded as an in dex of relative availability of references. Given a list of suitable search terms, a user will firstly have to decide on the ISE to be used. Due to huge potent ial references available for given search terms the use r has to create heuristics to choose the entries sh own on the computer screen. As there are varying breadths and depths of information revealed by various Inter net Search Engines (ISE’s), this paper will not attempt to make a comparison among ISE’s, rather will focu s only on a particular search engine and ascertain th e results it produces given a list of search terms. Google is chosen as a proxy, being one of the most popular ISE’s. By confining to only a search engine, the s tudy affords to control variability among ISE’s should m ultiple ISE’s be used. With the search engine place d under control, it is easy to achieve the primary ob jective of the study, i.e., to ascertain availabili ty of relative breadth of sub-themes of Analytic Hierarch y Process (AHP) in Google. It is natural for user, especially researcher to be concerned with number, quantity. If there is seemingly abundant literature , one would be motivated to pursue research along the the me or sub-theme. In this study, the strength of presence of a sub-theme is measured by using two measures: (i) result of a sub-theme of AHP over the sub-theme itself, and (ii) result of the sub-theme over the total results generated for all of the thi rty six sub-themes used in the search. This study controlle d biasness in specifying the sub-themes of AHP by adopting the sub-themes or search terms specified b y the 2013 AHP conference organizers. This decision helps make the study efficient without with it has to distill the sub-themes by surveying the AHP literature. The data for analysis was gathered by s urfing Google on 26 Feb 2013 8.55 p.m. - 9.26 p.m. Peninsular Malaysian time. The search results were computed to generate two types of ratios specified earlier. A composite index was created using the re sulting two types of ratios which are used to class ify the efficiency, hence dominance of the original res ults (hits). Kendall’s correlation produced statist ically significant correlations between the composite inde x and Rank of AHP specific and area results. Using indices greater than 1.000 as the base, 6 AHP specific areas occupy the top positions with ratios ranging from 19.341 to 61.574; 15 AHP specific areas occupy the second top positions with ratios ranging from 1.119 to 9.602, and 14 AHP specific areas occupy the third and last position with indi ces below 1.000 ranging from 0.050 to 0.894. The paper includes dis cussion, implications, limitations, conclusions and suggestions for further research.

CYBER NEWS

Nowadays, as the usage of Internet has incredibly increased, web search engines become the common approach to find and retrieve needed information. Hence, evaluating search engine quality is a hot topic which attracts many researches' attention. In this paper, we propose a framework named Similarity based Automatic Web Search Engine Evaluation, SAWSEE, to evaluate web search engines. SAWSEE measures the information retrieval effectiveness of web search engines by comparing and voting their returned results, particularly using nDCG metric to rank search engines. SAWSEE compares the search engines' results based on their similarity which is calculated in two consecutive levels, the web page address level and the main content of web page level. To find the similarity of the main content of two search engines' results, SAWSEE utilizes Winnowing algorithm, a well-known and widely-used plagiarism detection method. We compared our method with the results acquired from human assessors' evaluations. The promising comparison shows that SAWSEE provides rankings that are consistent with the rankings resulted from human assessors' evaluations. Hence, the proposed method can be applied in real world environments for evaluation of web search engines.

Social Media Posts and Search Engine Queries as the Canary in the Coal Mine for Public Health Surveillance

General purpose Web search engines can be a valuable tool to locate high quality information and can complement the use of specialized systems, such as LexisNexis or WestLaw. The effective use of Web search engines differs somewhat from that of other information retrieval systems and requires an understanding of search engine design and procedures before effective use. We present an overview of search engines and user behavior in Web searching as an introduction to a discussion of the effectiveness of using some of the advanced searching techniques common in other searching environments in Web searches. We empirically test the effectiveness on query operators for Web searching. The use of operators such as AND, OR, and PHRASE was found to have little or no difference on the number of relevant results retrieved for three major search engines, America Online, Google, and Microsoft Search.

Putting pharmacogenomics into practice