Using temporal bursts for query modeling

Ubiquitous access to internet has resulted in more and more people going online to get their daily dose of news. In a 2010 survey conducted by the Pew Project for Excellence in Journalism, 41% of the respondents said they get most of their news online, 10% more than those who said they got most of their news from a newspaper. A lot of socio-technical factors have contributed to this phenomenal rise in adoption of online news in recent years. One of the biggest reasons why people are increasingly reading news online is because it facilitates discussion with peers (Nguyen 2010), offering different viewpoints which aid in forming a rounded personal opinion about the news story. The Pew survey found that 37% of online news users (and 51% of 18-29 year olds) think that commenting on news stories is an important feature to have. A lot of people tend to shape their opinion by reading discussion comments, reflective articles, blogs and even tweets about the news. Hence, an increasing number of people rely on online sources of news – be it news websites or news aggregator services like Digg, Reddit, Google Reader, Flipboard, Pulse etc. The problem with these news websites and aggregators is that the only way people can gather public opinion is by actively searching through the endless stream of comments and feeds, filtering out spam (which is a growing problem) and then reading the relevant posts. A top trending story on Twitter will typically see multiple tweets per second, and keeping up with the rapid flow of incoming tweets is quite cumbersome and cognitively taxing. Hence it becomes increasingly difficult and time consuming for someone who wants to get the pulse of the people affected by a news story. Furthermore, in certain scenarios people might want to look at more fine grained opinions. Currently, there is no elegant way to extract geographic and demographic impact of a news story. What is the public sentiment in Indonesia about the Arab Spring? How did the public opinion about the Wikileaks disclosures change as the story unfolded during the course of a year? It is very difficult and tedious to observe such patterns using the currently available news providers. This work attempts to solve these problems by proposing a news aggregator platform which pulls news stories from various sources and also aggregates public responses, reflections, opinions and sentiments associated with those stories. This data is presented in ways that are easily understandable so readers can make better sense of the stories unfolding across the globe. Such a news aggregator platform that gathers and display public opinion and sentiments about a story, must deal with various challenges – 1. Opinions are very subjective. Different people feel about a story in different ways. With such an enormous amount of diverse opinions and subjectivity, how can we possibly aggregate the responses into something that makes sense as a whole? ________________________________ Acknowledgements: Prof. Yardi, S. for the guidance Sethi, P. (2013). Public opinion aggregation by annotation and tagging of online news stories. iConference 2013 Proceedings (pp. 891-894). doi:10.9776/13453 Copyright is held by the author. iConference 2013 February 12-15, 2013 Fort Worth, TX, USA 892 2. There isn’t really a unified web standard for expressing opinion (in textual form). Some people tweet in 140 characters, while others write elaborate blog posts. Some websites employ tags which a reader can use to define and classify their public opinion, while others rely on threaded comments and comment ranking systems. How can a platform be flexible enough to adapt to all these varied standards so that it can extract valuable data from various sources? Perhaps the platform can create a new standard of expression on the web which is flexible and comprehensive enough to be used to express diverse views about every news story in the world. 3. How to filter out spam while extracting public opinion? 4. Once the platform has access to the data it needs, how should it be displayed to the reader in a way that makes sense? What forms of visualizations, illustrations and graphical representations can be employed to give the reader a holistic view of how people feel about a story? 5. How can the platform determine and convey effects of geographical, demographic and temporal variations as the story unfolds? These are just a few out of possibly many issues which must be dealt with. Previous research on similar public opinion aggregation services has greatly focused on natural language processing, data mining and text categorization and clustering. Xiaojun (2010) proposed a framework for crawling the web for comments and applying various data mining algorithms on the data to extract relevant information. Diakopoulos and Shamma (2010) used tweets posted in conjunction with the live presidential debate between Barack Obama and John McCain to gauge public opinion. Brody and Diakopoulos (2011) studied the use of word lengthening to detect sentiment in microblogs. This research proposes a solution – The Opinionated Reader, which relies on sentiment tags and annotations associated with a news story. The essential idea is to create a commenting, discussion and sharing plug-in which can be used by news websites and aggregators as a commenting solution for their news pages. Users wanting to share or comment on a news story through the plug-in are asked to tag the news story with sentiment tags and annotate the story with their reaction (happy/positive or sad/negative). These tags and annotations are stored, aggregated and linked to each news story. A mobile application provides the front-end interface for users to access the news stories and the aggregated sentiment associated with each story. The basic architecture is explained as follows: The Opinionated Reader – Mobile/Tablet App The app fetches news articles from various web sources, based on the interests and preferences configured by the user. In every news article, a portion of the screen real estate is reserved for Opinions which shows graphical visualizations and illustrations of the public opinion surrounding the news story. These visualizations include: A Sentiment Graph indicating the popular tags associated with the story (E.g.: “Shocking”, “Inspiring”, “Amusing” etc.). See Figure 1 for example visualization. A Positivity Graph which plots the level of positivity associated with the story on a time scale from when the news broke. See Figure 2 for example visualization. Figure 1. Example Sentiment Graph Figure 2. Example Positivity Graph iConference 2013 February 12-15, 2013 Fort Worth, TX, USA 893 The user can choose to see these visualizations for a particular time period in the evolution history of the news story, or for a specific country. The app also facilitates people to tag and annotate news articles from within its interface. The Opinionated Reader – Commenting and Sharing Web Plug-in These days, a common way of adding discussion and commenting functionality to news websites is by using 3 rd party services (like DISQUS). The Opinionated Reader is a similar service which can be embedded into the news articles of various news websites to enable commenting and sharing. When someone wishes to comment on an article, the comment is directed through this plug-in, which allows the users to annotate the article with the sentiment tags and reactions along with their comments. The Opinionated Reader saves this information along with the commenter’s location and date of comment (See Figure 3). Figure 3. The Opinionated Reader Web Plug-in 'Add Comment' dialog mockup The Opinionated Reader – Back-end The Back-end maintains a database of news items extracted from RSS feeds of various news websites. Each news article is linked with the sentiment tags and reaction/positivity annotations extracted from the comments and annotations gathered by the commenting plug-in. This data is used by the mobile/tablet app to generate visualizations (Sentiment Graph and Positivity Graph). The back-end also performs data mining on the tags and annotations for geographies and tracks the opinions across time. The back end system responds to queries received from the mobile app with the news story and associated tags and annotations, which are then rendered by the mobile app for the user. iConference 2013 February 12-15, 2013 Fort Worth, TX, USA 894 Discussion and Conclusion This design idea is still in a nascent state and has long hours of research, brainstorming, designing and development to go before it can be realized into something tangible. Twitter has grown exponentially in importance as a news source and it would be vastly valuable to integrate Twitter with The Opinionated Reader. Possibilities include use of special hash tags and natural language processing of tweets to extract public sentiment. The current design supports only two reaction annotations – positive and negative. Not every news story fits this annotation paradigm. Further research about human reactions to news stories might unveil interesting insights which would help zero in on a more robust annotation rubric. Lastly, since this service is envisioned to be non-curated and non-moderated, the value served by the app depends on the users themselves. Greater adoption will lead to more annotations and tags, which translates into a more accurate public opinion as presented to the user.

This paper addresses blog feed retrieval where the goal is to retrieve the most relevant blog feeds for a given user query. Since the retrieval unit is a blog, as a collection of posts, performing relevance feedback techniques and selecting the most appropriate documents for query expansion becomes challenging. By assuming time as an effective parameter on the blog posts content, we propose a time-based query expansion method. In this method, we select terms for expansion using most relevant days for the query, as opposed to most relevant documents. This provide us with more trustable terms for expansion. Our preliminary experiments on Blog08 collection shows that this method can outperform state of the art relevance feedback methods in blog retrieval.

The exploding number of computational models produced by Systems Biologists over the last years is an invitation to structure and exploit this new wealth of information. Researchers would like to trace models relevant to specific scientific questions, to explore their biological content, to align and combine them, and to match them with experimental data. To automate these processes, it is essential to consider semantic annotations, which describe their biological meaning. As a prerequisite for a wide range of computational methods, we propose general and flexible similarity measures for Systems Biology models computed from semantic annotations. By using these measures and a large extensible ontology, we implement a platform that can retrieve, cluster, and align Systems Biology models and experimental data sets. At present, its major application is the search for relevant models in the BioModels Database, starting from initial models, data sets, or lists of biological concepts. Beyond similarity searches, the representation of models by semantic feature vectors may pave the way for visualisation, exploration, and statistical analysis of large collections of models and corresponding data.

Log data is of great value for operation and maintenance of systems and networks. With log data, system behaviors can be monitored, traced, analyzed, to detect unusual circumstances and identify warnings, for the purpose of taking timely measure. However, nowadays many general systems making queries and retrieval over log data is traditionally based on the full amount of raw data. When the amount of log data increase substantially, it's difficult to execute queries based on the raw data of extremely large scales to meet response time requirements. As log data will not be updated after generated, this paper proposes that data and queries can be preprocessed to form data preprocessing results of different granularities. Queries submitted by users can take advantage of corresponding preprocessing results, to improve the query response time. This paper proposes a model for queries over log data of various granularities. The main work includes the follows (1) develop a query model based on hybrid granularity, which enables a query to execute on various granularities of data sets after preprocessing, (2) analyze and prove the completeness and correctness of the proposed query model based on hybrid granularity, (3) describe the query action based on hybrid granularity model formally and present the algorithm framework of the query transformation, (4) analyze and demonstrate the advantages of efficiency of the proposed query model compared to the original data query model. The proposed solution is used in some practical systems. The results show that this solution can guarantee the correctness of the query results while it is able to improve the responsive efficiency of the query significantly. Preprocessing, (2) analyze and prove the completeness and correctness of the proposed query model based on hybrid granularity, (3) describe the query action based on hybrid-granularity model formally and present the algorithm framework of the query transformation, (4) analyze and demonstrate the advantages in efficiency of the proposed query model compared to the original data query model. The proposed solution is used in some practical systems. The results show that this solution can guarantee the correctness of query results while it is able to improve the responsive efficiency of the query significantly.

Improved TFIDF in big news retrieval: An empirical study

Considering the amount of time we spend on the internet, web pages have evolved over a period of time with rapid progression and momentum. With such advancement, we find ourselves fronting a few hostile ideologies, breaching the security levels of webpages as such. The most hazardous of them all is XSS, known as Cross-Site Scripting, is one of the attacks which frequently occur in website-based applications. Cross-Site Scripting (XSS) attacks happen when malicious data enters a web application through an untrusted source. The spam attacks happen in the form of Wall posts, News feed, Message spam and mostly when a user is open to download content of webpages. This paper investigates the use of machine learning to build classifiers to allow the detection of XSS. Establishing our approach, we target the detection modus operandi of XSS attack via two features: URLs and JavaScript. To predict the level of XSS threat, we will be using four machine learning algorithms (SVM, KNN, Random forest and Logistic Regression). Proposing these classified algorithms, webpages will be branded as malicious or benign. After assessing and calculating the dataset features, we concluded that the Random Forest Classifier performed most accurately with the lowest False Positive Rate of 0.34. This precision will ensure a method much efficient to evaluate threatening XSS for the smooth functioning of the system.

In Information Retrieval (IR), the semantic gap represents the mismatch between users’ queries and how retrieval models answer to these queries. In this paper, we explore how to use external knowledge resources to enhance bag-of-words representations and reduce the effect of the semantic gap between queries and documents. In this regard, we propose several simple but effective knowledge-based query expansion and reduction techniques, and we evaluate them for the medical domain. The query reformulations proposed are used to increase the probability of retrieving relevant documents through the addition to, or the removal from, the original query of highly specific terms. The experimental analyses on different test collections for Precision Medicine IR show the effectiveness of the developed techniques. In particular, a specific subset of query reformulations allow retrieval models to achieve top performing results in all the considered test collections.

Blogs are a dynamic communication medium which has been widely established on the web. The BlogForever project has developed an innovative system to harvest, preserve, manage and reuse blog content. This paper presents a key component of the BlogForever platform, the web crawler. More precisely, our work concentrates on techniques to automatically extract content such as articles, authors, dates and comments from blog posts. To achieve this goal, we introduce a simple and robust algorithm to generate extraction rules based on string matching using the blog's web feed in conjunction with blog hypertext. This approach leads to a scalable blog data extraction process. Furthermore, we show how we integrate a web browser into the web harvesting process in order to support data extraction from blogs with JavaScript generated content.

In this demo paper, a SPARQL Query Recommendation Tool (called SQUIRE) based on query reformulation is presented. Based on three steps, Generalization, Specialization and Evaluation, SQUIRE implements the logic of reformulating a SPARQL query that is satisfiable w.r.t a source RDF dataset, into others that are satisfiable w.r.t a target RDF dataset. In contrast with existing approaches, SQUIRE aims at recommending queries whose reformulations: (i) reflect as much as possible the same intended meaning, structure, type of results and result size as the original query and (ii) do not require to have a mapping between the two datasets. Based on a set of criteria to measure the similarity between the initial query and the recommended ones, SQUIRE demonstrates the feasibility of the underlying query reformulation process, ranks appropriately the recommended queries, and offers a valuable support for query recommendations over an unknown and unmapped target RDF dataset, not only assisting the user in learning the data model and content of an RDF dataset, but also supporting its use without requiring the user to have intrinsic knowledge of the data.

Contrarily to classical information retrieval systems, the systems that treat structured documents include the structural dimension through the document and query comparison. Thus, relevant results are all the document fragments that match the user need rather than the whole document. In such case, the document and query structure should be taken into account in the retrieval process as well as during the reformulation. Query reformulation should also include the structural dimension. In this paper we propose an approach of query reformulation based on structural relevance feedback. We start from the original query on one hand and the fragments judged as relevant by the user on the other. Structure hints analysis allows us to identify nodes that match the user query and to rebuild it during the relevance feedback step. The main goal of this paper is to show the impact of structural hints in XML query optimization. Some experiments have been undertaken into a dataset provided by INEX to show the effectiveness of our proposals.

Since more and more multimedia data associated with spoken documents have been made available to the public, spoken document retrieval (SDR) has become an important research subject in the past two decades. Following the research tendency, many efforts have been devoted towards developing indexing and modeling techniques for representing spoken documents, but only few have been made on improving query formulation for better representing users' information needs. The i-vector based language modeling (IVLM) framework, stemming from the state-of-the-art i-vector framework for language identification and speaker recognition, has been proposed and formulated to represent documents in SDR with good promise recently. However, a major challenge of using IVLM for query modeling is that a query usually consists of only a few words; thus, it is hard to learn a reliable representation accordingly. In this paper, we focus our attention on query reformulation and propose three novel methods on top of IVLM to more accurately represent users' information needs. In addition, we also explore the use of multi-levels of index features, including word- and subword-level units, to work in concert with the proposed methods. A series of empirical SDR experiments conducted on the TDT-2 (Topic Detection and Tracking) collection demonstrate the good effectiveness of our proposed methods as compared to existing state-of-the-art methods.

Content-based web image retrieval based on the query-by-example (QBE) principle remains a challenging problem due to the semantic gap as well as the gap between a user's intent and the representativeness of a typical image query. In this article, we propose to address this problem by integrating query-related contextual information into an advanced query model to improve the performance of QBE-based web image retrieval. We consider both the local and global context of the query image. The local context can be inferred from the web pages and the click-through log associated with the query image, while the global context is derived from the entire corpus comprising all web images and the associated web pages. To effectively incorporate the local query context we propose a language modeling based approach to deal with the combined structured query representation from the contextual and visual information. The global query context is integrated by the multi-modal relevance model to “reconstruct” the query from the document models indexed in the corpus. In this way, the global query context is employed to address the noise or missing information in the query and its local context, so that a comprehensive and robust query model can be obtained. We evaluated the proposed approach on a representative product image dataset collected from the web and demonstrated that the inclusion of the local and global query contexts significantly improves the performance of QBE-based web image retrieval.

Modeling multidimensional relevance in information retrieval (IR) has attracted much attention in recent years. However, most existing studies are conducted through relatively small‐scale user studies, which may not reflect a real‐world and natural search scenario. In this article, we propose to study the multidimensional user relevance model (MURM) on large scale query logs, which record users' various search behaviors (e.g., query reformulations, clicks and dwelling time, etc.) in natural search settings. We advance an existing MURM model (including five dimensions: topicality, novelty, reliability, understandability, and scope) by providing two additional dimensions, that is, interest and habit. The two new dimensions represent personalized relevance judgment on retrieved documents. Further, for each dimension in the enriched MURM model, a set of computable features are formulated. By conducting extensive document ranking experiments on Bing's query logs and TREC session Track data, we systematically investigated the impact of each dimension on retrieval performance and gained a series of insightful findings which may bring benefits for the design of future IR systems.

[1] How long does it take to start a revolution? The past decade has simultaneously hailed the coming of the eBook, declared that the eBook is dead, and greeted the eBook’s resurrection via a multitude of charming devices with sophisticated prices. The National Endowment for the Arts has noted the decline of readers who read for the purpose of pleasure and leisure.1 Wayne Wiegand, historian of libraries and library services, has stated on multiple occasions that there are more libraries in the United States than McDonald’s restaurants.2 Is the book dead or merely in a state of metamorphosis? [2] On the same day the NEA published its report in November 2007, Amazon.com released its eBook reader, the Kindle. A thousand blog posts were launched; bloggers, posters, and reviewers who had used the device declared it useful if clunky, while those who hadn’t declared it fugly or smarvelous (variously). It is difficult to judge a book by its cover when it does not have a cover but rather a leather carrying case. The information becomes a text instead of a book—a means rather than an end to itself. And then, of course, that same collection of blog posts reported both that the book was dead and thus the whole argument moot, and that the book was brand new all over again and better than ever. [3] Salon.com’s Farhad Manjoo evaluated the Kindle. He crunched numbers to determine whether it was worth the price tag for readers:

Today I am happy to announce that the Rogue Scholar science blog archive has joined the Fediverse, the federated social network that communicates using the ActivityPub protocol. I have launched a Mastodon instance at Rogue Scholar Social that accepts Science Blog bots as accounts, publishing summaries of blog posts. Science blogs are typically read by going to the blog homepage with a web browser or using an RSS reader.