Massive-scale analytics (MSA) applications are characterized by the large amount of data that they process and the complexity of algorithms used to process the data. The ideal MSA system will not only support processing of large amounts of data but also offer a high degree of parallelism and support scheduling and resource allocation of complex workloads. Designers of MSA systems must provide three necessities: programming abstractions, runtime systems, and hardware. Historically, two communities have undertaken the task of designing MSA systems: the database community, which has argued for an SQL (Structured Query Language)-influenced processing paradigm, and the high-performance computing community, which has focused on developing infrastructures for highly efficient, but complex, parallel implementations. These two communities have developed disparate technologies to meet the necessities of MSA systems, and the solutions provided by the individual communities are not completely satisfactory. In this paper, we attempt to characterize the strengths and weaknesses of the approaches of these two communities at all levels of the MSA stack, characterize implications with respect to resource management within the MSA system, and define how an MSA system should be designed.

A varied collection of scientific and engineering codes has been adapted and enhanced to take advantage of the IBM Blue Gene®/Q architecture and thus enable research that was previously out of reach. Computational research teams from a number of disciplines collaborated with the staff of the Argonne Leadership Computing Facility to assess which of Blue Gene/Q's many novel features could be exploited for each application to equip it to tackle existing problem classes with greater fidelity and in some cases to address new phenomena. The quad floating-point units and the five-dimensional torus interconnect are among the features that were demonstrated to be effective for a number of important applications. Furthermore, data obtained from the hardware counters provided insights that were valuable in guiding the code modifications. Hardware features and programming techniques that were effective across multiple codes are documented as well. First, we have confirmed that there is no significant code rewrite needed to run today's production codes with good performance on Mira, an IBM Blue Gene/Q supercomputer. Performance improvements are already demonstrated, even though our measurements are all on pre-production software and hardware. The application domains included biology, materials science, combustion, chemistry, nuclear physics, and industrial-scale design of nuclear reactors, jet engines, and the efficiency of transportation systems.

To produce business results, the globally integrated enterprise relies on people to collaborate, make choices, and take action. Though guided by process and enabled by technology, people's behavior is powerfully influenced by leadership and capacity for change. Facing continuing and sometimes large-scale transformation, IBM chose to add organizational change management to its core capabilities. This capability, coupled with a newly centralized worldwide IT (information technology) organization, enables a systemic approach to the adoption of global internal transformations by IBM employees. The Blue Harmony transformation program is the premier example of the IBM focus on organizational change management. It is considered by many to be the most significant operational transformation of IBM in decades and sets the operational foundation for the next 25 years. As part of the longer-term strategy of IBM to integrate the enterprise, it simplifies, integrates, and standardizes operations globally and across business units. When fully implemented, Blue Harmony will change the daily work of nearly 200,000 employees as well as clients, business partners, and suppliers. In such a massive transformation, the challenges of managing change become massive as well. Lessons learned reveal the effort needed to manage stakeholder expectations and engage executives across the enterprise in leading major change.

Weak economic growth, increasing business complexity, and intensifying competition are among the motivations for the growing use of analytics across organizations of all types. In this business environment, enterprise leadership requires new tools and new insights to make better decisions. While analytic techniques, no matter how primitive, have been used to improve the effectiveness and efficiency of organizations for decades, if not centuries, capability and sophistication are now growing rapidly. Advanced analytics is proving to be a transformational technology for businesses, providing a source of competitive advantage and delivering significant, measurable outcomes. Risk management, sales-force productivity, workforce optimization, enterprise planning, and market intelligence are a few examples of core business processes that have significantly benefited from emerging analytical methodologies. As analytic tools used to transform business processes are integrated into enterprise IT (information technology) systems for use by senior business leaders, they are driving new innovations to address the size, speed, and complexity issues faced by large, global enterprises. This position paper explores emerging trends in the leadership application of analytics in global business organizations.

Although the majority of evidence analysis in DeepQA is focused on unstructured information (e.g., natural-language documents), several components in the DeepQA system use structured data (e.g., databases, knowledge bases, and ontologies) to generate potential candidate answers or find additional evidence. Structured data analytics are a natural complement to unstructured methods in that they typically cover a narrower range of questions but are more precise within that range. Moreover, structured data that has formal semantics is amenable to logical reasoning techniques that can be used to provide implicit evidence. The DeepQA system does not contain a single monolithic structured data module; instead, it allows for different components to use and integrate structured and semistructured data, with varying degrees of expressivity and formal specificity. This paper is a survey of DeepQA components that use structured data. Areas in which evidence from structured sources has the most impact include typing of answers, application of geospatial and temporal constraints, and the use of formally encoded a priori knowledge of commonly appearing entity types such as countries and U.S. presidents. We present details of appropriate components and demonstrate their end-to-end impact on the IBM Watson™ system.

The final stage in the IBM DeepQA pipeline involves ranking all candidate answers according to their evidence scores and judging the likelihood that each candidate answer is correct. In DeepQA, this is done using a machine learning framework that is phase-based, providing capabilities for manipulating the data and applying machine learning in successive applications. We show how this design can be used to implement solutions to particular challenges that arise in applying machine learning for evidence-based hypothesis evaluation. Our approach facilitates an agile development environment for DeepQA; evidence scoring strategies can be easily introduced, revised, and reconfigured without the need for error-prone manual effort to determine how to combine the various evidence scores. We describe the framework, explain the challenges, and evaluate the gain over a baseline machine learning approach.

Answering natural-language questions may often involve identifying hidden associations and implicit relationships. In some cases, an explicit question is asked by the user to discover some hidden concept related to a set of entities. Answering the explicit question and identifying the implicit entity both require the system to discover the semantically related but hidden concepts in the question. In this paper, we describe a spreading-activation approach to concept expansion, backed by three distinct knowledge resources for measuring semantic relatedness. We discuss how our spreading-activation approach is applied to address these questions, exemplified in Jeopardy!™ by questions in the “COMMON BONDS” category and by many Final Jeopardy! questions. We demonstrate the effectiveness of the approach by measuring its impact on IBM Watson™ performance on these questions.

In 2007, IBM Research took on the grand challenge of building a computer system that could compete with champions at the game of Jeopardy!™. In 2011, the open-domain question-answering (QA) system, dubbed Watson, beat the two highest ranked players in a nationally televised two-game Jeopardy! match. This paper provides a brief history of the events and ideas that positioned our team to take on the Jeopardy! challenge, build Watson, IBM Watson™, and ultimately triumph. It describes both the nature of the QA challenge represented by Jeopardy! and our overarching technical approach. The main body of this paper provides a narrative of the DeepQA processing pipeline to introduce the articles in this special issue and put them in context of the overall system. Finally, this paper summarizes our main results, describing how the system, as a holistic combination of many diverse algorithmic techniques, performed at champion levels, and it briefly discusses the team's future research plans.

Many questions explicitly indicate the type of answer required. One popular approach to answering those questions is to develop recognizers to identify instances of common answer types (e.g., countries, animals, and food) and consider only answers on those lists. Such a strategy is poorly suited to answering questions from the Jeopardy!™ television quiz show. Jeopardy! questions have an extremely broad range of types of answers, and the most frequently occurring types cover only a small fraction of all answers. We present an alternative approach to dealing with answer types. We generate candidate answers without regard to type, and for each candidate, we employ a variety of sources and strategies to judge whether the candidate has the desired type. These sources and strategies provide a set of type coercion scores for each candidate answer. We use these scores to give preference to answers with more evidence of having the right type. Our question-answering system is significantly more accurate with type coercion than it is without type coercion; these components have a combined impact of nearly 5% on the accuracy of the IBM Watson™ question-answering system.

Access to a large amount of knowledge is critical for success at answering open-domain questions for DeepQA systems such as IBM Watson™. Formal representation of knowledge has the advantage of being easy to reason with, but acquisition of structured knowledge in open domains from unstructured data is often difficult and expensive. Our central hypothesis is that shallow syntactic knowledge and its implied semantics can be easily acquired and can be used in many areas of a question-answering system. We take a two-stage approach to extract the syntactic knowledge and implied semantics. First, shallow knowledge from large collections of documents is automatically extracted. Second, additional semantics are inferred from aggregate statistics of the automatically extracted shallow knowledge. In this paper, we describe in detail what kind of shallow knowledge is extracted, how it is automatically done from a large corpus, and how additional semantics are inferred from aggregate statistics. We also briefly discuss the various ways extracted knowledge is used throughout the IBM DeepQA system.