Semantic Interaction for Visual Analytics: Inferring Analytical Reasoning for Model Steering

The 21st century will be heavily impacted by the capability to create value from recorded data of all kinds. In this regard, industrial manufacturing intuitions increasingly rely on new data-driven technologies, such as the Internet of Things or Machine Learning. In terms of data collection, manufacturing processes increasingly include sophisticated sensor equipment, which results in interconnected networks of manufacturing all assembling parts and producing data. However, manufacturing institutions currently face two challenges. First, large amounts of parts and hence data are produced during fully automated manufacturing processes. Second, due to the overwhelming amount of recorded data, it is particularly challenging to efficiently analyze manufacturing data. Hence, it is important to efficiently store and share gained knowledge from performed data analyses. Information visualization and Visual Analytics are two prolific branches of data analysis exploiting sophisticated visualization techniques to support the execution of analytical tasks and to store gained knowledge. This thesis looks at how data visualization approaches can help industrial manufacturing organizations to create value from large amounts of manufacturing data, as well as how to efficiently store and share knowledge from manufacturing data analyses. The goal is to understand how Visual Analytics can improve manufacturing processes in the context of knowledge management. Five Visual Analytics systems were designed, developed, and evaluated to tackle different domain problems that emerged from manufacturing setups. Findings from each system were used to carry out additional studies to enhance established theories of knowledge management. As a result, five success stories are provided of how Visual Analytics can significantly improve manufacturing processes and how knowledge can be efficiently created, formalized, and shared in an organization with Visual Analytics.

The practice of data science is ad hoc and agile, where needs and requirements evolve continuously and are resolved through collaboration among stakeholders. To support such practice visual analytics systems need to evolve as the needs and requirements in terms of data, users, tasks, medium, visualization, and interaction capabilities change continuously. In this paper we present a case study and illustrate several dimensions of the requirements in visual analytics. We put forward a vision of a dynamic agile visual analytics process and system model in support of data science, in which the user and system can cooperate to facilitate discovery while requirements change on demand. We argue that such a system needs an underlying language and algebra that defines not only operands and operators for performing visual analytics but also specifies guidelines that take them into account and produce useful visual analytics transformations leading to a specific insight. Our intent is not to present a fully developed system but rather a vision, illustrated through a use case. While the algebra presented here is sufficiently defined to illustrate our viewpoints, further refinement and completion is necessary to facilitate application in a visual analytics system.

In this article, we develop guidelines for evaluating visual analytics environments based on a synthesis of reviews for the entries to the 2009 Visual Analytics Science and Technology (VAST) Symposium Challenge and from a user study with professional intelligence analysts. By analyzing the 2009 VAST Challenge reviews, we gained a better understanding of what is important to our reviewers, both visualization researchers and professional analysts. We also report on a small user study with professional analysts to determine the important factors that they use in evaluating visual analysis systems. We also looked at guidelines developed by researchers in various domains and synthesized the results from these three efforts into an initial set for use by others in the community. One challenge for future visual analytics systems is to help in the generation of reports. In our user study, we also worked with analysts to understand the criteria they used to evaluate the quality of analytic reports. We propose that this knowledge will be useful as researchers look at systems to automate some of the report generation.1From these two efforts, we produced some initial guidelines for evaluating visual analytics environments and for the evaluation of analytic reports. It is important to understand that these guidelines are initial drafts and are limited in scope as the visual analytics systems we evaluated were used in specific tasks. We propose these guidelines as a starting point for the Visual Analytics Community.

With the growing need for visualization to aid users in understanding large, complex datasets, the ability for users to interact and explore these datasets is critical. As visual analytic systems have advanced to leverage powerful computational models and data analytics capabilities, the modes by which users engage and interact with the information are limited. Often, users are taxed with directly manipulating parameters of these models through traditional GUIs (e.g., using sliders to directly manipulate the value of a parameter). However, the purpose of user interaction in visual analytic systems is to enable visual data exploration -- where users can focus on their task, as opposed to the tool or system. As a result, users can engage freely in data exploration and decision-making, for the purpose of gaining insight. In this position paper, we discuss how evaluating visual analytic systems can be approached through user interaction analysis, where the goal is to minimize the cognitive translation between the visual metaphor and the mode of interaction (i.e., reducing the "interaction junk"). We motivate this concept through a discussion of traditional GUIs used in visual analytics for direct manipulation of model parameters, and the importance of designing interactions the support visual data exploration.

Visual analytics (VA) is typically applied to complex data, thus requiring complex tools. While visual analytics empowers analysts in data analysis, analysts may get lost in the complexity occasionally. This highlights the need for intelligent assistance mechanisms. However, even the latest LLM-assisted VA systems only provide help when explicitly requested by the user, making them insufficiently intelligent to offer suggestions when analysts need them the most. We propose a ProactiveVA framework in which LLM-powered UI agent monitors user interactions and delivers context-aware assistance proactively. To design effective proactive assistance, we first conducted a formative study analyzing help-seeking behaviors in user interaction logs, identifying when users need proactive help, what assistance they require, and how the agent should intervene. Based on this analysis, we distilled key design requirements in terms of intent recognition, solution generation, interpretability and controllability. Guided by these requirements, we develop a three-stage UI agent pipeline including perception, reasoning, and acting. The agent autonomously perceives users' needs from VA interaction logs, providing tailored suggestions and intuitive guidance through interactive exploration of the system. We implemented the framework in two representative types of VA systems, demonstrating its generalizability, and evaluated the effectiveness through an algorithm evaluation, case and expert study and a user study. We also discuss current design trade-offs of proactive VA and areas for further exploration.

We present research focused on designing visual analytics (VA) systems for workers in organizational environments. We focus on business analysts and asset managers, who work collaboratively to analyze information and make decisions. Through extensive investigations in two organizational environments, we found that these users struggle with managing and analyzing information from multiple perspectives. Their current tools lack support for aggregating, organizing, and sharing such information. To address their needs, we characterized their analytic workflows, extracted specific key knowledge actions for each task commonly found in these workflows, and designed and evaluated two visual analytics systems that support and encapsulate these knowledge actions. We provide design guidelines that should be used when designing visual analytics systems, and illustrate their effectiveness with two systems built by following them.

Several visual analytics (VA) systems are used for analyzing eye-tracking data because they synergize human-in-the-loop exploration with speed and accuracy of the computer. In the VA systems, the choices of visualization techniques could afford discovering certain types of insights while hindering others. Understanding these affordances and hindrances is essential to design effective VA systems. In this paper, we focus on two approaches for visualizing AOI transitions: the transition-based approach (exemplified by the radial transition graph, RTG) and the sequence-based approach (exemplified by the Alpscarf). We captured the insights generated by two analysts who individually use each visualization technique on the same dataset. Based on the results, we identify four phases of analytic activities and discuss opportunities that the two visualization approaches can complement each other. We point out design implications for VA systems that combine these visualization approaches.

Image and video data analysis has become an increasingly important research area with applications in different domains such as security surveillance, healthcare, augmented and virtual reality, video and image editing, activity analysis and recognition, synthetic content generation, distance education, telepresence, remote sensing, sports analytics, art, non-photorealistic rendering, search engines, and social media. Recent advances in Artificial Intelligence (AI) and particularly deep learning have sparked new research challenges and led to significant advancements, especially in image and video analysis. These advancements have also resulted in significant research and development in other areas such as visualization and visual analytics, and have created new opportunities for future lines of research. In this survey article, we present the current state of the art at the intersection of visualization and visual analytics, and image and video data analysis. We categorize the visualization articles included in our survey based on different taxonomies used in visualization and visual analytics research. We review these articles in terms of task requirements, tools, datasets, and application areas. We also discuss insights based on our survey results, trends and patterns, the current focus of visualization research, and opportunities for future research.

This paper firstly provides a general introduction in the most important aspects and ideas of Visual Analytics. This multidisciplinary field focuses on the analytical reasoning of typically large and complex (often heterogeneous) data sets and combines techniques from interactive visualizations with computational analysis methods. Hereby, intuitive and efficient user interactions are a fundamental component which has to be efficiently supported by any Visual Analytics system. This integration of interaction techniques into both visual representations and automatic analysis methods supports the human-information discourse and can be realized in various ways which is discussed in the second part of the paper. We give examples of possible applications of Visual Analytics from the domain of biological simulations and highlight the importance and role of the human in the analysis loop.

Collaborative visual analytics (CVA) involves sensemaking activities within teams of analysts based on coordination of work across team members, awareness of team activity, and communication of hypotheses, observations, and insights. We introduce a new type of CVA tools based on the notion of “team-first” visual analytics, where supporting the analytical process and needs of the entire team is the primary focus of the graphical user interface before that of the individual analysts. To this end, we present the design space and guidelines for team-first tools in terms of conveying analyst presence, focus, and activity within the interface. We then introduce InsightsDrive, a CVA tool for multidimensional data, that contains team-first features into the interface through group activity visualizations. This includes (1) in-situ representations that show the focus regions of all users integrated in the data visualizations themselves using color-coded selection shadows, as well as (2) ex-situ representations showing the data coverage of each analyst using multidimensional visual representations. We conducted two user studies, one with individual analysts to identify the affordances of different visual representations to inform data coverage, and the other to evaluate the performance of our team-first design with exsitu and in-situ awareness for visual analytic tasks. Our results give an understanding of the performance of our team-first features and unravel their advantages for team coordination.

Visual Analytics provides with a combination of automated techniques and interactive visualizations huge analysis possibilities in technology and innovation management. Thereby not only the use of machine learning data mining methods plays an important role. Due to the high interaction capabilities, it provides a more user-centered approach, where users are able to manipulate the entire analysis process and get the most valuable information. Existing Visual Analytics systems for Trend Analytics and technology and innovation management do not really make use of this unique feature and almost neglect the human in the analysis process. Outcomes from research in information search, information visualization and technology management can lead to more sophisticated Visual Analytics systems that involved the human in the entire analysis process. We propose in this paper a new interaction approach for Visual Analytics in technology and innovation management with a special focus on technological trend analytics.

Currently, growing data sources and long-running algorithms impede user attention and interaction with visual analytics applications. Progressive visualization (PV) and visual analytics (PVA) alleviate this problem by allowing immediate feedback and interaction with large datasets and complex computations, avoiding waiting for complete results by using partial results improving with time. Yet, creating a progressive visualization requires more effort than a regular visualization but also opens up new possibilities, such as steering the computations towards more relevant parts of the data, thus saving computational resources. However, there is currently no comprehensive overview of the design space for progressive visualization systems. We surveyed the related work of PV and derived a new taxonomy for progressive visualizations by systematically categorizing all PV publications that included visualizations with progressive features. Progressive visualizations can be categorized by well-known visualization taxonomies, but we also found that progressive visualizations can be distinguished by the way they manage their data processing, data domain, and visual update. Furthermore, we identified key properties such as uncertainty, steering, visual stability, and real-time processing that are significantly different with progressive applications. We also collected evaluation methodologies reported by the publications and conclude with statistical findings, research gaps, and open challenges.

Time is an important data dimension with distinct characteristics that is common across many application domains. This demands specialized methods in order to support proper analysis and visualization to explore trends, patterns, and relationships in different kinds of time-oriented data. The human perceptual system is highly sophisticated and specifically suited to spot visual patterns. For this reason, visualization is successfully applied in aiding these tasks. But facing the huge volumes of data to be analyzed today, applying purely visual techniques is often not sufficient. Visual analytics systems aim to bridge this gap by combining both, interactive visualization and computational analysis. In this paper, we introduce a concept for designing visual analytics frameworks and tailored visual analytics systems for time and time-oriented data. We present a number of relevant design choices and illustrate our concept by example.

Visual analytic systems, especially mixed-initiative systems, can steer analytical models and adapt views by making inferences from users’ behavioral patterns with the system. Because such systems rely on incorporating implicit and explicit user feedback, they are particularly susceptible to the injection and propagation of human biases. To ultimately guard against the potentially negative effects of systems biased by human users, we must first qualify what we mean by the term bias. Thus, in this chapter we describe four different perspectives on human bias that are particularly relevant to visual analytics. We discuss the interplay of human and computer system biases, particularly their roles in mixed-initiative systems. Given that the term bias is used to describe several different concepts, our goal is to facilitate a common language in research and development efforts by encouraging researchers to mindfully choose the perspective(s) considered in their work.

PurposeVisual analytics is increasingly becoming a prominent technology for organizations seeking to gain knowledge and actionable insights from heterogeneous and big data to support decision-making. Whilst a broad range of visual analytics platforms exists, limited research has been conducted to explore the specific factors that influence their adoption in organizations. The purpose of this paper is to develop a framework for visual analytics adoption that synthesizes the factors related to the specific nature and characteristics of visual analytics technology.Design/methodology/approachThis study applies a directed content analysis approach to online evaluation reviews of visual analytics platforms to identify the salient determinants of visual analytics adoption in organizations from the standpoint of practitioners. The online reviews were gathered from Gartner.com, and included a sample of 1,320 reviews for six widely adopted visual analytics platforms.FindingsBased on the content analysis of online reviews, 34 factors emerged as key predictors of visual analytics adoption in organizations. These factors were synthesized into a conceptual framework of visual analytics adoption based on the diffusion of innovations theory and technology–organization–environment framework. The findings of this study demonstrated that the decision to adopt visual analytics technologies is not merely based on the technological factors. Various organizational and environmental factors have also significant influences on visual analytics adoption in organizations.Research limitations/implicationsThis study extends the previous work on technology adoption by developing an adoption framework that is aligned with the specific nature and characteristics of visual analytics technology and the factors involved to increase the utilization and business value of visual analytics in organizations.Practical implicationsThis study highlights several factors that organizations should consider to facilitate the broad adoption of visual analytics technologies among IT and business professionals.Originality/valueThis study is among the first to use the online evaluation reviews to systematically explore the main factors involved in the acceptance and adoption of visual analytics technologies in organizations. Thus, it has potential to provide theoretical foundations for further research in this important and emerging field. The development of an integrative model synthesizing the salient determinants of visual analytics adoption in enterprises should ultimately allow both information systems researchers and practitioners to better understand how and why users form perceptions to accept and engage in the adoption of visual analytics tools and applications.