ProactiveVA: Proactive Visual Analytics with LLM-Based UI Agent.

The 21st century will be heavily impacted by the capability to create value from recorded data of all kinds. In this regard, industrial manufacturing intuitions increasingly rely on new data-driven technologies, such as the Internet of Things or Machine Learning. In terms of data collection, manufacturing processes increasingly include sophisticated sensor equipment, which results in interconnected networks of manufacturing all assembling parts and producing data. However, manufacturing institutions currently face two challenges. First, large amounts of parts and hence data are produced during fully automated manufacturing processes. Second, due to the overwhelming amount of recorded data, it is particularly challenging to efficiently analyze manufacturing data. Hence, it is important to efficiently store and share gained knowledge from performed data analyses. Information visualization and Visual Analytics are two prolific branches of data analysis exploiting sophisticated visualization techniques to support the execution of analytical tasks and to store gained knowledge. This thesis looks at how data visualization approaches can help industrial manufacturing organizations to create value from large amounts of manufacturing data, as well as how to efficiently store and share knowledge from manufacturing data analyses. The goal is to understand how Visual Analytics can improve manufacturing processes in the context of knowledge management. Five Visual Analytics systems were designed, developed, and evaluated to tackle different domain problems that emerged from manufacturing setups. Findings from each system were used to carry out additional studies to enhance established theories of knowledge management. As a result, five success stories are provided of how Visual Analytics can significantly improve manufacturing processes and how knowledge can be efficiently created, formalized, and shared in an organization with Visual Analytics.

This book discusses semantic interaction, a user interaction methodology for visual analytic applications that more closely couples the visual reasoning processes of people with the computation. This methodology affords user interaction on visual data representations that are native to the domain of the data. User interaction in visual analytics systems is critical to enabling visual data exploration. Interaction transforms people from mere viewers to active participants in the process of analyzing and understanding data. This discourse between people and data enables people to understand aspects of their data, such as structure, patterns, trends, outliers, and other properties that ultimately result in insight. Through interacting with visualizations, users engage in sensemaking, a process of developing and understanding relationships within datasets through foraging and synthesis. The book provides a description of the principles of semantic interaction, providing design guidelines for the integration of semantic interaction into visual analytics, examples of existing technologies that leverage semantic interaction, and a discussion of how to evaluate these technologies. Semantic interaction has the potential to increase the effectiveness of visual analytic technologies and opens possibilities for a fundamentally new design space for user interaction in visual analytics systems.

The practice of data science is ad hoc and agile, where needs and requirements evolve continuously and are resolved through collaboration among stakeholders. To support such practice visual analytics systems need to evolve as the needs and requirements in terms of data, users, tasks, medium, visualization, and interaction capabilities change continuously. In this paper we present a case study and illustrate several dimensions of the requirements in visual analytics. We put forward a vision of a dynamic agile visual analytics process and system model in support of data science, in which the user and system can cooperate to facilitate discovery while requirements change on demand. We argue that such a system needs an underlying language and algebra that defines not only operands and operators for performing visual analytics but also specifies guidelines that take them into account and produce useful visual analytics transformations leading to a specific insight. Our intent is not to present a fully developed system but rather a vision, illustrated through a use case. While the algebra presented here is sufficiently defined to illustrate our viewpoints, further refinement and completion is necessary to facilitate application in a visual analytics system.

In this article, we develop guidelines for evaluating visual analytics environments based on a synthesis of reviews for the entries to the 2009 Visual Analytics Science and Technology (VAST) Symposium Challenge and from a user study with professional intelligence analysts. By analyzing the 2009 VAST Challenge reviews, we gained a better understanding of what is important to our reviewers, both visualization researchers and professional analysts. We also report on a small user study with professional analysts to determine the important factors that they use in evaluating visual analysis systems. We also looked at guidelines developed by researchers in various domains and synthesized the results from these three efforts into an initial set for use by others in the community. One challenge for future visual analytics systems is to help in the generation of reports. In our user study, we also worked with analysts to understand the criteria they used to evaluate the quality of analytic reports. We propose that this knowledge will be useful as researchers look at systems to automate some of the report generation.1From these two efforts, we produced some initial guidelines for evaluating visual analytics environments and for the evaluation of analytic reports. It is important to understand that these guidelines are initial drafts and are limited in scope as the visual analytics systems we evaluated were used in specific tasks. We propose these guidelines as a starting point for the Visual Analytics Community.

Because of the rapid growth of the web and its users, web usability and accessibility become more and more important. This chapter describes two automated methods for evaluating usability/accessibility of webpages. The first method, for usability evaluation, is based on interaction logging and analyses, and the second method, for accessibility evaluation, is based on machine learning. In the following Sections 2 and 3, the two methods are described respectively. Several methods have been proposed and developed for usability evaluation based on user interaction logs. Interaction logs can be recorded by computer logging programs (automated logging) or by human evaluators (observational logging). Analysis methods and tools for the former type of logs are well summarized by Ivory (Ivory & Hearst, 2001; Ivory, 2003), and those for the latter type of logs have also been developed (for example, by Qiang (Qiang et al., 2005)). Our method is for analyzing former type of logs. In the survey by Ivory, analysis methods for automatically captured log files for WIMP (Window, Icon, Menu and Pointing device) applications and web applications are categorized in terms of their approaches including metric-based, pattern-based, task-based and inferential ones. Some of the methods with the task-based approach compare user interaction logs for a test task with desired (expected) interaction sequences for the task and detect inconsistencies between the user logs and the desired sequences (Kishi, 1995; Uehling & Wolf, 1995; Okada & Asahi, 1996; Al-Qaimari & Mcrostie, 1999; Helfrich & Landay, 1999; Zettlemoyer et al., 1999). The inconsistencies are useful cues for finding usability problems: for example, an evaluator can find that users selected some unexpected link on a webpage when another link on the page was expected for the test task and that the link selected by the users may have some usability problem in its design (labeling, layout, etc.). The existing methods require widget-level logs for the comparisons. For example, the method proposed by Okada (Okada & Asahi, 1996) requires interaction logs to include data of widget properties such as widget label, widget type, title of parent window, etc. This requirement degrades independency and completeness of the methods in logging user interactions with systems under evaluation. Section 2 in this chapter describes our method that detects inconsistencies between user logs and desired sequences based on logs of clicked points ((x, y) coordinate values). Coordinate values of clicked points can be easily and fully logged independently of what widgets are clicked on. Several existing methods have also

The reconstruction of analysts’ reasoning processes ( reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos’s data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

Image and video data analysis has become an increasingly important research area with applications in different domains such as security surveillance, healthcare, augmented and virtual reality, video and image editing, activity analysis and recognition, synthetic content generation, distance education, telepresence, remote sensing, sports analytics, art, non-photorealistic rendering, search engines, and social media. Recent advances in Artificial Intelligence (AI) and particularly deep learning have sparked new research challenges and led to significant advancements, especially in image and video analysis. These advancements have also resulted in significant research and development in other areas such as visualization and visual analytics, and have created new opportunities for future lines of research. In this survey article, we present the current state of the art at the intersection of visualization and visual analytics, and image and video data analysis. We categorize the visualization articles included in our survey based on different taxonomies used in visualization and visual analytics research. We review these articles in terms of task requirements, tools, datasets, and application areas. We also discuss insights based on our survey results, trends and patterns, the current focus of visualization research, and opportunities for future research.

Several visual analytics (VA) systems are used for analyzing eye-tracking data because they synergize human-in-the-loop exploration with speed and accuracy of the computer. In the VA systems, the choices of visualization techniques could afford discovering certain types of insights while hindering others. Understanding these affordances and hindrances is essential to design effective VA systems. In this paper, we focus on two approaches for visualizing AOI transitions: the transition-based approach (exemplified by the radial transition graph, RTG) and the sequence-based approach (exemplified by the Alpscarf). We captured the insights generated by two analysts who individually use each visualization technique on the same dataset. Based on the results, we identify four phases of analytic activities and discuss opportunities that the two visualization approaches can complement each other. We point out design implications for VA systems that combine these visualization approaches.

We present research focused on designing visual analytics (VA) systems for workers in organizational environments. We focus on business analysts and asset managers, who work collaboratively to analyze information and make decisions. Through extensive investigations in two organizational environments, we found that these users struggle with managing and analyzing information from multiple perspectives. Their current tools lack support for aggregating, organizing, and sharing such information. To address their needs, we characterized their analytic workflows, extracted specific key knowledge actions for each task commonly found in these workflows, and designed and evaluated two visual analytics systems that support and encapsulate these knowledge actions. We provide design guidelines that should be used when designing visual analytics systems, and illustrate their effectiveness with two systems built by following them.

User interaction logs allow us to analyze the execution of tasks in a business process at a finer level of granularity than event logs extracted from enterprise systems. The fine-grained nature of user interaction logs open up a number of use cases. For example, by analyzing such logs, we can identify best practices for executing a given task in a process, or we can elicit differences in performance between workers or between teams. Furthermore, user interaction logs allow us to discover repetitive and automatable routines that occur during the execution of one or more tasks in a process. Along this line, this chapter introduces a family of techniques, called Robotic Process Mining (RPM), which allow us to discover repetitive routines that can be automated using robotic process automation technology. The chapter presents a structured landscape of concepts and techniques for RPM, including techniques for user interaction log preprocessing, techniques for discovering frequent routines, notions of routine automatability, as well as techniques for synthesizing executable routine specifications for robotic process automation.

Visual Analytics provides with a combination of automated techniques and interactive visualizations huge analysis possibilities in technology and innovation management. Thereby not only the use of machine learning data mining methods plays an important role. Due to the high interaction capabilities, it provides a more user-centered approach, where users are able to manipulate the entire analysis process and get the most valuable information. Existing Visual Analytics systems for Trend Analytics and technology and innovation management do not really make use of this unique feature and almost neglect the human in the analysis process. Outcomes from research in information search, information visualization and technology management can lead to more sophisticated Visual Analytics systems that involved the human in the entire analysis process. We propose in this paper a new interaction approach for Visual Analytics in technology and innovation management with a special focus on technological trend analytics.

This paper firstly provides a general introduction in the most important aspects and ideas of Visual Analytics. This multidisciplinary field focuses on the analytical reasoning of typically large and complex (often heterogeneous) data sets and combines techniques from interactive visualizations with computational analysis methods. Hereby, intuitive and efficient user interactions are a fundamental component which has to be efficiently supported by any Visual Analytics system. This integration of interaction techniques into both visual representations and automatic analysis methods supports the human-information discourse and can be realized in various ways which is discussed in the second part of the paper. We give examples of possible applications of Visual Analytics from the domain of biological simulations and highlight the importance and role of the human in the analysis loop.

A theoretical model for pattern discovery in visual analytics

Data-driven approaches to problem solving and data analysis are becoming more and more important problems to consider and on which apply research ideas. In this respect, the capability to explore data, understands how algorithmic approaches work and steer them toward the desired goals make Visualization and Visual Analytics strong research fields in which to invest efforts. While this importance has been understood by several countries (e.g., USA, Germany, France) in Italy the research efforts in these fields are still disjointed. With the first edition of the ITA.WA. (Italian Visualization & Visual Analytics) workshop the goal is to make a step toward the creation of an Italian research community on these topics, allowing identification of research direction, joining forces in achieving them and developing common guidelines and programs for teaching activities.

Visual analytics has been widely studied in the past decade both in academia and industry to improve data exploration, minimize the overall cost, and improve data analysis. In this chapter, we explore the idea of visual analytics in the context of simulation data. This would then provide us with the capability to not only explore our data visually but also to apply machine learning models in order to answer high-level questions with respect to scheduling, choosing optimal simulation parameters, finding correlations, etc. More specifically, we examine state-of-the-art tools to be able to perform these above-mentioned tasks. Further, to test and validate our methodology we followed the human-centered design process to build a prototype tool called ViDAS (Visual Data Analytics of Simulated Data). Our preliminary evaluation study illustrates the intuitiveness and ease-of-use of our approach with regards to visual analysis of simulated data.