From 3d bimanual toward distant collaborative interaction techniques: an awareness issue

Virtual Reality (VR) applications often require users to perform actions with two hands when performing tasks and interacting with objects in virtual environments. Although bimanual interactions in VR can resemble real-world interactions—thus increasing realism and improving immersion—they can also pose significant accessibility challenges to people with limited mobility, such as for people who have full use of only one hand. An opportunity exists to create accessible techniques that take advantage of users’ abilities, but designers currently lack structured tools to consider alternative approaches. To begin filling this gap, we propose Two-In-One, a design space that facilitates the creation of accessible methods for bimanual interactions in VR from unimanual input. Our design space comprises two dimensions, bimanual interactions and computer assistance, and we provide a detailed examination of issues to consider when creating new unimanual input techniques that map to bimanual interactions in VR. We used our design space to create three interaction techniques that we subsequently implemented for a subset of bimanual interactions and received user feedback through a video elicitation study with 17 people with limited mobility. Our findings explore complex tradeoffs associated with autonomy and agency and highlight the need for additional settings and methods to make VR accessible to people with limited mobility.

For Collaborative Virtual Environments (CVEs), many interaction techniques have been developed. Depending on the purpose of the collaborative work, techniques of interaction and manipulation change from one application to another. There is no general, good and efficient solution for all the collaborative systems. In addition, people in CVEs also use communication channels to share task goals, task decomposition and task progress. Therefore, awareness and communications are usually considered as important instruments to complete collaborative task. In this paper, we have combined different virtual navigation aids i.e. 3DML + audio, 3DML + textual, 3DML + arrows-casting, arrows-casting + audio, arrows-casting + textual and audio + textual; and presented a comparative study of user performance to perform an assembly task in CVEs. We reported the results of a precise experiment containing, 30 virtual teams of 60 individual students. Overall, results showed that students performed task faster using 3DML + arrows-casting while they were slow with audio + textual support in navigation.

Analysis of raw volume data generated from different scanning technologies faces a variety of challenges, related to search, pattern recognition, spatial understanding, quantitative estimation, and shape description. In a previous study, we found that the Volume Cracker (VC) 3D interaction (3DI) technique mitigated some of these problems, but this result was from a tethered glove-based system with users analyzing simulated data. Here, we redesigned the VC by using untethered bare-hand interaction with real volume datasets, with a broader aim of adoption of this technique in research labs. We developed symmetric and asymmetric interfaces for the Bare-Hand Volume Cracker (BHVC) through design iterations with a biomechanics scientist. We evaluated our asymmetric BHVC technique against standard 2D and widely used 3D interaction techniques with experts analyzing scanned beetle datasets. We found that our BHVC design significantly outperformed the other two techniques. This study contributes a practical 3DI design for scientists, documents lessons learned while redesigning for bare-hand trackers, and provides evidence suggesting that 3D interaction could improve volume data analysis for a variety of visual analysis tasks. Our contribution is in the realm of 3D user interfaces tightly integrated with visualization, for improving the effectiveness of visual analysis of volume datasets. Based on our experience, we also provide some insights into hardware-agnostic principles for design of effective interaction techniques.

In this paper, we discuss interaction interoperability in Collaborative Virtual Environments (CVEs), intended to mean the ability of two or more users to cooperate despite the heterogeneity of their interfaces. To allow such interoperability, rather than focusing on the decoupling of input devices from Interaction Techniques (ITs) and from interaction tasks, we suggest integrating interactive systems at higher level through an interface standardisation. To achieve this aim, we propose: i an architectural model able to handle differences in input devices and interaction tasks ii an agent-based middleware that provides basic components to integrate heterogeneous user interfaces. We also present a prototype of an agent-based middleware able to support developers in the interconnection of monolithic applications and we introduce tools and languages we have used to formalise the interaction tasks considered in the case study.

Empirical evidence of the benefits of immersion is an important goal for the virtual environment (VE) community. Direct comparison of immersive systems and non-immersive systems is insufficient because differences between such systems may be due not only to the level of immersion, but also to other factors, such as the input devices and interaction techniques used. In this paper, a study is presented that separates the effects of level of immersion and 3D interaction technique for a six-degree-of-freedom manipulation task. In the study, two components of immersion -- stereoscopy and field of regard -- were varied and three 3D interaction techniques -- HOMER, Go-Go, and DO-IT (a new keyboard- and mouse-based technique) -- were tested. The results of the experiment show that the interaction technique had a significant effect on object manipulation time, while the two components of immersion did not. The implications of these results are discussed for VE application developers.

In this paper, we present a study to show that matching the dimensionality of interaction techniques with the dimensional demands of the task results in an interface that facilitates superior performance on interaction tasks without sacrificing performance on 2D tasks in favor of 3D tasks and vice versa. We describe the concept of dimensional congruence and how to identify the dimensional characteristics of a task so that appropriate interaction techniques can be applied. We constructed a prototypical application in a Virtual Environment Enclosure (VEE) using a hand-held device to show how this might be done in this type of apparatus. We then describe a study that evaluates both 2D and 3D tasks as performed using typical 2D and 3D interaction techniques. Results show that an appropriate mix of 2D and 3D interaction techniques is preferred over exclusive use of one or the other. The challenge lies not in selecting independent interaction techniques for specific tasks, but rather in constructing an overall interface that mixes 2D and 3D interactions appropriately.

This project aims to present a methodology to study the relationships between physiological measures and evaluation metrics for 3D interaction techniques using methods for multivariate data analysis. Physiological responses, such as heart rate and skin conductance, offer objective data about the user stress during interaction. This could be useful, for instance, to evaluate qualitative aspects of interaction techniques without relying on solely subjective data. Moreover, these data could contribute to improve task performance analysis by measuring different responses to 3D interaction techniques. With this in mind, we propose a methodology that defines a testing protocol, a normalization procedure and statistical techniques, considering the use of physiological measures during the evaluation process. A case study comparison between two 3D interaction techniques (ray-casting and HOMER) shows promising results, pointing to heart rate variability, as measured by the NN50 parameter, as a potential index of task performance. Further studies are needed in order to establish guidelines for evaluation processes based on welldefined associations between human behaviors and human actions realized in 3D user interfaces.

Emerging novel 3D interaction technologies allow precise tracking of bare hands and fingers, but due to the differences between these devices and traditional trackers, it is not clear how to design effective interaction techniques using these technologies. Using the Leap Motion Controller, we designed travel techniques with bare-hand interaction. We prototyped both unimanual and bimanual techniques using various metaphors (e.g., airplane and camera-in-hand), control mappings (position- vs. rate-control), camera movements (scene vs. camera dependent) and methods for speed control. Based on our experiences with these prototypes, we discuss the challenges and design issues for bare-hand interaction techniques. We present the results of a user study comparing the usability of five representative techniques for three travel tasks: absolute travel, naive search and path following. We found that the limited workspace of the Leap caused movements with the camera-in-hand metaphor to be faster and less accurate, making it more effective for search but less effective for path following tasks. In addition, the Leap's ability to precisely track small finger movements benefited the usability of continuous speed control techniques.

Cooperative manipulation refers to the simultaneous manipulation of a virtual object by multiple users in an immersive virtual environment (VE). In this work, we present techniques for cooperative manipulation based on existing single-user techniques. We discuss methods of combining simultaneous user actions, based on the separation of degrees of freedom between two users, and the awareness tools used to provide the necessary knowledge of the partner activities during the cooperative interaction process. We also present a framework for supporting the development of cooperative manipulation techniques, which are based on rules for combining single user interaction techniques. Finally, we report an evaluation of cooperative manipulation scenarios, the results indicating that, in certain situations, cooperative manipulation is more efficient and usable than singleuser manipulation.

Real-time 3D interaction with augmented reality environments is one of the main features of any AR system. However, selecting and manipulating distal 3D virtual objects in AR still suffer from lack of accuracy and precision. In this paper, we propose an alternate 3D interaction technique called for selection and manipulation distant objects in immersive video see-through augmented reality. Zoom-in interaction technique is based on the idea of zooming the captured images. This allows bringing closer both of real and virtual distant objects, while keeping the spatial registration between the virtual and the real scenes thanks to a robust real-time computer vision algorithm for pose estimation. An evaluation and comparison with other well-known technique are given at the end of this paper, in order to validate our proposed approach.

Balloon selection is a 3D interaction technique that is modeled after the real world metaphor of manipulating a helium balloon attached to a string. Balloon selection allows for precise 3D selection in the volume above a tabletop surface by using multiple fingers on a multi-touch-sensitive surface. The 3DOF selection tasks is decomposed in part into a 2DOF positioning task performed by one finger on the tabletop in an absolute 2D Cartesian coordinate system and a 1DOF positioning task performed by another finger on the tabletop in a relative 2D polar coordinate system. We have evaluated balloon selection in a formal user study that compared it to two well-known interaction techniques for selecting a static 3D target: a 3DOF tracked wand and keyboard cursor keys. We found that balloon selection was significantly faster than using cursor keys and had a significantly lower error rate than the wand. The lower error rate appeared to result from the user's hands being supported by the tabletop surface, resulting in significantly reduced hand tremor and arm fatigue.

Dual-Thumb pointing and command selection techniques for tablets: Enhancing Distant Interaction with Large Displays using a Two-Handed Tablet

Analysis of volume datasets often involves peering inside the volume to understand internal structures. Traditional approaches involve removing part of the volume through slicing, but this can result in the loss of context. Focus+context visualization techniques can distort part of the volume, or can assume prior definition of a region of interest or segmentation of layers of the volume. We propose a new bimanual 3D interaction technique, called Volume Cracker (VC), which allows the user to crack open a raw volume like a book to analyze the internal structures. VC preserves context by always displaying all the voxels, and by connecting the sub-volumes with curves joining the cracked faces. We discuss the design choices that we made, based on observations from prior user studies, input from domain scientists, and design studios. We also report the results of a user study comparing VC with a standard desktop interaction technique and a standard 3D bimanual interaction technique. The study used tasks from two categories of a generic volume analysis task taxonomy. We found VC had significant advantages over the other two techniques for search and pattern recognition tasks.

The CubTile is a cubic-shaped device with 5 tactile multi-touch faces. It was initially designed for 3D interaction. In this article we explore its use for navigation and pointing in a 2D space. In this way, we propose a bi-manual interaction technique, based on the manipulation of two faces of the CubTile, in a focus+context interface: The non-dominant hand pans the focus with one face of the CubTile while the dominant hand points in the focus area using another face of the CubTile. The results of a first user experiment indicate that an asymmetric tuning, with a low amplification for the non-dominant hand gestures and a high amplification for the dominant hand gestures, provides better performance in a pointing task. These results are a first step towards optimal tuning of the tactile faces for asymmetric bimanual interaction.

Palpation is a physical examination technique where objects, e.g., organs or body parts, are touched with fingers to determine their size, shape, consistency and location. Many medical procedures utilize palpation as a supplementary interaction technique and it can be therefore considered as an essential basic method. However, palpation is mostly neglected in medical training simulators, with the exception of very specialized simulators that solely focus on palpation, e.g., for manual cancer detection. In this article we propose a novel approach to enable haptic palpation interaction for virtual reality-based medical simulators. The main contribution is an extensive user study conducted with a large group of medical experts. To provide a plausible simulation framework for this user study, we contribute a novel and detailed interaction algorithm for palpation with tissue dragging, which utilizes a multi-object force algorithm to support multiple layers of anatomy and a pulse force algorithm for simulation of an arterial pulse. Furthermore, we propose a modification for an off–the–shelf haptic device by adding a lightweight palpation pad to support a more realistic finger grip configuration for palpation tasks. The user study itself has been conducted on a medical training simulator prototype with a specific procedure from regional anesthesia, which strongly depends on palpation. The prototype utilizes a co-rotational finite-element approach for soft tissue simulation and provides bimanual interaction by combining the aforementioned techniques with needle insertion for the other hand. The results of the user study suggest reasonable face validity of the simulator prototype and in particular validate medical plausibility of the proposed palpation interaction algorithm.