SurfShare

This paper investigates how different collaboration styles and view independence affect remote collaboration. Our remote collaboration system shares a live video of a local user’s real-world task space with a remote user. The remote user can have an independent view or a dependent view of a shared real-world object manipulation task and can draw virtual annotations onto the real-world objects as a visual communication cue. With the system, we investigated two different collaboration styles; (1) remote expert collaboration where a remote user has the solution and gives instructions to a local partner and (2) mutual collaboration where neither user has a solution but both remote and local users share ideas and discuss ways to solve the real-world task. In the user study, the remote expert collaboration showed a number of benefits over the mutual collaboration. With the remote expert collaboration, participants had better communication from the remote user to the local user, more aligned focus between participants, and the remote participants’ feeling of enjoyment and togetherness. However, the benefits were not always apparent at the local participants’ end, especially with measures of enjoyment and togetherness. The independent view also had several benefits over the dependent view, such as allowing remote participants to freely navigate around the workspace while having a wider fully zoomed-out view. The benefits of the independent view were more prominent in the mutual collaboration than in the remote expert collaboration, especially in enabling the remote participants to see the workspace.

Modeling physiological and perceived responses in remote collaborative virtual environments: A comparative study of HMD and desktop platforms

We present a novel Mixed Reality (MR) remote collaboration system that integrates a telepresence drone to overcome viewpoint limitations in previous approaches. Current remote collaboration systems often restrict remote users to the local user's perspective or a fixed viewpoint, which may reduce spatial awareness and collaboration effectiveness. Our proposed system enables a remote user wearing a VR headset to independently navigate the local environment through a drone while guiding a local AR headset user. The drone represents the remote user physically and virtually with an AR avatar. The system provides the remote user with a 3D reconstruction of the local environment, the local user's avatar, and real-time drone camera video to facilitate drone operation and collaboration. A user study comparing our drone-enhanced MR system with 2D video and 360-degree video MR systems during inspection tasks revealed that the telepresence drone significantly improved collaboration efficiency and enhanced users' social presence and spatial presence. However, the remote users perceived a higher workload. Our findings demonstrate the benefits of independent spatial navigation through drones in remote collaboration and offer insights for future drone-enhanced MR remote collaboration systems.

Remote collaboration has been studied for more than two decades and now there is the possibilities for new types of collaboration with the recent advances in immersive technologies such as Virtual, Augmented, Mixed Reality (VR/AR/MR). However, despite the increasing research interest in remote collaboration study with VR/AR/MR technologies, there is still a lack of academic venues specifically focusing on VR/AR/MR remote collaboration research. This special issue provides high-quality papers on the topic of remote collaboration research and increases visibility of this timely interesting and important research area. We particularly focus on three research aspects in remote collaboration: (1) use of multimodal communication cues, (2) awareness of the task space, and (3) human factors understanding. In this editorial, we first describe five essential factors for remote collaboration: task, local user, remote user, communication, and tool/interface, and then summarize a brief history of the research areas. We also cover the feature papers accepted in this issue, which introduce novel multimodal interfaces for remote collaboration and the effects on task performance and perceptual factors. Finally, we discuss some potential future research directions while concluding the editorial.

We describe an augmented reality conferencing system which uses the overlay of virtual images on the real world. Remote collaborators are represented on virtual monitors which can be freely positioned about a user in space. Users can collaboratively view and interact with virtual objects using a shared virtual whiteboard. This is possible through precise virtual image registration using fast and accurate computer vision techniques and head mounted display (HMD) calibration. We propose a method for tracking fiducial markers and a calibration method for optical see-through HMD based on the marker tracking.

This study presents Portal Display, a screen-based telepresence system that mediates the interaction between two distinct spaces, each using a single display system. The system synchronizes the users’ viewpoint with their head position and orientation to provide stereoscopic vision through this single monitor. This research evaluates the impact of graphically rendered and video-streamed backgrounds and remote user representations on social telepresence, usability, and concentration during conversations and collaborative tasks. Our results indicate that the type of background has a negligible impact on these metrics. However, point cloud streaming of remote users significantly improves social telepresence, usability, and concentration compared with graphical avatars. This study implies that Portal Display can operate more efficiently by substituting the background with graphical rendering and focusing on higher-resolution 3D point cloud streaming for narrower regions for remote user representations. This configuration may be especially advantageous for applications where the remote user’s background is not essential to the task, potentially enhancing social telepresence.

Situation-dependent remote AR collaborations: Image-based collaboration using a 3D perspective map and live video-based collaboration with a synchronized VR mode

The correct distance perception is important for executing various interactive tasks such as navigation, selection and manipulation. It is known, however, that, in general, there is a significant distance perception compression in virtual environments, mainly when using Head-Mounted Displays -- HMDs. This perceived distance compression may bring various problems to the applications and even hinder the utility of those applications that depends on the correct judgment of distances. The scientific community, so far, have not been able to determine the causes of the distance perception compression in virtual environments. For this reason, it was the objective of this work to investigate, through user experiments, the influence of both the field-of-view -- FoV -- and the distance estimation methods on this perceived compression. For that, we conducted an experimental study with 32 participants, in which we compared the my3D device and a HMD trying to determine the causes of the compressed perception. The results showed that user interacting with my3D reported worst distance estimations, on average, when compared to the results of the same users interacting with a HMD, for both tested estimation methods. The causes of that are believed to be the incorrect stimulus of the peripheral vision of the user, the smaller FoV and the smaller immersion sense, as described by the participants of the experiment.

Collaborative Augmented Reality (CAR) offers disruptive ways for people to collaborate. However, this emerging technology must improve its acceptance, efficiency, and usability to scale up and, for example, support augmented operations executed by technicians. This paper presents our CAR system and its experimentation during a cooperative puzzle-solving task. Our system provides collaborators with a shared virtual space allowing verbal and non-verbal interpersonal communications, and intuitive interactions with shared virtual replicas of real objects. Our system also integrates avatars embodied by remote users. We conducted a dual-user study comparing colocated and remote solving of a puzzle virtual replica with its real solving. We evaluated task performance, collaboration, mutual awareness, spatial presence, and copresence, usability, and preference. We found that, if real is preferred and more efficient than our CAR system, CAR is reaching favorable usability levels. We also found that remote augmented reality including full-body avatars offers similar results to colocated augmented reality. This preliminary work paves the way for future research aiming to support and enhance the design and making of Collaborative Augmented Reality systems dedicated to augmented operations.

We present ConnectBoard, a new system for remote collaboration where users experience natural interaction with one another, seemingly separated only by a vertical, transparent sheet of glass. It overcomes two key shortcomings of conventional video communication systems: the inability to seamlessly capture natural user interactions, like using hands to point and gesture at parts of shared documents, and the inability of users to look into the camera lens without taking their eyes off the display. We solve these problems by placing the camera behind the screen, where the remote user is virtually located. The camera sees through the display to capture images of the user. As a result, our setup captures natural, frontal views of users as they point and gesture at shared media displayed on the screen between them. Users also never have to take their eyes off their screens to look into the camera lens. Our novel optical solution based on wavelength multiplexing can be easily built with off-the-shelf components and does not require custom electronics for projector-camera synchronization.

Supporting natural communication cues is critical for people to work together remotely and face-to-face. In this paper we present a Mixed Reality (MR) remote collaboration system that enables a local worker to share a live 3D panorama of his/her surroundings with a remote expert. The remote expert can also share task instructions back to the local worker using visual cues in addition to verbal communication. We conducted a user study to investigate how sharing augmented gaze and gesture cues from the remote expert to the local worker could affect the overall collaboration performance and user experience. We found that by combing gaze and gesture cues, our remote collaboration system could provide a significantly stronger sense of co-presence for both the local and remote users than using the gaze cue alone. The combined cues were also rated significantly higher than the gaze in terms of ease of conveying spatial actions.

Since the introduction of augmented reality (AR) technology, in-situ instructions for manual tasks have been a central use case for a large body of previous work. However, most implementations provide identical sets of instructions to each user disregarding the user's current mental load. This is a major issue since previous work has shown the importance and potential of an adapted instruction fidelity for manual tasks such as playing an instrument. To implement a low-cost mental load adaptation for AR instructions, we evaluated a mobile off-the-shelf electroencephalographic (EEG) device for its suitability and feasibility to measure mental load while wearing a video see-through AR head-mounted display (HMD). In a first user experiment (n=12), data of EEG power band values and proprietary performance metrics of the manufacturer were collected and analysed regarding their validity to estimate the user's mental load. Our results indicate that our setup successfully induced different levels of mental effort. The proprietary performance metrics, however, only partially reflected the participants' current mental effort and require further analysis.

The objective of this study is to test the correlation between a Physical Task and a Digital Task through integrated sensory feedback mechanism in Virtual Build Methodology. The research question posed regards whether the pressure feedback mechanism in Virtual Build Methodology proposes high fidelity for push-pull tasks. There are many research studies that have been done on DHM, MOCAP, VR and Haptic interfaces individually, but integrating those with a tactile feedback mechanism is still challenging. While being increasingly used, the Virtual Build Methodology has not been studied regarding its human integration through a multi-sensory feedback system. It may seem intuitive, but disregarded many times, that the tactile feedback mechanism is essential for product design and development practices. This study aims to fill this gap by introducing a pressure based sensory feedback system to provide a higher fidelity in virtual product design practices.

Effects of Room-sized Sharing on Remote Collaboration on Physical Tasks

While shared visual space plays a very important role in remote collaboration on physical tasks, it is challenging and expensive to track users' focus of attention (FOA) during these tasks. In this paper, we propose to identify a user's FOA from his/her intention based on task properties, people's actions in the workspace, and conversational content. We employ a conditional Markov model to characterize a subject's FOA. We demonstrate the feasibility of the proposed method using a collaborative laboratory task in which one partner (the helper) instructs another (the worker) on how to assemble online puzzles. We model a helper's FOA using task properties, workers' actions, and conversational content. The accuracy of the model ranged from 65.40% for puzzles with easy-to-name pieces to 74.25% for puzzles with more difficult-to-name pieces. The proposed model can be used to predicate a user's FOA in a remote collaborative task without tracking the user's eye gaze.