Improving Information Sharing and Collaborative Analysis for Remote GeoSpatial Visualization Using Mixed Reality

We present a novel Mixed Reality (MR) remote collaboration system that integrates a telepresence drone to overcome viewpoint limitations in previous approaches. Current remote collaboration systems often restrict remote users to the local user's perspective or a fixed viewpoint, which may reduce spatial awareness and collaboration effectiveness. Our proposed system enables a remote user wearing a VR headset to independently navigate the local environment through a drone while guiding a local AR headset user. The drone represents the remote user physically and virtually with an AR avatar. The system provides the remote user with a 3D reconstruction of the local environment, the local user's avatar, and real-time drone camera video to facilitate drone operation and collaboration. A user study comparing our drone-enhanced MR system with 2D video and 360-degree video MR systems during inspection tasks revealed that the telepresence drone significantly improved collaboration efficiency and enhanced users' social presence and spatial presence. However, the remote users perceived a higher workload. Our findings demonstrate the benefits of independent spatial navigation through drones in remote collaboration and offer insights for future drone-enhanced MR remote collaboration systems.

Supporting natural communication cues is critical for people to work together remotely and face-to-face. In this paper we present a Mixed Reality (MR) remote collaboration system that enables a local worker to share a live 3D panorama of his/her surroundings with a remote expert. The remote expert can also share task instructions back to the local worker using visual cues in addition to verbal communication. We conducted a user study to investigate how sharing augmented gaze and gesture cues from the remote expert to the local worker could affect the overall collaboration performance and user experience. We found that by combing gaze and gesture cues, our remote collaboration system could provide a significantly stronger sense of co-presence for both the local and remote users than using the gaze cue alone. The combined cues were also rated significantly higher than the gaze in terms of ease of conveying spatial actions.

A tension exists between designers' comfort with physical artifacts and the need for effective remote collaboration: physical objects live in one place. Previous research and technologies to support remote collaboration have focused on shared electronic media. Current technologies force distributed teams to choose between the physical tools they prefer and the electronic communication mechanisms available. We present Distributed Designers' Outpost, a remote collaboration system based on The Designers' Outpost, a collaborative web site design tool that employs physical Post-it notes as interaction primitives. We extended the system for synchronous remote collaboration and introduced two awareness mechanisms: transient ink input for gestures and a blue shadow of the remote collaborator for presence. We informally evaluated this system with six professional designers. Designers were excited by the prospect of physical remote collaboration but found some coordination challenges in the interaction with shared artifacts.

We present ConnectBoard, a new system for remote collaboration where users experience natural interaction with one another, seemingly separated only by a vertical, transparent sheet of glass. It overcomes two key shortcomings of conventional video communication systems: the inability to seamlessly capture natural user interactions, like using hands to point and gesture at parts of shared documents, and the inability of users to look into the camera lens without taking their eyes off the display. We solve these problems by placing the camera behind the screen, where the remote user is virtually located. The camera sees through the display to capture images of the user. As a result, our setup captures natural, frontal views of users as they point and gesture at shared media displayed on the screen between them. Users also never have to take their eyes off their screens to look into the camera lens. Our novel optical solution based on wavelength multiplexing can be easily built with off-the-shelf components and does not require custom electronics for projector-camera synchronization.

Automatic Speech Recognition (ASR) technologies can be used to address people with auditory disabilities by integrating them in an interpersonal communication via textual visualization of speech. Especially in avatar-based Mixed Reality (MR) remote collaboration systems, speech is an important additional modality and allows natural human interaction. Therefore, we propose an easy to integrate ASR and textual visualization extension for an avatar-based MR remote collaboration system that visualizes speech via spatial floating speech bubbles. In a small pilot study, we achieved word accuracy of our extension of 97% by measuring the widely used word error rate.

Recently MR (Mixed Reality) techniques are used in many fields, and one of them is work support. Work support using MR techniques can display work instructions directly in the work space and help user to work effectively especially in assembling tasks. MR work support for assembling tasks often uses HMD (Head Mounted Display) to construct MR environment. However, there are some problems for the use of HMD, such as a burden to the head, a narrow view and a motion picture sickness. One of the techniques to solve such problems is pepper's ghost which is an optical illusion using a glass. In this paper, we propose the naked eye MR work supporting system for assembling tasks using pepper's ghost. This system enables a beginner to assemble some blocks into one object by the naked eye with a few burdens.

In this research, we explore how different types of augmented gesture communication cues can be used under different view sharing techniques in a remote collaboration system. In a pilot study, we compared four conditions: (1) Pointers on Still Image, (2) Pointers on Live Video, (3) Annotation on Still Image, and (4) Annotation on Live Video. Through this study, we found three results. First, users collaborate more efficiently using annotation cues than pointer cues for communicating object position and orientation information. Second, live video becomes more important when quick feedback is needed. Third, the type of gesture cue has more influence on performance and user preference than the type of view sharing method.

Typical view sharing system has same camera alignment that camera take images from back of remote instructor. We change this alignment to camera take images from front of remote instructor for preventing occlusions caused by a body of remote instructor self. Also as visual feedbacks, a mirror image of remote instructor is indicated in display of remote instructor side. Eventually remote instructor can confirm own instruction in the display. Therefore due to displaying the mirror image of remote instructor and changing camera alignment, we proposed and implement a novel remote collaboration system which prevents occlusion problems caused by instructor body self when he/she sends clear instructions by whole body gesture and allows instructor to use direct manipulation.

We present a Mixed Reality system for remote collaboration using Virtual Reality (VR) headsets with external depth cameras attached. By wirelessly sharing a 3D point-cloud data of a local workers' workspace with a remote helper, and sharing the remote helper's hand gestures back to the local worker, the remote helper is able to assist the worker to perform manual tasks.Displaying the point-cloud video in a conventional way, such as a static front view in VR headsets, does not provide helpers with sufficient understanding of the spatial relationships between their hands and the remote surroundings. In contrast, we propose a Mixed Reality (MR) system that shares with the remote helper, not only 3D captured environment data but also real-time orientation info of the worker's viewpoint. We conducted a pilot study to evaluate the usability of the system, and we found that extra synchronized orientation data can make collaborators feel more connected spatially and mentally.

Global events, including pandemics and geopolitical conflicts, have catalyzed a fundamental reevaluation of remote collaboration paradigms across training, technical assistance, maintenance, and educational domains. While conventional video conferencing technologies have achieved widespread adoption, significant potential remains for advancing augmented reality (AR)-based remote collaboration methodologies. Though hand-drawn AR annotations (AAS) remain prevalent for visual instruction delivery, their inherent informality often proves insufficient for conveying precise operational directives in industrial contexts. Consequently, multimodal verbal communication emerges as a critical compensatory mechanism to resolve ambiguities and mitigate linguistic barriers. This research introduces a novel Spatial AR (SAR) remote collaboration platform designed to bridge this communication gap through an Adaptive Transformation of Instructions (ATI) methodology. The proposed system enables dynamic conversion of freehand annotations into standardized symbolic representations, thereby enhancing instructional clarity and fostering universal comprehension. A rigorous formative user study was conducted to evaluate the platform’s efficacy across multiple dimensions: task completion time, annotation complexity, subjective collaborative experience, system usability (via standardized rankings), and user preference metrics. Results demonstrate statistically significant improvements in user experience when ATI-based cues are integrated into remote collaborative training scenarios. Beyond practical implications, this work provides actionable insights for future investigations into AR-mediated remote collaboration within technical training environments.

Remote collaboration has been studied for more than two decades and now there is the possibilities for new types of collaboration with the recent advances in immersive technologies such as Virtual, Augmented, Mixed Reality (VR/AR/MR). However, despite the increasing research interest in remote collaboration study with VR/AR/MR technologies, there is still a lack of academic venues specifically focusing on VR/AR/MR remote collaboration research. This special issue provides high-quality papers on the topic of remote collaboration research and increases visibility of this timely interesting and important research area. We particularly focus on three research aspects in remote collaboration: (1) use of multimodal communication cues, (2) awareness of the task space, and (3) human factors understanding. In this editorial, we first describe five essential factors for remote collaboration: task, local user, remote user, communication, and tool/interface, and then summarize a brief history of the research areas. We also cover the feature papers accepted in this issue, which introduce novel multimodal interfaces for remote collaboration and the effects on task performance and perceptual factors. Finally, we discuss some potential future research directions while concluding the editorial.

A Remote Collaboration System with Force Feedback

Collaboration in scientific visualization

In the mysterious realm full of emerging technologies, there is a fantastic role played by Mixed Reality techniques. The blending of real and virtual worlds that allows individualized user control opportunities of the environment around user remarks the unique feature of Mixed Reality. The current work shows the effort of many researchers who have tried to explore the field of Mixed Reality in their own planned ways for the creation of Mixed Reality smart classroom. The objective is to introduce, how Mixed Reality device can act as real-time analytical tool for the teacher in a classroom to enhance the learning process of the students. The first part introduces the advent of Mixed Reality with some defined theories that are widely accepted and a glimpse of approaches done by many researchers. Later, an overview of an attempt to create a Mixed Reality classroom, two different hypotheses incorporating the same Mixed Reality tool called Lumilo were experimented with the assistance of AIED and ITS systems. Followed by the results of the hypotheses, aiming one of them to be employed in the classroom environment where both teachers and students were most benefitted. This type of environment resulted in better modes of education when compared to the existing approaches of Mixed Reality classroom.

This paper investigates the effect of avatar appearance on Social Presence and users' perception in an Augmented Reality (AR) telep-resence system. Despite the development of various commercial 3D telepresence systems, there has been little evaluation and discussions about the appearance of the collaborator's avatars. We conducted two user studies comparing the effect of avatar appearances with three levels of body part visibility (head & hands, upper body, and whole body) and two different character styles (realistic and cartoon-like) on Social Presence while performing two different remote collaboration tasks. We found that a realistic whole body avatar was perceived as being the best for remote collaboration, but an upper body or cartoon style could be considered as a substitute depending on the collaboration context. We discuss these results and suggest guidelines for designing future avatar-mediated AR remote collaboration systems.