Providing Dual Awareness using Multimodal Cues for Collaborative Manipulation in Virtual Environments

Though virtual reality (VR) has been advanced to certain levels of maturity in recent years, the general public, especially the population of the blind and visually impaired (BVI), still cannot enjoy the benefit provided by VR. Current VR accessibility applications have been developed either on expensive head-mounted displays or with extra accessories and mechanisms, which are either not accessible or inconvenient for BVI individuals. In this paper, we present a mobile VR app that enables BVI users to access a virtual environment on an iPhone in order to build their skills of perception and recognition of the virtual environment and the virtual objects in the environment. The app uses the iPhone on a selfie stick to simulate a long cane in VR, and applies Augmented Reality (AR) techniques to track the iPhone’s real-time poses in an empty space of the real world, which is then synchronized to the long cane in the VR environment. Due to the use of mixed reality (the integration of VR & AR), we call it the Mixed Reality cane (MR Cane), which provides BVI users auditory and vibrotactile feedback whenever the virtual cane comes in contact with objects in VR. Thus, the MR Cane allows BVI individuals to interact with the virtual objects and identify approximate sizes and locations of the objects in the virtual environment. We performed preliminary user studies with blind-folded participants to investigate the effectiveness of the proposed mobile approach and the results indicate that the proposed MR Cane could be effective to help BVI individuals in understanding the interaction with virtual objects and exploring 3D virtual environments. The MR Cane concept can be extended to new applications of navigation, training and entertainment for BVI individuals without more significant efforts.

Resumo: O presente artigo descreve uma investigação que avaliou a utilização de ambientes virtuais de aprendizagem colaborativa no processo de construção de conhecimentos sobre ciências da natureza em um grupo de 16 estudantes do terceiro ano do ensino médio de uma escola pública localizada na região metropolitana de Porto Alegre. A pesquisa teve abordagem qualitativa do tipo estudo de caso e os relatos coletados foram analisados por Análise Textual Discursiva (ATD), com categorias emergentes. O ambiente virtual utilizado foi disponibilizado pela empresa Microsoft® na plataforma OneDrive®. Os participantes identificaram contribuições proporcionadas pelo uso do ambiente virtual e da tecnologia em sala de aula, assinalando seus benefícios para o processo educacional. Entre os argumentos encontrados nas respostas dos participantes, estão os seguintes: o aumento da mobilidade proporcionada pelos recursos tecnológicos; a capacidade de a virtualidade possibilitar o surgimento de múltiplos processos de experimentação; a redução da sensação de distanciamento dos educandos entre si, e destes em relação ao professor; a superação dos limites impostos pelo tempo e espaço da sala de aula transmissiva e; a potencialização do questionamento reconstrutivo, à medida que os recursos para a busca de informações ficam à disposição e podem ser prontamente acessados e questionados pelo grupo de colaboração. Como resultado, a investigação permitiu concluir que os participantes reconhecem que a aprendizagem colaborativa em ambientes virtuais possibilita a implementação de uma forma prática de educação pela pesquisa, pois se desenvolve ao longo de um processo que implica interação com fontes de consulta e com grupos de trabalho, abrindo a possibilidade do surgimento de novas compreensões e relações. Palavras-chave: Aprendizagem colaborativa. Ambientes virtuais.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Advances in Virtual Reality (VR) systems, Intelligent Tutoring Systems and Agent Technology make it possible to design and develop Virtual Training Environments (VTEs), where the trainees can immerse themselves and interact directly with the learning domain. This paper presents the Interaction Specification Workspace (ISW) architecture for the specification and design of virtual environments for training purposes. ISW architecture provides the interaction designer with the capability to specify the training interaction with the virtual environment using the Virtual Reality Multi Flow Graph (VR-MFG) as the underlying interaction specification model. ISW implements a design space, where the processes of interaction specification and design of VTEs take place inside a three-dimensional virtual environment, the objects of which are tools by themselves. Inside this workspace, the designer can associate the abstract objects (the components of the VR-MFG) with “actual” objects of the target virtual environment (kept in a “world” database) and apply a number of agent templates with training capabilities.

Micro assembly involves the assembly of micron-sized devices. Given the complexity of this domain, the role of virtual environments becomes important as they provide a basis to propose and compare assembly alternatives virtually prior to physical assembly. This paper proposes an integrated approach which includes the use of virtual reality-based assembly environments that interface with physical micro assembly environments. Such an approach can be an intrinsic part of a collaborative manufacturing framework that seeks to support the rapid assembly of micro devices. In this paper, the design of VREM (Virtual Reality based Environment for Micro Assembly) is discussed which is based on this integrated approach involving use of virtual and physical resources.

Human spatial navigation: cognitive maps, sexual dimorphism, and neural substrates

How can we increase awareness and understanding of other cultures using interactive online digital visualizations of past civilizations? In order to answer the above question, this chapter first questions the success of current virtual environments and asks whether they are capable of producing a platform that supports the experience and understanding of place-inscribed culture. To answer this question, a typology of successful virtual environment applications is suggested. These include online communities, virtual realityVirtual environmentVirtual Reality (VR) travel websites, game engines, virtual reality used for training, and virtual reality used as part of therapeutic means to either relax or divert patients, or to cure phobiasPhobias. Impeding the success of virtual environments in terms of technological constraints, lack of evaluation techniques and results, and a lack of content-specific applications that best utilize desktop computing capabilities and respond to user needs. It suggests that much more work needs to be done on not just usable but also useful content. Specifically, there is a large gap in knowledge on what might constitute useful and appropriate virtual travel environments, and how contextual interaction may affect our cultural understanding in these environments.

The use of journal clubs and, more recently, case-based discussions in order to stimulate debate among orthopaedic surgeons lies at the heart of orthopaedic training and education. A virtual learning environment can be used as a platform to host virtual journal clubs and case-based discussions. This has many advantages in the current climate of constrained time and diminishing trainee and consultant participation in such activities. The virtual environment model opens up participation and improves access to journal clubs and case-based discussions, provides reusable educational content, establishes an electronic record of participation for individuals, makes use of multimedia material (including clinical imaging and photographs) for discussion, and finally, allows participants to link case-based discussions with relevant papers in the journal club. The Leicester experience highlights the many advantages and some of the potential difficulties in setting up such a virtual system and provides useful guidance for those considering such a system in their own training programme. As a result of the virtual learning environment, trainee participation has increased and there is a trend for increased consultant input in the virtual journal club and case-based discussions. It is likely that the use of virtual environments will expand to encompass newer technological approaches to personal learning and professional development.

The conceptual design review process is a critical cost determining step for complex products such as ground vehicles. The use of virtual environments (VEs) in this process has become increasingly popular with the advancement of 3D visualization technologies. Important to feelings experienced by participants in a VE are two parameters: presence and immersion. Presence is defined as the subjective experience of being in one place while physically being situated in another. Immersion is a state characterized by perceiving oneself to be enveloped by, included in, and interacting in an environment that provides a continuous stream of stimuli. While different virtual reality (VR) devices provide different degrees of presence and immersion, the amount of each is also dependent on the individual. In this paper, a relationship among presence, immersive tendencies of individuals, and design comprehension are explored. Further, a methodology to find the best design review process subject to certain criteria is presented. The results of two experiments involving the CAVE Automatic Virtual Environment CAVE™2 are discussed and evaluated. In one case, the U.S. Army investigates the use of the CAVE as a conceptual design review environment for advanced military vehicles in a comparison test with its present method of concept presentation and review. In a second experiment, CAVE users responded to a survey to determine the potential of VEs to improve design comprehension and immersive tendencies. In both cases, positive presence and immersion results support the idea that VEs offer advantages for conceptual design reviews over more traditional methods.

The effects of the presence of others during a rowing exercise in a virtual reality environment

In this paper, we introduce approaches to navigating and manipulating objects in a Collaborative Virtual Environment (CVE) that engage tangible objects and an interactive table interface. We also identify three design concerns that are common to physical-virtual connectivity for interaction with CVE systems. At last, we propose solutions to these issues within the context of CVEs.

Presents a combined system for hand and finger kinesthetic feedback for the exploration and manipulation of different kinds of rigid objects in virtual environments. A three-degrees-of-freedom haptic interface, called DeKiFeD3 (Desktop Kinesthetic Feedback Device 3), with the capability of generating kinesthetic hand feedback, is combined with a commercial haptic glove to generate additional kinesthetic feedback to the fingers of a human operator. This system results in a new quality of kinesthetic perception and needs appropriate rendering algorithms, as described in this paper. The paper also presents an approach to improve the operator's immersion into a virtual scenario by adding auditory and visual feedback.

We present in this paper researches focused on specification and production of virtual training environments. The use of Virtual Environments (VE) for training is strongly stimulated by important needs of training on sensitive equipment. But the development of such applications is generally done in new projects, where the reusability of existing developments is a major issue. We present here a platform of development, based on internal models that have been designed to achieve this reusability and standardization for the efficient development of new virtual environments. We defined a new generic model named STORM, used to describe reusable behaviors for 3D objects and reusable interactions between those objects or with humans. We also defined a scenario language named LORA, which allow non-computer scientists author various and complex sequences of tasks in a virtual scene. Based on those models, as an industrial validation with Nexter-Group, we have created 20 operational scenarios of maintenance virtual training on military equipments. We are also begining to create collaborative environments, where virtual and real humans have to collaborate to achieve their procedures.

This chapter examines virtual environments which usually represent computer simulated environments, otherwise called virtual worlds or virtual reality and its application in physical therapy. Virtual environments present computer-generated three-dimensional (3D) representation of a physical presence of places in the real world as well as in the imaginary world. Users mainly interact with virtual environments using virtual reality headsets. Many of the current virtual reality environments are primarily visual experiences display either on a computer screens or through other stereoscopic displays. Applications of virtual environments are mainly focused on artificial reality that projects users into a 3D space generated by the computer with some sort of tracking devices which may be the virtual reality headsets worn or another 3D tracking device attached to the body of the user such as the electromagnetic tracker system. Research in virtual environments has made significant impact in the entertainment industry and in the training field such as the flight simulator (Stern, 1995).

The automatic generation of direction in natural language, for the location of objects, is an ongoing research area heavily supported by the use of virtual environments (VEs). Important components of spatial language such as the selected reference object, along with specific features related to the situation of the scenario and the user, have to be properly combined in order to create a helpful direction to locate an object within a VE. In this paper we present a scheme, constructed upon literature review and specific empirical data, to link those different elements related to the location of objects, aimed to establish the suitable algorithms for the automatic generation of spatial language in VEs.