Estimating Cognitive Processes Related to Haptic Interaction within Virtual Environments

The purpose of this case study was to determine if learning occurred within a 3D virtual learning environment by determining if elements of collaboration and Nonaka and Takeuchi's (1995) knowledge spiral were present. A key portion of this research was the creation of a Virtual Learning Environment. This 3D VLE utilized the Torque Game Engine provided by Garage Games and a content pack created by BraveTree Productions, LLC. By creating this 3D didactic constructivist virtual environment, conversations were observed for collaborative and knowledge creation elements. Data for this mixed-design case study were gathered through three sources: the 3D virtual environment; a survey created by the researchers; and follow-up interviews. Findings revealed that the five forms of collaboration (Elementary Clarification, In-Depth Clarification, Inference, Judgment, and Application) amplified the knowledge creation process and indeed occurred within the virtual learning environment. It was also determined that all four modes for knowledge creation (socialization, externalization, combination, and internalization) were observed as well.

This chapter aimed to reveal how the analysis, design, and development processes should be carried out while history teaching environments are being developed in 3D virtual environments and to set a concrete definition of teaching process in 3D virtual environments. In the history teaching session, the participants were represented in the 3D virtual environment developed in OpenSim simultaneously with their own avatars. They role-played the historical characters and actively participated in the events of that period by using historical reenactment technique in historical spaces. Additionally, this chapter describes the development process of a 3D virtual teaching environment from the design of the teaching process to the implementation. The procedures performed during the preparation of the 3D virtual teaching environment are presented with a flowchart. It is expected that this flowchart, suggested as a 3D virtual instructional design model, will guide researchers through the process of developing teaching implementations in 3D virtual environments in future studies.

In this paper, an immersive virtual roller coaster is introduced. A three Degree-Of-Freedom (3DOF) computer controlled simulator with special effects is designed, implemented, and tested. The simulator platform moves in response to control signals generated by a three-dimensional (3D) virtual gaming environment. In order to add more reality to the gaming environment, external effects, that represent a fourth dimension to virtual environment, are added to the simulator environment such as smell, sound, light, and sense of temperature (heat or cold) that are synchronized with the events happening in the 3D virtual gaming environment. The controlled simulator is developed for entertainment purposes and to suit all ages. The proposed system was tested and verified in the sense that the platform interacts with all events occur in the 3D virtual environment and external effects. The proposed 4D entertainment machine can be used in amusement parks and family entertainment centers and malls.

The current study concerns the identification of possible differences in perception between the virtual and the real world in terms of the effect on brain activity. For this reason, an EEG device was used to capture participants' brain activity in different brain areas during their exposure to different virtual and real environments. The environments considered in this study portray a classroom environment with a scenario suitable for teacher training and professional development. The first aim of the experiment is to investigate if exposure to a virtual environment can affect motor, cognitive or other function of the users, and the second aim is to test if the graphics content and nature of such an environment can influence the user experience. During the study, the optimum duration of exposure in a virtual environment was also assessed by measuring the time that the brain needs to perceive and adapt to the new state. Our results, consisting of EEG data analyzed in 10 Regions of Interest (ROIs) and responses from an Igroup Presence questionnaire, indicated a significant difference in each brain area, especially in the frontal and occipital region, when a participant was exposed to a non-realistic virtual environment, compared to a realistic one, highlighting the impact of the selected virtual environment design. The results of the experiment can play an important role in defining the characteristics of optimal virtual environments for virtual reality-based training applications.

In recent years, there has been a growing interest in developing force feedback interfaces that allow people to access information presented in 3D virtual reality environments (VEs). One of the major application fields is medicine. Several cases exist where standard medical treatment cannot be applied to patients for several reasons, such as distance from hospital, lack of specialized doctor, and lack of medical equipment. The use of haptic‐enabled virtual environments provides an alternative solution in a variety of such cases, thus enabling assessment and treatment of patients. Furthermore, haptic‐enabled virtual environments provide a nonhazardous means of treatment for several disorders like phobia treatment, where the patient has to be exposed to the phobia object. Training of disabled people, like the visually impaired, is another important application of haptic‐enabled VEs. This chapter presents an extended description of haptic virtual environments and the major methodologies and techniques used in medical applications. Further, it demonstrates indicatively a number of applications in order to illustrate the applicability of VEs in several areas of medical treatment and training.

Amyotrophic Lateral Sclerosis (ALS) is a neurodegenerative disease that causes progressive loss of voluntary movement, including the ability to speak. As the disease rapidly progresses, there is a need for augmented and alternative communication that removes physical barriers required for engagement. Brain-Computer Interfaces (BCI) are a technology that allow for communication using only brain signals in response to visual stimuli. This project aimed to design an accessible and practical BCI for continuous, at-home use, and to increase BCI usability by using familiar facial stimuli to elicit a stronger brain response. The overall goal for enhancement of BCI usability is to maximize ease of use for individuals with ALS and their caregivers so that BCI technologies can make a useful impact and improve the everyday lives of people with ALS. In this project, a new miniaturized and low-cost P300 BCI system was designed using wireless and battery-operated EEG neuroimaging, modular software, and the use of an individualized BCI presentation stimuli. A 3-dimensional virtual environment platform was integrated as a navigational control output mechanism that executes commands received from the BCI to move a user's avatar in a virtual maze. Together, these form the virtual environment BCI (veBCI). The BCI stimuli presentation consisted of a custom approach of integrating different human face images with the aim of increasing user engagement and elicited brain activity, and therefore improving BCI online accuracy. The impact and effectiveness of the approach was tested via a verification BCI study in which 6 different stimuli conditions were utilized: male celebrity face, female celebrity face, male stranger face, female stranger face, a family member/close friend of the participant, and blank/no face as a control. A total of 16 volunteers (average age 56.5 and eight diagnosed with ALS) used the BCI system with all 6 face conditions in ecologically valid settings. Results indicate that this new miniaturized BCI is usable for both people with ALS and healthy controls. Significant differences existed in the amplitude of the P300 response in ALS and healthy controls. Additionally, differences between male and female participants demonstrated the potential role of personalization to optimize the visually evoked potential response and further increase BCI online accuracy. In conclusion, a home-use P300 BCI system was developed with a new generation of miniaturized, low-cost, and mobile neuroimaging running on an embedded computational platform. BCI stimuli with human face inclusion could help personalization of the BCI. Furthermore, virtual environments provide unique new ways to implement BCI output beyond typical spelling tasks and communication. Future work could focus on developing dedicated online signal processing to eliminate noise and improve online accuracy during home use.

Online systems, especially 3D virtual environments, hold great potential to enrich and expand learning opportunities for those who are challenged by traditional modes of instruction and interaction. In the process of developing a 3D Virtual Learning Environment to support the development and practice of social competence for individuals with Autism Spectrum Disorders, the iSocial project explored and advanced ideas for social orthotics in virtual environments. By social orthotics the authors mean structures in the environment that overcome barriers to facilitate social interaction and social learning. The vision of social orthotics in a 3D world is to be both assistive and adaptive for appropriate social behavior when the student, peers and guide are represented by avatars in a 3D virtual world designed to support learning and development. This chapter describes the formulation of social orthotics for avatar orientation and conversational turn-taking and describes experiences and lessons from early tests of prototype orthotics.

As educational systems emphasize and experiment with forms of online and remote learning, it is increasingly important to investigate the cultural competence of instructional designers. This chapter addresses the experiences of instructional designers in a 3D virtual learning environment designed for development of cultural competence. Design-based research (DBR) and user experience (UX) methodologies were employed to explore experience of six instructional designers in 3D virtual environment. A taxonomy of experience (ToE) established by Coxon guided qualitative data collection and analysis. Through examples and data, the chapter emphasizes the necessity for instructional designers to keep in mind the challenge of cultural diversity in the backgrounds of students and their own, and bring guidelines and principles into culturally sensitive and responsive instructional design processes. The authors recommend four future research directions, including cross-cultural instructional designer competencies along with research into cultural personas, avatars, and guest-host relations.

Haptic interaction in slow virtual environments (VEs) can become unstable due to the phase lag introduced in the control loop by the slow update rate of the VE. Increasing the physical damping and/or limiting the contact stiffness rendered to users can mitigate the destabilizing effect of the low VE update rate. However, large physical damping and compliant virtual contacts decrease the sense of presence in VEs, especially during interaction with rigid virtual objects. To increase the maximum virtual contact stiffness that can be rendered to users without increasing the interface damping, this paper proposes a control strategy based on multirate wave communications between a haptic interface and a VE updated at a slow and fixed rate. The multirate wave communications are shown to be guaranteed passive only if the decrease of the wave sampling rate at the connection between the haptic interface and the VE does not cause aliasing. Therefore, an antialiasing low-pass filter is placed before the wave rate drop in the communications. The passivity condition is verified analytically and numerically for multirate haptic interaction in VEs with various contact stiffnesses and update rates. The transparency of haptic interaction in slow VEs to which users connect via passive multirate wave communications is investigated analytically in the frequency domain. Experiments validate that passive multirate wave communications can render stiffer contact in slow VEs than conventional direct coupling, and illustrate the destabilizing effect of the aliasing caused by the sampling rate drop in the communications.

Trails, rails, and over-reliance: How navigation assistance affects route-finding and spatial learning in virtual environments

Recent developments in Brain Computer Interfacing (BCI) and neuroprosthetics have played a vital role for disable people to expect better life quality. In this contribution Electroencephalographic (EEG) signals acquired from six healthy test subjects, are used for the offline analysis of BCI through classification of four lower limb movements including talocrural (ankle) joint dorsi-planter flexion and knee joint extension-flexion. Fourteen channel Emotive EPOC head set is used to acquire EEG signals from sensorimotor cortex area of brain, using a particular data acquisition timeline protocol. Features are extracted in time domain from raw EEG data. Power spectral density, variance, mean value and kurtosis features are applied on raw EEG signals. Multiple classification algorithms are implemented for discrimination of four lower limb movements within data set. The paper uses Quadratic discriminant analysis, Naïve Bayes and Support vector machine classifiers to stratify the movement intent of lower limb. Maximum classification accuracies achieved through various classifiers are; 86.35% with average band power & QDA, 84.38% with mean value & QDA and 78.13% with power spectral density & Quadratic-SVM. The presented findings are optimistic in making the path easier towards the development of BCIs with rich EEG based control signals using noninvasive technology.

Three-dimensional (3D) virtual environments bring people together in real time irrespective of their geographical location to facilitate collaborative learning and working together in an engaging and fulfilling way. However, it can be difficult to amass suitable data to gauge how well students perform in these environments. With this in mind, the current study proposes a methodology for monitoring students’ learning experiences in 3D virtual worlds (VWs). It integrates a computer-based mechanism that mixes software agents with natural agents (users) in conjunction with a fuzzy logic model to reveal evidence of learning in collaborative pursuits to replicate the sort of observation that would normally be made in a conventional classroom setting. Software agents are used to infer the extent of interaction based on the number of clicks, the actions of users, and other events. Meanwhile, natural agents are employed in order to evaluate the students and the way in which they perform. This is beneficial because such an approach offers an effective method for assessing learning activities in 3D virtual environments.

In the rapidly emerging era of untact (“contact-free”) technologies, the requirement for three-dimensional (3D) virtual environments utilized in virtual reality (VR)/augmented reality (AR) and the metaverse has seen significant growth, owing to their extensive application across various domains. Current research focuses on the automatic transfer of the style of rendering images within a 3D virtual environment using artificial intelligence, which aims to minimize human intervention. However, the prevalent studies on rendering-based 3D environment-style transfers have certain inherent limitations. First, the training of a style transfer network dedicated to 3D virtual environments demands considerable style image data. These data must align with viewpoints that closely resemble those of the virtual environment. Second, there was noticeable inconsistency within the 3D structures. Predominant studies often neglect 3D scene geometry information instead of relying solely on 2D input image features. Finally, style adaptation fails to accommodate the unique characteristics inherent in each object. To address these issues, we propose a novel approach: a neural rendering-based 3D scene-style conversion technique. This methodology employs semantic nearest-neighbor feature matching, thereby facilitating the transfer of style within a 3D scene while considering the distinctive characteristics of each object, even when employing a single style image. The neural radiance field enables the network to comprehend the geometric information of a 3D scene in relation to its viewpoint. Subsequently, it transfers style features by employing the unique features of a single style image via semantic nearest-neighbor feature matching. In an empirical context, our proposed semantic 3D scene style transfer method was applied to 3D scene style transfers for both interior and exterior environments. This application utilizes the replica, 3DFront, and Tanks and Temples datasets for testing. The results illustrate that the proposed methodology surpasses existing style transfer techniques in terms of maintaining 3D viewpoint consistency, style uniformity, and semantic coherence.

The high potential for creating brain-computer interfaces (BCIs) and video games for upper limb rehabilitation has been demonstrated in recent years. In this work, we describe the implementation of a prototype BCI with feedback based on a virtual environment to control the lateral movement of a character by predicting the subject’s motor intention. The electroencephalographic signals were processed employing a Finite Impulse Response (FIR) filter, Common Spatial Patterns (CSP), and Linear Discriminant Analysis (LDA). Also, a video game was used as a virtual environment, which was written in C# on the Unity3D platform. The test results showed that the prototype implemented based on electroencephalographic signal acquisition has the potential to take on real-time applications such as avatar control or assistive devices, obtaining a maximum control time of 65 s. In addition, it was noticed that the feedback in an interface plays a crucial role, since it helps the person not only to feel motivated, but also to learn how to have a more consistent motor intention and when little calibration data is recorded, the probability that the system makes erroneous predictions increases. These results demonstrate the usefulness of the development as support for people who require some treatment in the form of upper limb motor rehabilitation, and that the use of virtual environments, such as video games, can motivate such people during the rehabilitation processes.

We have implemented a computer interface that renders synchronized auditory and haptic stimuli with very low (0.5ms) latency. The audio and haptic interface (AHI) includes a Pantograph haptic device that reads position input from a user and renders force output based on this input. We synthesize audio by convolving the force profile generated by user interaction with the impulse response of the virtual surface. Auditory and haptic modes are tightly coupled because we produce both stimuli from the same force profile. We have conducted a user study with the AHI to verify that the 0.5ms system latency lies below the perceptual threshold for detecting separation between auditory and haptic contact events. We discuss future applications of the AHI for further perceptual studies and for synthesizing continuous contact interactions in virtual environments.