Pseudo-Haptic Feedback in Teleoperation.

Dysphagia, which results from various disorders, may increase the risk of aspiration pneumonia, dehydration, and malnutrition. The aim of this study was to develop an innovative evaluation and treatment system for swallowing therapy using virtual reality (VR) feedback and electrical stimulation (ES), and to make an initial evaluation of its potential. In this system, the activation of the submental muscle and acceleration of laryngeal movement are used as the evaluation and feedback information. Twenty-one patients with chronic dysphagia for an average of 26.3 months were recruited for the VR feedback study. Each participant underwent 16 treatment sessions. After therapy, the Functional Oral Intake Scale results changed from 3.3 ± 1.5 to 5.0 ± 1.6 with statistical significance (p = 0.000). Thirteen healthy subjects were enrolled in the ES study. ES was applied for more than 2 s while the subjects were swallowing. With and without ES, swallowing triggering times were 456.17 ± 106.92 and 552.13 ± 105.97 ms, respectively. These differences were statistically significant (p = 0.04). Accelerations of laryngeal movement were 0.23 ± 0.1 g (g = 9.8 m/s2) and 0.20 ± 0.08 g, respectively, with a significant statistical difference (p = 0.033). The feasibility of a prototype that combines swallowing evaluation, VR feedback therapy, and synchronized ES is demonstrated for further clinical trials. Further comprehensive clinical studies are needed to verify the clinical efficacy of the device.

Visual representations are used in various digitization processes to display objects in immersive environments. In industrial environments, the self-representation can be very helpful when reviewing 3D models to conform realistic proportions, particularly in combi-nation with haptic feedback that goes beyond vibrating controllers. For instance, haptic feedback in combination with a virtual representation supports working on use cases where collision is an important part to maintain data quality. To understand the dependency of both, we conducted a pilot study with 15 users from the automotive sector to examine the influence of upper limb representations on haptic feedback in Virtual Reality. Each participant was assigned with one of three upper limb representations which was used in different scenarios using haptic feedback devices. Overall, we found that the realistic arm representation was rated highest in terms of perceived realism and achieved the best task performance.

Ultrasound mid-air haptic (UMH) devices are promising for tactile feedback in virtual reality (VR), as they do not require users to be tethered to, hold, or wear any device. This approach is less cumbersome, easy to set up, can simplify tracking, and leaves the hands free for concurrent interactions. This chapter explores work conducted at CNRS-IRISA dealing with the challenges arising from the integration of UMH interfaces in immersive VR through three main axes. These are discussed in the wider context of the state of the art on UMH for augmented and virtual reality, and illustrated through several VR use-cases. A first axis deals with device integration into the VR ecosystem. Interaction in immersive VR is based on the synergy between complex input devices allowing real-time tracking of the user and multimodal feedback devices delivering a coherent visual, auditory and haptic picture of a simulated virtual environment (VE). Using UMH in immersive VR therefore hinges on integrating UMH devices such that their operation does not interfere with other input and feedback devices. It is also critical to ensure that UMH feedback is adequately synchronized and co-located with respect to other stimuli, and delivered within a workspace that is compatible with that of VR interaction. Regarding this final point, we propose PUMAH, a robotic solution for increasing the usable workspace of UMH devices. The second and third axes, respectively, focus on stimulus perception and rendering of VE properties. Virtual object properties can be rendered in a variety of ways, through, e.g., amplitude modulation (AM) or spatiotemporal modulation (STM), with many parameters (modulation frequency, spatial sampling, etc.) coming into play, raising questions about the limits of the design space. To tackle this challenge, we begin by conducting psychophysical experimentation to understand the usable ranges for stimulus parameters and understand the perceptual implications of stimulus design choices. We propose an open-source software framework intended to facilitate UMH stimulus design and perceptual evaluation. These results in turn serve as the basis for the design and evaluation of rendering schemes for VR. Using amplitude variations along a focal point path in STM, we investigate the possibility of rendering geometric details and in a second step, sensations of stiffness in VR.

Once an accident occurs, it will cause serious damage to the surroundings and loss of life and social trust.Therefore, researchers studies for training system using virtual reality (VR) technology. The advantage of VR training is that it can reproduce accidents in operating plants that cannot occur in the real world, and trainees can train how to deal with them. VR training requires immersion to make it more effective. However, there is not currently convenient interface in VR training such as a valve used in industrial plants. Most common interface in industrial plants is valve so that if there is gap between training interface and actual interface, user cannot obtain immersion, for example, in case that mouse or keyboard of PC are interface for training system.To solve this problem, we aim to develop touch valve user interface (UI) which operability and feedback are close to actual valve by using touch panel and vibration presentation instead of reaction force as pseudo-haptic feedback. That operability and feedback near to actual valve will cause user perceptual illusion and can keep them immerse in VR environment.This paper explains our think why the operability and feedback of our valve UI can approach to that of actual valve, and the result of questionnaire which asked whether the operability and feedback of our valve is near that of an actual valve when vibration generates or not. The result of the Student’s t-test for paired data was df = 12, t = −4.382, P = 0.001 (<5%). Calculation of the t-test confirmed that the vibration that our valve presented is significant for user to percept feedback like that from an actual valve.

While feedback is frequently emphasized as a crucial principle of presentation courses in higher education, previous studies revealed that teachers outperform peers in terms of impact on students’ development of oral presentation competence. Further, presentation research showed that the lack of quality of peer feedback can be considered as an essential argumentation for the identified differences in effect. Follow-up field experiments demonstrated that Virtual Reality (VR) can be considered as a valuable alternative feedback source for developing public speaking skills, since this technology is able to simulate real-life presentation situations as well as to deliver feedback from the VR system to the individual learner. Recent technological developments allowed to convert quantitative information from VR systems into qualitative feedback messages that directly relate to the standards for high-quality feedback. If students are able to individually interpret the feedback messages without the intervention of a human feedback source, it could enrich the quality of feedback in peer and self-learning and further increase students’ oral presentation competence development. This chapter provides a synthesis of the literature in presentation research with the aim to construct a research agenda on computer-mediated feedback in VR for peer learning in this field. Further, two recent VR experiments in presentation research are discussed with the aim to effectively construct feedback messages in VR for improving peer learning.

Aim/Purpose: This paper investigates three key aspects of metaverse-based platforms in English language teaching: (1) their effectiveness in enhancing language learning outcomes, (2) their impact on digital literacy development in higher education, and (3) their accessibility and inclusivity implications for diverse student populations. Background: Despite growing interest in digital language learning, research gaps exist in understanding the long-term effectiveness of immersive technologies, particularly in resource-constrained settings. The metaverse – an integrated platform combining augmented reality (AR) for real-world digital overlays and virtual reality (VR) for immersive experiences – remains little studied in language education. While existing research examines AR or VR separately, this study investigates their combined impact through metaverse technology, addressing the unexplored intersection of language proficiency, digital literacy, and learning sustainability in higher education. Methodology: A mixed-methods study was conducted with 60 undergraduate students divided equally between experimental (metaverse-based) and control (traditional online) groups for an 8-week English course. The experimental group used an integrated metaverse platform combining AR and VR technologies for one hour daily. Quantitative assessment included pre-test, post-test, and six-month retention measurements of language proficiency (reading, writing, speaking; 10-point scale each) and digital literacy (36-item standardized scale). Semi-structured interviews with 15 participants from resource-limited environments explored accessibility and inclusivity challenges. Contribution: This research contributes to understanding technology-enhanced language learning in immersive environments, challenging assumptions about digital literacy’s impact on academic performance and providing insights into the long-term effects of metaverse-based learning on language skills. Findings: The results revealed significant improvements in reading, writing, and speaking skills among the experimental group, with reading and writing gains maintained over six months. While participants showed substantial improvement in digital literacy, no significant relationship was found between digital literacy and academic performance. Qualitative analysis uncovered two key themes: inclusive social interaction opportunities and technical barriers in the learning process. Recommendations for Practitioners: To maximize the effectiveness of metaverse-based language learning, three key recommendations emerge: institutions should prioritize robust technological infrastructure alongside comprehensive training for both educators and students, educational platforms should be developed with greater inclusivity and customization options, and regular practice sessions or hybrid learning models should be implemented to maintain speaking skills over time. Recommendation for Researchers: Future research directions should focus on three critical areas: conducting longer-term studies to better understand how metaverse-based learning affects academic performance and language proficiency, exploring the complex relationship between digital literacy and academic achievement through alternative success measures, and investigating how adaptive learning systems within metaverse environments can effectively address diverse learner needs. Impact on Society: The findings suggest that metaverse technology has the potential to revolutionize language education, making it more engaging and effective. However, they also highlight the need to address technical challenges and ensure inclusivity to prevent new forms of educational inequality. Future Research: Future investigation should address three key areas: examining how digital skills gained in metaverse environments transfer to real-world contexts and other academic disciplines, developing standardized assessment tools for evaluating language proficiency and digital literacy in immersive virtual settings, and studying the long-term impact of metaverse-based learning across different age groups and educational levels.

Tactile codes and devices are widely used in virtual reality, robotics, and telemanipulation by blind and sighted people. Based upon the Braille code, we provide a rigorous analysis of tactile spatial codes, and propose a code that is improved in the information theoretical sense. Based upon the criterion of a minimum average code word weight, we derive a code that provides improved reliability and longer lifetime of tactile devices. Such an approach can be also employed for robot hands, and tactile feedback in virtual reality and dextrous telemanipulation.

This paper presents the first results on a user study in which people with visual impairments (PVI) explored a virtual environment (VE) by walking in a virtual reality (VR) treadmill. As recently suggested, we have now acquired first results from our feasibility study investigating this walk-in-place interaction. This represents a new, more intuitive way of for example virtually exploring unknown spaces in advance. Our prototype consists of off-the-shelf VR components (i.e., treadmill, headphones, glasses, and controller) providing a simplified white cane simulation and was tested by six visually impaired subjects. Our results indicate that this interaction is yet difficult, but promising and an important step to make VR more and better usable for PVIs. As an impact on the CHI community, we would like to make this research field known to a wider audience by sharing our intermediate results and suggestions for improvements, on some of which we are already working on.

We propose a concept of a novel interaction strategy for providing rich haptic feedback in Virtual Reality (VR), when each user's finger is connected to micro-quadrotor with a wire. Described technology represents the first flying wearable haptic interface. The solution potentially is able to deliver high resolution force feedback to each finger during fine motor interaction in VR. The tips of tethers are connected to the centers of quadcopters under their bottom. Therefore, flight stability is increasing and the interaction forces are becoming stronger which allows to use smaller drones.

In recent years, providing additional visual feedback about the interaction forces has been found to offer benefits to haptic-assisted teleoperation. However, there is limited insight into the effects of the design of force feedback-related visual cues and the type of visual display on the performance of teleoperation of robotic arms executing industrial tasks. In this study, we provide new insights into this interaction by extending these findings to the haptic assistance teleoperation of a simulated robotic arm in a virtual environment, in which the haptic assistance is comprised of a set of virtual fixtures. We design a novel method for providing visual cues about the interaction forces to complement the haptic assistance and augment visual feedback in virtual reality with a head-mounted display. We evaluate the visual cues method and head-mounted display method through human factors experiments in a teleoperated dross removal use case. The results show that both methods are beneficial for task performance, each of them having stronger points in different aspects of the operation. The visual cues method was found to significantly improve safety in terms of peak collision force, whereas the head-mounted display additionally improves the performance significantly. Furthermore, positive scores of the subjective analysis indicate an increased user acceptance of both methods. This work provides a new study on the importance of visual feedback related to (interaction) forces and spatial information for haptic assistance and provides two methods to take advantage of its potential benefits in the teleoperation of robotic arms.

This paper outlines a study into the effects of co-location (the term `co-location' is used throughout to refer to the co-location of haptic and visual sensory modes, except where otherwise specified) of haptic and visual sensory modes in VR simulations. The study hypothesis is that co-location of these sensory modes will lead to improved task performance within a VR environment. Technical challenges and technological limitations are outlined prior to a description of the implementation adopted for this study. Experiments were conducted to evaluate the effect on user performance of co-located haptics (force feedback) in a 3D virtual environment. Results show that co-location is an important factor, and when coupled with haptic feedback the performance of the user is greatly improved.

It is known that systematic visual distortions using virtual reality technologies, prisms or mirrors, may have therapeutic effects for patients suffering from stroke or body image identity disorders. However, there are few studies which directly investigate neural activity changes during visual feedback manipulation. In the present study we created an experimental setup for investigating the effects of systematic virtual reality-mediated visual feedback manipulation of finger movements on cortical activity. We performed tests with two healthy female subjects who performed a line-tracking task under four conditions manipulating visual feedback of their own hand. To investigate hemodynamic responses in motor areas during the line tracking task we used functional near-infrared spectroscopy (fNIRS). We predicted that viewing larger or smaller virtual movements of fingers, compared to the real movements, would affect activity in motor areas and thus the hemodynamic response. Our preliminary results showed changes in the hemodynamic responses between stimulation period and baseline. There were indications of possible differences between conditions, and also of adaptation effects within conditions. However these effects were not significant in our preliminary data and we therefore need to collect additional data to draw further conclusions.

Pseudo haptic sensation is an illusion based on visual stimuli. In virtual environments the principle of pseudo haptic feedback is used to simulate material properties such as stiffness, mass and friction. Transfering the principle of pseudo haptic feedback to real haptic teleoperation systems can provide a haptic sensation for properties of material, manipulated without active haptic feedback. Hence, the implementation of pseudo haptic feedback to teleoperation systems reduces the requirements for kinematic structures and actuators of the used haptic user interface.

Virtual reality (VR) and augmented reality (AR) systems are increasingly available to neurosurgeons. These systems may provide opportunities for technical rehearsal and assessments of surgeon performance. The assessment of neurosurgeon skill in VR and AR environments and the validity of VR and AR feedback has not been systematically reviewed. A systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines was conducted through MEDLINE and PubMed. Studies published in English between January 1990 and February 2021 describing the use of VR or AR to quantify surgical technical performance of neurosurgeons without the use of human raters were included. The types and categories of automated performance metrics (APMs) from each of these studies were recorded. Thirty-three VR studies were included in the review; no AR studies met inclusion criteria. VR APMs were categorized as either distance to target, force, kinematics, time, blood loss, or volume of resection. Distance and time were the most well-studied APM domains, although all domains were effective at differentiating surgeon experience levels. Distance was successfully used to track improvements with practice. Examining volume of resection demonstrated that attending surgeons removed less simulated tumor but preserved more normal tissue than trainees. More recently, APMs have been used in machine learning algorithms to predict level of training with a high degree of accuracy. Key limitations to enhanced-reality systems include limited AR usage for automated surgical assessment and lack of external and longitudinal validation of VR systems. VR has been used to assess surgeon performance across a wide spectrum of domains. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. AR systems have not yet been used to provide metrics for surgeon performance assessment despite potential for intraoperative integration. VR-based APMs may be especially useful for metrics that are difficult to assess intraoperatively, including blood loss and extent of resection.

The potential of robotics: A systematic review of neuroplastic changes following advanced lower limb rehabilitation in neurological disorders.