Implementation of an Authoring Tool for Wheelchair Simulation with Visual and Vestibular Feedback

In most applications of virtual environments (VEs), like training and design evaluation, a good sense of orientation is needed in the VE. Orientation performance when moving around in the real world relies on visual as well as proprioceptive feedback. However, the navigation metaphors which are used to move around the VE often lack proprioceptive feedback. Furthermore, the visual feedback in a VE is often relatively poor compared to the visual feedback available in the real world. Therefore, we have quantified the influence of visual and proprioceptive feedback on orientation performance in VEs. Subjects were immersed in a virtual forest and were asked to turn specific angles using three navigation metaphors, differing in the kind of proprioceptive feedback which is provided (no proprioceptive feedback, vestibular feedback, and vestibular and kinesthetic feedback). The results indicate that the most accurate turn performance is found when kinesthetic feedback is present, in a condition where subjects use their legs to turn around. This indicates that incorporating this kind of feedback in navigation metaphors is quite beneficial. Orientation on only the visual component is most inaccurate, leading to progressively larger undershoots for larger angles.

Recent advances in the field of Virtual Reality (VR) using Head Mounted Display (HMD) has presented a new medium for immersive experiences. Designing for this medium does have its challenges, and designers and developers must adopt new design processes and pipeline for creating and manipulating virtual spaces in VR. There have been several works of research within similar fields of 3D manipulation with an HMD, but recent technological strides demand new efforts to translate prior knowledge toward the development of new and innovative tools. In this paper, we propose the design and implementation of a web-based authoring tool that exploits VR technology in the creation of the virtual spaces using an immersive Collaborative Virtual Environment (CVE). The purpose is a multi-user collaboration platform for the web browser adaptable to a wide variety of cases and purposes and which allows multiple and dissimilar stakeholders to create and manipulate virtual spaces in collaboration, either recreate real-world scenes or construct new digital environments using their imagination.

In toad hopping, the hindlimbs generate the propulsive force for take-off while the forelimbs resist the impact forces associated with landing. Preparing to perform a safe landing, in which impact forces are managed appropriately, likely involves the integration of multiple types of sensory feedback. In toads, vestibular and/or proprioceptive feedback is critical for coordinated landing; however, the role of vision remains unclear. To clarify this, we compare pre-landing forelimb muscle activation patterns before and after removing vision. Specifically, we recorded EMG activity from two antagonistic forelimb muscles, the anconeus and coracoradialis, which demonstrate distance-dependent onset timing and recruitment intensity, respectively. Toads were first recorded hopping normally and then again after their optic nerves were severed to remove visual feedback. When blind, toads exhibited hop kinematics and pre-landing muscle activity similar to when sighted. However, distance-dependent relationships for muscle activity patterns were more variable, if present at all. This study demonstrates that blind toads are still able to perform coordinated landings, reinforcing the importance of proprioceptive and/or vestibular feedback during hopping. But the increased variability in distance-dependent activity patterns indicates that vision is more responsible for fine-tuning the motor control strategy for landing.

The contribution of cervical proprioception, vision, and vestibular feedback to the dynamic head–trunk orientation error in the yaw direction was investigated to further the understanding over the mechanism of coordination among different sensory modalities for dynamic head–trunk orientation. To test the contribution of each sensory modality, individually and together, to dynamic head–trunk orientation, 10 healthy human subjects participated in the extended cervical joint position error test, measuring the ability of repositioning the head back to the reference orientation after 45° yaw rotation of head or trunk. The error between initial and returned angles was measured. The test was repeated under eight different conditions of sensory feedback, with or without each of three sensory modalities. Each subject completed 64 trials (8 per condition) in a random order for fair comparison. No change was found in bias when one of the three modalities was missing, while variance was largest at the lack of dynamic cervical proprioception. When two of the three modalities were missing (i.e., one of the three modalities was present), both bias and variance were minimum at the presence of cervical proprioception. Additionally, both visual and vestibular feedback was redundant (i.e., no further improvement in both bias and variance), if the other one (visual or vestibular feedback) was present with dynamic cervical proprioception. In sum, the experimental results suggest that dynamic cervical proprioception is the most significant sensory modality for reducing the dynamic head–trunk orientation error in the yaw direction.

In the present study, the function of the somatosensory feedback system in postural control was investigated. For the sake of simplicity, the present study considered only balancing in the anteroposterior direction using the ankle strategy, in which the ankle moment is mainly used to maintain balance. To suppress the vestibular and visual feedback paths, a subject stood on a force-measuring platform with a fixed back support. Because the subject's body was immovable under these conditions, the subject controlled a computer model that simulated the subject's load at the ankles. Information about the sway angle of the model was fed through the somatosensory feedback path. Frequency response functions of the ankle moment in response to the sway angle were calculated. The experimental results suggest that the human somatosensory feedback system has derivative characteristics and, consequently, can maintain an upright posture by itself. The results were compared with those of previous studies on vestibular and visual feedback systems. The comparison reveals that subject-to-subject variance in the somatosensory system is significantly smaller than that in the other systems. This may indicate that the somatosensory feedback is the most automatic of the systems and plays a dominant role when a subject maintains an upright posture using the ankle strategy.

The use of Head-Mounted Displays (HMDs) has rapidly expanded across various sectors, including entertainment, education, healthcare, and training. HMDs provide real-world simulations that enhance learning processes, surgical simulations, and doctor training, and are effective for industrial training. However, the use of these devices can negatively impact posture and muscle activity, posing a risk of increased muscle strain and injury to the musculoskeletal system. This study aims to evaluate the effects of HMD use on posture and muscle activity by analyzing articles published between January 2019 until September 2024. Through the PRISMA method, 15 final articles from Scopus database were analyzed, including aspects of HMDs, physical characteristics, experimental characteristics, and impact of use on posture and muscle activity. The results indicate that the use of non-ergonomically designed HMDs can increase muscle strain in the neck, back, and eyes, as well as affect body posture. Several factors influencing this impact include headset weight, visual effects, field of view, and user body movement. The analysis showed that lighter and ergonomically designed headsets can reduce the risk of musculoskeletal injuries and improve user comfort. Visual effects and field of view of HMDs also play a key role in reducing muscle strain, especially through visual feedback and postural interaction with an optimal field of view. Ergonomic adjustments, such as user body movement, can help alleviate discomfort and reduce injury risk. This study’s findings provide recommendations for developers to consider ergonomic factors in HMDs design in VR or AR to create safe and comfortable user experiences.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

In recent years, providing additional visual feedback about the interaction forces has been found to offer benefits to haptic-assisted teleoperation. However, there is limited insight into the effects of the design of force feedback-related visual cues and the type of visual display on the performance of teleoperation of robotic arms executing industrial tasks. In this study, we provide new insights into this interaction by extending these findings to the haptic assistance teleoperation of a simulated robotic arm in a virtual environment, in which the haptic assistance is comprised of a set of virtual fixtures. We design a novel method for providing visual cues about the interaction forces to complement the haptic assistance and augment visual feedback in virtual reality with a head-mounted display. We evaluate the visual cues method and head-mounted display method through human factors experiments in a teleoperated dross removal use case. The results show that both methods are beneficial for task performance, each of them having stronger points in different aspects of the operation. The visual cues method was found to significantly improve safety in terms of peak collision force, whereas the head-mounted display additionally improves the performance significantly. Furthermore, positive scores of the subjective analysis indicate an increased user acceptance of both methods. This work provides a new study on the importance of visual feedback related to (interaction) forces and spatial information for haptic assistance and provides two methods to take advantage of its potential benefits in the teleoperation of robotic arms.

Under an interactive broadcasting environment, viewers have been requesting additional and linked information of individual contents in TV programs. In order to satisfy these demands from viewers, first of all, it is required to develop a new authoring tool that can edit AV (Audio Visual) contents as independent objects and add related information of the contents. In this paper, we propose an MPEG-4 based authoring tool. The characteristics of our authoring tool are that it generates an XMT file for an object and a scene description, which is then coded into an mp4 file according to the MPEG-4 specification. In order to show the usefulness of the proposed authoring tool, rote show the authored AV contents rising the reference software (IM1 2D player) provided by MPEG.

Tactile feedback is pivotal for grasping and manipulation in humans. Providing functionally effective sensory feedback to prostheses users is an open challenge. Past paradigms were mostly based on vibro- or electrotactile stimulations. However, the tactile sensitivity on the targeted body parts (usually the forearm) is greatly less than that of the hand/fingertips, restricting the amount of information that can be provided through this channel. Visual feedback is the most investigated technique in motor learning studies, where it showed positive effects in learning both simple and complex tasks; however, it was not exploited in prosthetics due to technological limitations. Here, we investigated if visual information provided in the form of augmented reality (AR) feedback can be integrated by able-bodied participants in their sensorimotor control of a pick-and-lift task while controlling a robotic hand. For this purpose, we provided visual continuous feedback related to grip force and hand closure to the participants. Each variable was mapped to the length of one of the two ellipse axes visualized on the screen of wearable single-eye display AR glasses. We observed changes in behavior when subtle (i.e., not announced to the participants) manipulation of the AR feedback was introduced, which indicated that the participants integrated the artificial feedback within the sensorimotor control of the task. These results demonstrate that it is possible to deliver effective information through AR feedback in a compact and wearable fashion. This feedback modality may be exploited for delivering sensory feedback to amputees in a clinical scenario.

Researchers at the University of Queensland, Australia, have developed and refined an optical feedback technique for measuring surface roughness. We spoke to one of the authors, Jeremy Herbert, to find out more. Optical feedback interferometry is a non-contact sensing technique that uses a laser as not only a coherent light source, but also as a receiver of information. The beam from the laser is reflected off a target where it is modulated by some property of the material. This light is then reinjected back into the laser cavity where it mixes with the standing wave and disturbs the normal operation of the laser. By monitoring this disturbance through either the output power or the terminal voltage of the laser, information about the target can be determined. Schematic of the roughness measurement experiment showing the direction of translation as parallel to the lasing axis The primary advantage of optical feedback is that a sensor can detect changes in optical path length on the order of nanometres, and/or minute changes in target reflectivity using compact and low cost components; in some configurations, a laser diode is the only optical component. This allows extremely sensitive measurement of quantities such as displacement, velocity and refractive index of transparent media. Another more recent application is the imaging of targets using exotic wavelengths (such as terahertz radiation) for which simple detectors are not widely available. The sensitivity of instrumentation based on optical feedback is a double-edged sword. While this type of sensor can measure minute changes in the properties of the target, it is also very sensitive to changes in environmental parameters such as temperature and vibration. A common approach to minimise the effects of these unwanted influences on the output signal is to perform repeated measurements and average them. Unfortunately in some circumstances, such as the one outlined in the article, it is not possible to perform repeated measurements, and so an alternative strategy is required. Roughness target locations where a focused translating beam would experience higher (green) and lower reflectivity (red). The black arrow shows the direction of scanning. In the article, a method of determining information about the roughness of a surface using optical feedback was described and demonstrated. The method does not rely on repeated measurement, but instead uses the windowed standard deviation of the optical feedback signal to track the effective reflectivity of the surface as it moves. A method to compute the windowed standard deviation in a real-time manner was presented in an earlier article also published in Electronics Letters (Herbert, J, Wilson, S, Rakic, A and Taimre, T, ‘FPGA implementation of a high-speed, real-time, windowed standard deviation filter’, Electronics Letters, 2016, 52, (1), pp. 22–23). This work is significant because it demonstrates a viable method for the use of optical feedback signals to measure surface roughness without necessitating repeated measurements of the target. It provides evidence for the idea that an online surface roughness sensor could be developed, with the small size and low cost advantages of laser feedback, for use in the manufacturing process of objects with specific roughness tolerances. More broadly, this approach could be applicable to other scenarios in which repeating a measurement is impossible, such as in determining information about a particle randomly passing through the beam. Possible applications of this work in the short term could be in the area of tribology, the measurement of mechanical wear, and in manufacturing processes that require tight control of surface roughness. Longer-term applications are also possible in areas where the measurement quantity of interest can be modelled as the problem of surface roughness, such as in the measurement of particle size. Since this work was completed, further efforts have been directed toward applying the methods in the article to other scenarios in which repeated measurements are not possible. Examples of these situations include the characterisation of uniquely shaped particles passing through the beam, and in the online control of manufacturing processes such as polishing. Optical feedback sensors are unusual in that they are able to measure nanometre-sized changes in optical path length, and yet can be constructed using a small number of compact and relatively inexpensive components. As manufacturers compete to have the most effective quality control and manufacturing processes, sensing techniques such as the one outlined in this article become more and more relevant. Over the coming years, further work in this area could result in low cost sensors that could improve the yield of manufacturing processes through the increased availability of information about process parameters.

Current virtual and augmented reality head-mounted displays usually include no or only a single vibration motor for haptic feedback and do not use it for guidance. We present HapticHead, a system utilizing multiple vibrotactile actuators distributed in three concentric ellipses around the head for intuitive haptic guidance through moving tactile cues. We conducted three experiments, which indicate that HapticHead vibrotactile feedback is both faster (2.6 s vs. 6.9 s) and more precise (96.4% vs. 54.2% success rate) than spatial audio (generic head-related transfer function) for finding visible virtual objects in 3D space around the user. The baseline of visual feedback is as expected more precise (99.7% success rate) and faster (1.3 s) in comparison, but there are many applications in which visual feedback is not desirable or available due to lighting conditions, visual overload, or visual impairments. Mean final precision with HapticHead feedback on invisible targets is 2.3° compared to 0.8° with visual feedback. We successfully navigated blindfolded users to real household items at different heights using HapticHead vibrotactile feedback independently of a head-mounted display.

We present the results of an empirical study investigating the effect of visual feedback and body postures on gesture interaction techniques in a dual task setup found, for example, in wearable computing. The conducted experiment uses a novel apparatus called Hot Wire that allows retaining the properties of wearable computing even in laboratory environments. Visual feedback was found to impair user performance and caused users to be caught in an attention demanding closed feedback loop once presented in a head-mounted display. Even though continuous feedback was not necessary for gesture interaction, users were unable to ignore it and remain focused on the primary task. The design of an alternative gesture recognition method using a body-centric frame of reference instead of a conventional static one to improve usability, is shown to have an opposed impact both on the performance and subjective perception of users. The presence of novel devices in gesture interaction, such as data gloves, is found to be a major source of erroneous gesture recognition due to unpredictable user behavior. Our detailed result discussion provides guidelines for designing better gesture interaction.

We developed a new respiratory-gated positron emission tomography (PET) imaging method (RGV-PET) that phase-based respiratory gated PET imaging (RG-PET) combine with head-mounted display (HMD)-guided "visual feedback." The purpose of this study was to investigate whether RGV-PET is effective at improving the quantitative measurement of tracer uptake in tumors using the phase-based respiratory gating method. Of the 41 enrolled patients with hepatobiliary or pancreatic cancer, 20 patients underwent RGV-PET and the remaining 21 patients underwent RG-PET. We measured the peak standardized uptake value (SUVpeak) of the primary lesion in each five bins obtained from both RG-PET and RGV-PET. The SUVpeak change rate calculated based on the ungated PET imaging. To evaluate the quantitative variation, the coefficient of variation of the SUVpeak change rate was compared between RG-PET and RGV-PET. In addition, we performed qualitative evaluation using visual score for the incidence of artifacts on four-dimensional-CT. The coefficient of variation of the average SUVpeak change rate in RGV-PET was 7.01 ± 4.43, which was significantly lower than the values in RG-PET (10.72 ± 5.74, p < 0.05). A significant improvement in the SUVpeak change rate of RGV-PET was obtained in bins 1 and 2 compared to RG-PET ( p < 0.05). The visual score of RGV-PET was significantly lower than RG-PET ( p < 0.05). RGV-PET was effective for respiratory stabilization and improved respiratory gating imaging quality. The RGV-PET is applicable to various PET/CT respiratory gating imaging and may improve the quantitativeness of PET images.

Practical skills training in enclosure fires: An experimental study with cadets and firefighters using CAVE and HMD-based virtual training simulators