Aiding orientation performance in virtual environments with proprioceptive feedback

In toad hopping, the hindlimbs generate the propulsive force for take-off while the forelimbs resist the impact forces associated with landing. Preparing to perform a safe landing, in which impact forces are managed appropriately, likely involves the integration of multiple types of sensory feedback. In toads, vestibular and/or proprioceptive feedback is critical for coordinated landing; however, the role of vision remains unclear. To clarify this, we compare pre-landing forelimb muscle activation patterns before and after removing vision. Specifically, we recorded EMG activity from two antagonistic forelimb muscles, the anconeus and coracoradialis, which demonstrate distance-dependent onset timing and recruitment intensity, respectively. Toads were first recorded hopping normally and then again after their optic nerves were severed to remove visual feedback. When blind, toads exhibited hop kinematics and pre-landing muscle activity similar to when sighted. However, distance-dependent relationships for muscle activity patterns were more variable, if present at all. This study demonstrates that blind toads are still able to perform coordinated landings, reinforcing the importance of proprioceptive and/or vestibular feedback during hopping. But the increased variability in distance-dependent activity patterns indicates that vision is more responsible for fine-tuning the motor control strategy for landing.

The contribution of cervical proprioception, vision, and vestibular feedback to the dynamic head–trunk orientation error in the yaw direction was investigated to further the understanding over the mechanism of coordination among different sensory modalities for dynamic head–trunk orientation. To test the contribution of each sensory modality, individually and together, to dynamic head–trunk orientation, 10 healthy human subjects participated in the extended cervical joint position error test, measuring the ability of repositioning the head back to the reference orientation after 45° yaw rotation of head or trunk. The error between initial and returned angles was measured. The test was repeated under eight different conditions of sensory feedback, with or without each of three sensory modalities. Each subject completed 64 trials (8 per condition) in a random order for fair comparison. No change was found in bias when one of the three modalities was missing, while variance was largest at the lack of dynamic cervical proprioception. When two of the three modalities were missing (i.e., one of the three modalities was present), both bias and variance were minimum at the presence of cervical proprioception. Additionally, both visual and vestibular feedback was redundant (i.e., no further improvement in both bias and variance), if the other one (visual or vestibular feedback) was present with dynamic cervical proprioception. In sum, the experimental results suggest that dynamic cervical proprioception is the most significant sensory modality for reducing the dynamic head–trunk orientation error in the yaw direction.

Presents a combined system for hand and finger kinesthetic feedback for the exploration and manipulation of different kinds of rigid objects in virtual environments. A three-degrees-of-freedom haptic interface, called DeKiFeD3 (Desktop Kinesthetic Feedback Device 3), with the capability of generating kinesthetic hand feedback, is combined with a commercial haptic glove to generate additional kinesthetic feedback to the fingers of a human operator. This system results in a new quality of kinesthetic perception and needs appropriate rendering algorithms, as described in this paper. The paper also presents an approach to improve the operator's immersion into a virtual scenario by adding auditory and visual feedback.

This chapter examines virtual environments which usually represent computer simulated environments, otherwise called virtual worlds or virtual reality and its application in physical therapy. Virtual environments present computer-generated three-dimensional (3D) representation of a physical presence of places in the real world as well as in the imaginary world. Users mainly interact with virtual environments using virtual reality headsets. Many of the current virtual reality environments are primarily visual experiences display either on a computer screens or through other stereoscopic displays. Applications of virtual environments are mainly focused on artificial reality that projects users into a 3D space generated by the computer with some sort of tracking devices which may be the virtual reality headsets worn or another 3D tracking device attached to the body of the user such as the electromagnetic tracker system. Research in virtual environments has made significant impact in the entertainment industry and in the training field such as the flight simulator (Stern, 1995).

A self paced linear positioning task was used to study the effects of visual and proprioceptive feedback on learning and performance. Subjects were trained with knowledge of results (KR) and tested without it. The analysis of the absolute error scores of the no-KR trials is discussed in this paper. Visual feedback was the more effective source of sensory feedback, but proprioceptive feedback was also effective. An observation that the response did not become independent of sensory feedback as a result of learning, was interpreted as supporting Adams closed loop theory of motor learning in preference to the motor program hypothesis. Other data showed that the presence of visual feedback during learning could inhibit the later effectiveness of proprioceptive feedback.

Kinematic and EMG characteristics of simple shoulder movements with proprioception and visual feedback

This paper presents the creation of an assembly simulation environment with multisensory feedback (auditory and visual), and the evaluation of the effects of auditory and visual feedback on the task performance in the context of assembly simulation in a virtual environment (VE). This VE experimental system platform brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly task have been used as the task cases to conduct the human factor experiment, using sixteen participants. Both objective performance data (i.e., task completion time, TCT; and human performance error rate, HPER) and subjective opinions (i.e., questionnaires) on the utilization of auditory and visual feedback in a virtual assembly environment (VAE) have been gathered from the experiment. Results showed that the introduction of auditory and/or visual feedback into VAE did improve the assembly task performance. They also indicated that integrated feedback (auditory plus visual) offered better assembly task performance than either feedback used in isolation. Most participants preferred integrated feedback to either individual feedback (auditory or visual) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had a negative effect on the task performance, and easily made them frustrated.

The current study concerns the identification of possible differences in perception between the virtual and the real world in terms of the effect on brain activity. For this reason, an EEG device was used to capture participants' brain activity in different brain areas during their exposure to different virtual and real environments. The environments considered in this study portray a classroom environment with a scenario suitable for teacher training and professional development. The first aim of the experiment is to investigate if exposure to a virtual environment can affect motor, cognitive or other function of the users, and the second aim is to test if the graphics content and nature of such an environment can influence the user experience. During the study, the optimum duration of exposure in a virtual environment was also assessed by measuring the time that the brain needs to perceive and adapt to the new state. Our results, consisting of EEG data analyzed in 10 Regions of Interest (ROIs) and responses from an Igroup Presence questionnaire, indicated a significant difference in each brain area, especially in the frontal and occipital region, when a participant was exposed to a non-realistic virtual environment, compared to a realistic one, highlighting the impact of the selected virtual environment design. The results of the experiment can play an important role in defining the characteristics of optimal virtual environments for virtual reality-based training applications.

Presents the Remote Attribute Virtual Environment Library (RAVEL), a support system for the development and implementation of distributed, multi-user virtual environment (VE) applications. The system extends earlier work in low-latency communication between sub-task processes of an application to provide communication of arbitrary environment attribute information between tasks, independent of the location of the tasks involved or which task produces or consumes attribute data. In addition, the RAVEL system provides for consistent updates to a particular attribute, and provides for causally consistent ordering of events that occur in the environment, while providing the level of service efficiency required for immersive and interactive VE applications.

Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.

Context: Spastic diplegia is a common subtype of cerebral palsy (CP) and is frequently associated with impaired trunk control, leading to deficits in balance and functional activities. Improving trunk stability is a key component of rehabilitation in children with spastic diplegic CP. Aims and Objectives: To compare the effectiveness of Swiss ball exercises and Spider cage suspension therapy, when combined with conventional physiotherapy, on trunk control and functional balance in children with spastic diplegic CP. Settings and Design: A randomized controlled trial conducted at VAPMS College of Physiotherapy and RK Mission, Visakhapatnam. Subjects and Methods: Thirty children aged 6–12 years with spastic diplegic CP (Gross Motor Function Classification System Levels I–III) were randomly allocated into two groups. Group A received Swiss ball exercises along with conventional physiotherapy, while Group B received Spider cage suspension therapy along with conventional physiotherapy. Interventions were administered three times per week for 6 weeks. Trunk control and balance were assessed using the trunk control measurement scale (TCMS) and the pediatric balance scale (PBS) before and after the intervention. Statistical analysis: Paired t -test was used for within-group analysis and unpaired t -test for between-group comparison. A P < 0.05 was considered statistically significant. Results: Both groups showed statistically significant improvements in TCMS static, TCMS dynamic, and PBS scores following the intervention ( P < 0.05). However, the Spider cage therapy group demonstrated greater improvement compared to the Swiss ball group. Conclusion: Spider cage suspension therapy, by providing enhanced vestibular, proprioceptive, and visual feedback, is superior to Swiss ball exercises in enhancing trunk control and functional balance in children with spastic diplegic CP. These findings support the incorporation of Spider cage therapy into rehabilitation protocols to improve proximal stability and balance in this population.

Influence of aging and visual feedback on the stability of hand grip control in elderly adults.

In this study, we develop a wheelchair simulator that provides visual and vestibular feedback at a low cost by leveraging the combination of vection-inducing movies displayed on a head-mounted display and vestibular feedback provided by an electric-powered wheelchair. However, this simulator requires users to manually create a syn-chronized pair of a VR movie and a motion scenario (chronological control commands for the wheelchair), which necessitates consid-erable effort and experience. In this paper, we introduce a novel authoring tool that generates a pair of a VR movie and a motion scenario based on a few parameters entered by the user.

To better prepare future generations, knowledge about computers and programming are one of the many skills that are part of almost all Science, Technology, Engineering, and Mathematic programs; however, teaching and learning programming is a complex task that is generally considered difficult by students and teachers alike. One approach to engage and inspire students from a variety of backgrounds is the use of educational robots. Unfortunately, previous research presents mixed results on the effectiveness of educational robots on student learning. One possibility for this lack of clarity may be because students have a wide variety of styles of learning. It is possible that the use of kinesthetic feedback, in addition to the normally used visual feedback, may improve learning with educational robots by providing a richer, multi-modal experience that may appeal to a larger number of students with different learning styles. It is also possible, however, that the addition of kinesthetic feedback, and how it may interfere with the visual feedback, may decrease a student’s ability to interpret the program commands being executed by a robot, which is critical for program debugging. In this work, we investigated whether human participants were able to accurately determine a sequence of program commands performed by a robot when both kinesthetic and visual feedback were being used together. Command recall and end point location determination were compared to the typically used visual-only method, as well as a narrative description. Results from 10 sighted participants indicated that individuals were able to accurately determine a sequence of movement commands and their magnitude when using combined kinesthetic + visual feedback. Participants’ recall accuracy of program commands was actually better with kinesthetic + visual feedback than just visual feedback. Although the recall accuracy was even better with the narrative description, this was primarily due to participants confusing an absolute rotation command with a relative rotation command with the kinesthetic + visual feedback. Participants’ zone location accuracy of the end point after a command was executed was significantly better for both the kinesthetic + visual feedback and narrative methods compared to the visual-only method. Together, these results suggest that the use of both kinesthetic + visual feedback improves an individual’s ability to interpret program commands, rather than decreases it.

The development of autonomous robots for agricultural applications includes motion planning, fruit picking, and collision avoidance with surrounding environments, and these become challenging tasks. For harvesting applications, robust control of the manipulator is needed for the effective motion of the robot. Several combinations of Proportional(P)- Integrative(I)- Derivative(D) controllers are modelled and a simulation study was performed for trajectory tracking of a redundant manipulator in virtual agricultural environments. The article presents a comprehensive study on kinematic modelling and dynamic control of redundant manipulator for fruit-picking applications in virtual environments. The collisions with surrounding environment were eliminated using ‘bounding box technique’. The joint variables are obtained by constructing Inverse Kinematics (IK) problem and are determined using a classical optimization technique. Different controllers are modelled in the ‘Simulink’ environment and are tuned to generate error-free trajectory tracking during harvesting. The task space locations (TSLs) are considered as via-points, and joint variables at each TSLs are obtained by Sequential Quadratic Programming (SQP) technique. Joint-level trajectories are generated using Quintic and B-spline polynomials. For effective trajectory tracking, torque variations are controlled using the PID and Feedforward (FF) controller. The dynamic simulations of the robot manipulator are performed in Simscape Multibody software. Results show that the during the trajectory tracking of the manipulator, the Feed-forward controller performs best with Quintic polynomial trajectory.