Visual, auditory and haptic navigation feedbacks among older pedestrians

We routinely have physical interactions with others, whether it be handing someone a glass of water or jointly moving a heavy object together. These sensorimotor interactions between humans typically rely on visual feedback and haptic feedback. Recent single-participant studies have highlighted that the unique noise and time delays of each sense must be considered to estimate the state, such as the position and velocity, of one's own movement. However, we know little about how visual feedback and haptic feedback are used to estimate the state of another person. Here, we tested how humans utilize visual feedback and haptic feedback to estimate the state of their partner during a collaborative sensorimotor task. Across two experiments, we show that visual feedback dominated haptic feedback during collaboration. Specifically, we found that visual feedback led to comparatively lower task-relevant movement variability, smoother collaborative movements, and faster trial completion times. We also developed an optimal feedback controller that considered the noise and time delays of both visual feedback and haptic feedback to estimate the state of a partner. This model was able to capture both lower task-relevant movement variability and smoother collaborative movements. Taken together, our empirical and modeling results support the idea that visual accuracy is more important than haptic speed to perform state estimation of a partner during collaboration.NEW & NOTEWORTHY Physical collaboration between two or more individuals involves both visual and haptic feedback. Here, we investigated how visual and haptic feedback is used to estimate the movements of a partner during a collaboration task. Our experimental and computational modeling results parsimoniously support the notion that greater visual accuracy is more important than faster yet noisier haptic feedback when estimating the state of a partner.

Most prosthetic hand users are limited to visual feedback of movement performance. To characterize the benefit of vibrotactile feedback for a task that lacks haptic feedback, a virtual environment was used to experimentally manipulate visual, task-relevant haptic, and remote vibrotactile feedback on simple object manipulation for unimpaired subjects. The combination of visual and remote vibrotactile feedback was compared to visual feedback alone, and to simultaneous visual and direct haptic feedback to represent ideal performance. Visual and vibrotactile feedback resulted in improvement of most performance variables including difficulty ratings relative to visual feedback alone. However addition of sensory cues to visual feedback increased trial times and the increase was steeper for vibrotactile than for haptic feedback. Specifically, during vibrotactile feedback the velocity did not change, but the duration of execution increased due to improved performance, resulting in increased trial times. This result suggests future exploration of performance improvement and execution speed for augmented sensory feedback.

This research evaluated visual and haptic feedback for enhancing operator situational awareness during multitasking drone operations, investigating how task complexity influenced feedback effectiveness. Two virtual reality experiments were conducted: Study 1 assessed three types of visual feedback (Static, Blink, and Shake), while Study 2 compared visual, haptic, and visual-haptic multimodal feedback across varying task complexity levels determined by target density. Participants performed drone operation tasks requiring simultaneous obstacle avoidance and target counting. Results revealed that dynamic visual feedback (Shake) significantly improved primary task accuracy, reduced cognitive load, and was the preferred visual feedback type. Furthermore, haptic and visual-haptic feedback significantly shortened obstacle avoidance reaction times and decreased cognitive workload compared to visual feedback. Task complexity proved to be a critical factor: while visual feedback sufficed for low complexity scenarios, visual-haptic feedback offered distinct advantages in medium and high complexity tasks, providing insights for complexity-responsive drone interface design.

This paper presents a human-scale virtual environment (VE) with haptic feedback along with two experiments performed in the context of product design. The user interacts with a virtual mock-up using a large-scale bimanual string-based haptic interface called SPIDAR (Space Interface Device for Artificial Reality). An original self-calibration method is proposed. A vibro-tactile glove was developed and integrated to the SPIDAR to provide tactile cues to the operator. The purpose of the first experiment was: (1) to examine the effect of tactile feedback in a task involving reach-and-touch of different parts of a digital mock-up, and (2) to investigate the use of sensory substitution in such tasks. The second experiment aimed to investigate the effect of visual and auditory feedback in a car-light maintenance task. Results of the first experiment indicate that the users could easily and quickly access and finely touch the different parts of the digital mock-up when sensory feedback (either visual, auditory, or tactile) was present. Results of the of the second experiment show that visual and auditory feedbacks improve average placement accuracy by about 54 % and 60% respectively compared to the open loop case

The ergonomic design of control elements in real life has been researched in detail. Various studies exist on the optimal dimensioning, their haptic and acoustic feedback to achieve high control accuracy and user experience. But the development of products is increasingly done with virtual prototypes. Virtual reality (VR) allows these prototypes to be tested in a highly immersive environment. However, the findings from reality cannot be transferred to VR directly. For example, users in VR interact with the prototypes using controllers, which affects haptic feedback. This study investigates how rotary dials and joysticks must be designed and programmed in VR so that control tasks can be performed efficiently and generate a high user experience and perceived presence.In user tests, subjects (n = 25) evaluate the control of a joystick and a rotary dial in VR. In a virtual crane operator's cabin or at a virtual table, the subjects (f = 10, m = 15, age: 24 +- 3) perform four predefined tasks per control element. On two screens in VR, subjects see a vertical bar graph with a scale from 0 to 100 % controlled by the joystick and separately a numerical value between 0 and 100 % which is controlled by the rotary dial. The screens display the task to the subjects, e.g., "Set the value from 0 % to 42 %". According to the method “design of experiments”, 14 factors, such as vibration feedback, acoustic feedback, position of the subject or the sensitivity of the control element are systematically varied on two respectively three levels (e.g., diameter of actuator 40 mm, 80 mm or 110 mm). For each trial, the control accuracy and the time required to complete the task are determined. In addition, the perceived presence is assessed using the Slater-Usoh-Steed-Questionnaire and the user experience is surveyed using the User Experience Questionnaire. The effect of a change of level on the response parameters is investigated using multifactorial ANOVA (α = .05). Linear regression is used to calculate a mathematical relationship between factor and response parameter. These mathematical models are used to calculate which factor values can be used to achieve a high level of control accuracy with a low time requirement and a high level of user experience and perceived presence. The factors angular resolution, inclination, shape of the rotary dial and position of the subject have a highly significant effect (p ≤ .001) on the time required to complete the tasks with the rotary dial. On the control accuracy of the rotary dial, the angular resolution, the VR-controller and the interaction of angular resolution and diameter of the rotary dial have a significant effect. On the user experience, a total of six factors and two interactions have a significant effect. On the perceived presence of the subjects, the VR environment and the diameter of the rotary dial have a significant effect. The calculated optimized design is a rotary dial with vibration feedback, without acoustic feedback, with visualization of a rough knurling, an angular resolution of 10-12 degree/value, a 40 mm diameter and no inclination. Visualization of the hand should be avoided.Sensitivity, size, subject position, VR environment, and the interaction of subject position and VR environment have a significant effect on the time required to perform the control tasks with the joystick. Three factors and one interaction have a significant effect on the control accuracy of the joystick. The interaction of the factors vibration feedback and visualization of the hand has a significant effect on the perceived presence of the subjects. On user experience, nine factors and five interactions have a significant effect. The calculated optimized levels of factors for the joystick are vibration and acoustic feedback, no visual feedback, vertical handle with a height of 20-24 cm, a five-level angular resolution, a maximum deflection angle of +- 15°, a sensitivity of 8 %/sec and a visualization of the hand.The trials show a high degree of scatter. The residuals show outliers in the experiments. These deviations are mainly due to the individual previous experience of the test subjects in handling VR systems. Nevertheless, significant effects could be identified. A screening experimental design was used in this study. In a follow-up study, detailed investigations with a full factorial experimental design must be performed with the significant factors. The factors will be tested at multiple levels and with a significantly increased number of trials to further increase the accuracy of the mathematical models.

This paper proposes a new method of object manipulation in a Virtual Environment that uses a visual and haptic feedback based manipulation aid. If the displayed position of the object is constrained on a face of another object in the virtual environment, a user can place the virtual object at a precise position as easily as in a real environment. The visual feedback based manipulation aid constrains the motion of the virtual object as if it were moved in a real environment. Furthermore, in simultaneous correspondence with the visual constraints, the haptic feedback based manipulation aid constrains the motion of the user’s hand in the real environment. The proposed method introduces a magnetic metaphor: the user has the sensation of manipulating a virtual object on a rigid surface without using a large and powerful robot arm and the time consuming simulation of detailed physical phenomena. The properties of this manipulation aid based on two kinds of feedback are discussed. Experimental results show the effectiveness of our proposed method.

For educational purposes, virtual reality (VR) is often used to produce life-like experiences. However, the use of haptic feedback in educational practices for science and mathematics remained under-studied. Haptically-enabled science simulations (HESSs) enable students to physically experience the concepts being taught via haptic feedback. We present a study on the effect of a buoyancy HESS designed to aid in the understanding of basic physics concepts. We hypothesize that introducing both visual and haptic feedback of the underlying simulated forces will improve participants understanding. We investigate this hypothesis with a 2 (haptics: yes, no) × 2 (visuals: yes, no) between subjects design user study, where all participants were randomly assigned to one of the four conditions. Participants were given a pretest of buoyancy knowledge, then instructed to interact with the buoyancy simulation, then given a post-test of buoyancy knowledge. The present study is still in the process of data collection, with 40 out of 60 participants. Preliminary results highlight a significant improvement in performance of participants in the haptic-and-visual condition, while no significant differences were observed in other conditions.

PurposeHaptic feedback could improve the efficiency of needle insertion procedures by providing surgeons with enhanced sensing and guiding capabilities. A framework has been developed to provide physicians with enhanced haptic feedback during CT-guided needle insertion procedures in oncology.MethodsThe physicians encountered needle-tissue interaction and guidance forces with visual feedback to accurately reach the tumor. The force feedback to users was enhanced by amplifying several parameters in the feedback model, such as tip forces and radial forces. The study evaluated the effect of multiple haptic and visual feedback algorithms on user performance in efficiently inserting the needle. In this experimental pilot study, 12 participants including three interventional radiologists engaged in five experimental scenarios simulating a needle insertion.ResultsThe results showed that enhanced force feedback for tumor perception reduced tumor targeting error and trajectory deviation, compared to natural force feedback. This was also the case when tumor perception and haptic guidance were both enhanced. Additionally, real-time visual feedback and enhanced force feedback for guidance reduced the duration to finish the task significantly. Participants still preferred real-time visual feedback over enhanced haptic feedback cues.ConclusionsOn average, small tumors (around 3mm in diameter) can be successfully targeted with enhanced haptic feedback in the radial and axial directions. Additionally, critical regions, such as veins within the liver, can be avoided more effectively as users maintain the desired trajectory with greater accuracy.

Incidental Findings at Chest CT: A Needs Assessment Survey of Radiologists' Knowledge

The focus of this research is on the perception and control of forces produced by the hand. It is known that the change in force that subjects can reliably discriminate is approximately 6%, and that subjects are able to match forces generated by two corresponding muscle groups with an accuracy of approximately 15%. The present study was concerned with measuring the ability to control finger force using only haptic feedback and both haptic and visual feedback. It was found that subjects were surprising good at controlling forces for up to 120 s using only haptic feedback with mean absolute errors of less than 0.3 N over a force range of 2–6 N. The variation in force averaged 10.5% when haptic feedback was available and decreased to 6% when visual feedback was also provided. These results provide a framework for specifying the characteristics of force-reflecting haptic interfaces that should optimize human perception.

The ability to recognize and classify human gestures bears a huge potential of improving haptic feedback in virtual environments: when it is possible to predict how a user wants to explore or manipulate his environment both the visual and the auditive feedback may be adapted in order to enhance the immersiveness of haptic displays. Within this work a software approach is suggested that allows for such a real-time classification of gestures in a continuous data stream. A visually based tracking system is used to record the hand movements, while Hidden Markov models are applied to analyze the data. After presenting the methodological background and the software implementation, the outcome of an evaluation study is discussed. It reveals satisfying results of gesture classification and, what is particularly important, the recognition is fast enough to enable multi-modal haptic feedback.KeywordsGesture recognitionHidden Markov modelMulti-modal haptic feedbackVirtual reality

This paper presents the development and evaluation of a smartphone application designed to improve accessibility for blind users. It uses the lightweight EfficientDet-lite2 model and the comprehensive COCO dataset in order to provide real-time object detection. The novelty of the application is in the integration of haptic feedback, which is activated when users touch objects that are detected on the screen, combined with audio notifications that announce the name of the detected object in multiple languages. This multimodal feedback mechanism helps blind users to recognize, explore, and move within their environment more effectively and safely. Extensive usability and user experience evaluation was conducted with blind and blindfolded users. The evaluation assessed the usability, effectiveness, accessibility, and user satisfaction and experience of the application. Additionally, a comparative analysis was performed between the use of haptic feedback and scenarios where haptic feedback was disabled. The results show a higher level of user satisfaction, greater ease of use, and significant potential for improving the independence of blind people when the haptic feedback is enabled. The findings also suggest that the inclusion of haptic feedback significantly enhances the user experience. This study underlines the importance of multimodal feedback systems in assistive technologies and the potential of mobile applications to provide accessible solutions for blind users.

Mid-air haptic (MAH) feedback is an interesting means to provide augmented haptic feedback for gesture-based technology as it enables a sense of touch without physical contact with an actuator. Although quite some work already investigated the user experience (UX) of MAH feedback during initial encounter, we are not aware of studies testing the UX after repeated use, with regard to both pragmatic and hedonic UX, as well as emotional reactions. In this article, we tested how the UX of MAH feedback changed over the course of five weeks by collecting both questionnaire as well as interview data of 31 participants. Our results showed that MAH feedback significantly increased the enjoyment, engagement, valence and arousal of the emotional response. However, the added value of valence was due to a novelty effect as it was only significantly elevated during initial use, and not after repeated use. Interestingly, the added value of MAH feedback in terms of enjoyment, engagement and arousal remained elevated over the course of five weeks. Moreover, the interview data hinted at substantial individual differences underlying the global trends from the questionnaire data, showing the importance of combining quantitative and qualitative data when testing the UX of MAH feedback.

Implementing In-Room Visual Guidance Using a Digital Projector for Breath-Hold Treatment of Left Breast Cancer

Age differences in pedestrian navigational skills and performance: A systemic review and meta-analysis.