Pointing and Visual Feedback for Spatial Interaction in Large-Screen Display Environments

: Hand gestures provide a useful interface for humans to interact with not only other humans but also machines. Especially for a high degree-of-freedom manipulation tasks such as the operation of 3D objects in virtual scenes, the traditional interface composed of a keyboard and mouse is neither intuitive nor easy to operate. In collaborative environments using large screen displays for display of both 3D and 2D information, participants would benefit greatly from an interface that is unencumbered, natural, and effective for communication during discussions. The goal of this project was to investigate the feasibility of implementing an image triangulation technique to track the position of a passive device pointing towards a large screen display.

It is important to consider the physiological and behavioral mechanisms that allow users to physically interact with virtual environments. Inspired by a neuroanatomical model of perception and action known as the two visual systems hypothesis, we conducted a study with two controlled experiments to compare four different kinds of spatial interaction: (1) voice-based input, (2) pointing with a visual cursor, (3) pointing without a visual cursor, and (4) pointing with a time-lagged visual cursor. Consistent with the two visual systems hypothesis, we found that voice-based input and pointing with a cursor were less robust to a display illusion known as the induced Roelofs effect than pointing without a cursor or even pointing with a lagged cursor. The implications of these findings are discussed, with an emphasis on how the two visual systems model can be used to understand the basis for voice and gestural interactions that support spatial target selection in large screen and immersive environments.

With the advent of increasing advances in visual display technology, MIL-STD-1472F, Design Criteria Standard for Human Engineering, was designated for an upgrade to capture how new technologies may have impacted its display standards. Large-Screen Displays (LSDs) were subject to review and, henceforth, the following study emerged. Six variables were chosen for inclusion in the present experiment: display type (monitor v. projector), ambient lighting (light v. dark), viewing distance (8 ft, 16 ft, 32 ft), viewing offset (0°, 25°), object type (letter, number, symbol), and object size (in terms of visual angle, i.e., 6 min, 8 min, 10 min). The variables chosen were expected to impact operator performance in terms of response time and accuracy based on stimulus legibility. The results did not support any reductions in current minimum text size, but new recommendations for preferred text size, symbol size, viewing distance, and viewing offset were established.

Effects of aging on pointing movements under restricted visual feedback conditions

We introduce a novel posture guidance office chair and evaluate the effectiveness of vibrotactile and visual feedback methods for guiding seated postures. For visually dominant office work such as typing on the computer, it is possible that delivering posture feedback visually may overload the visual sense while haptic feedback may be a viable alternative. We performed two experiments to compare vibrotactile and visual feedback--posture compliance and dual-task cognitive workload assessment. In the first experiment, our results showed no statistically significant difference in effectiveness between using vibrotactile and visual feedback to obtain postural compliance to a reference posture. In the second experiment, participants experienced typing performance and response time degradations from both types of feedback. However the differences in performance degradation were not statistically significant between the two feedback methods. We conclude that vibrotactile and visual feedback are similarly effective for guiding quasistatic postures in routine tasks such as seated office work.

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface.

In spite of growing demand for mobile data visualization, few design guidelines exist to address its many challenges including small screens and low touch interaction precision. Both of these challenges can restrict the number of data points a user can reliably select and view in more detail, which is a core requirement for interactive data visualization. In this study, we present a comparison of the conventional tap technique for selection with three variations including visual feedback to understand which interaction technique allows for optimal selection accuracy. Based on the results of the user study, we provide actionable solutions to improve interaction design for mobile visualizations. We find that visual feedback, such as selection with a handle, improves selection accuracy three- to fourfold compared to tap selection. With a 75% accuracy, users could select a target item among 176 items total using the handle, but only from 60 items using tap. On the other hand, techniques with visual feedback took about twice as long per selection when compared to tap. We conclude designers should use selection techniques with visual feedback when the data density is high and improved selection precision is required for a visualization.

Navigation in outdoor terrain is difficult due to lack of easily and uniquely identifiable landmarks. This problem is further complicated for a system with multiple robots navigating a common terrain. The paper describes a field-capable system for navigation, obstacle avoidance, simulated visual training of mobile robots, group world perception modeling using visual feedback from multiple robots, and fusion of sonar range data with vision information for the purpose of terrain learning. A neural network approach is proposed for fusion of the robots visual feedback. In this approach, each mobile robot is presumed to be equipped with one camera and sonar sensor. In the proposed technique, self-localization of the robots and localization of obstacles are performed based on the visual and sonar feedback from the neural network. Computer simulation of the technique is done with FMCell simulation software an interactive graphical simulation environment. Results of simulation runs illustrating the capabilities of this technique are provided. The technique provides a better and simplified approach for visual servoing of a multi-agent system.

Recupera algunos de los elementos menos estudiados y reconocidos en el ensayo latinoamericano: su profunda vocación de reflexión, diálogo y mediación intelectual; su plasticidad y apertura a la búsqueda del sentido; su vínculo con la sociabilidad, la conversación y la generación de un espacio simbólico de encuentro; su capa cidad de establecer una nueva retroalimentación entre lectura y escritura.

We propose an interactive visualization approach to finding a mathematical model for a real world process, commonly known in the field of control theory as system identification. The use of interactive visualization techniques provides the modeller with instant visual feedback which facilitates the model validation process. When working interactively with such large data sets, as are common in system identification, methods to handle this data efficiently are required. We are developing approaches based on data streaming to meet this need.

Typing in Japanese is a difficult process for novice and intermediate learners of Japanese dueto the writing system of the Japanese language and its comparatively involved input method on akeyboard. Considering that spell checkers, which enable the user to check and correct their ownerrors and select the correct kanji word, are designed for native speakers, the learners of Japaneseas a foreign language (JFL) may not recognize their spelling errors and are thus unable to selfcorrect using this built-in tool.The present study addresses this problem and conducts an experiment to evaluate theeffectiveness of a visual feedback tool by its error recognition rate on the learners spelling errorswhen typing in Japanese. The participants were 46 beginner level JFL learners in a third semesterJapanese course, and the majority consist of native speakers of Chinese or English. Theparticipants participated in two experimental sessions. In both sessions, participants were audiorecorded while reading aloud a list of words in Japanese for pronunciation analysis and screenrecorded while typing the same list of Japanese words. These recordings are used to analyze thecharacteristics of error patterns in both pronunciation and typing. During the typing sessions, visualfeedback is provided to the participants via a customized dictionary tool when participants makea spelling error.The results show that regardless of the native language, the learners have difficulty on certainwords that include long vowels or double consonants. The recorded error patterns align with thefindings of previous studies (Hatasa, 2001; Nakazawa, 2003; Tsuchiya, 2000), and the visualfeedback showed an average error recognition rate of 76% of the participants’ spelling errors. Theparticipants also assessed the dictionary tool in terms of usability, and their responses indicate thatsuch tools are very useful during typing. The researcher concludes that using a visual feedbackdictionary tool is effective in recognizing the spelling errors of the learners when typing, and itincreases the learner’s awareness of spelling accuracy.

In an initial study, we characterized the properties of human reach motion in the presence or absence of visuo-haptic feedback in real and Immersive Virtual Environments (IVEs) or virtual reality within a participant’s maximum arm reach. Our goal is to understand how physical reaching actions to the perceived location of targets in the presence or absence of visuo-haptic feedback are different between real and virtual viewing conditions. Typically, participants reach to the perceived location of objects in the three-dimensional (3D) environment to perform selection and manipulation actions during 3D interaction in applications such as virtual assembly or rehabilitation. In these tasks, participants typically have distorted perceptual information in the IVE as compared to the real world, in part due to technological limitations such as minimal visual field of view, resolution, latency, and jitter. In an empirical evaluation, we asked the following questions: (i) how do the perceptual differences between virtual and real world affect our ability to accurately reach to the locations of 3D objects, and (ii) how do the motor responses of participants differ between the presence or absence of visual and haptic feedback? We examined factors such as velocity and distance of physical reaching behavior between the real world and IVE, both in the presence or absence of visuo-haptic information. The results suggest that physical reach responses vary systematically between real and virtual environments, especially in situations involving the presence or absence of visuo-haptic feedback. The implications of our study provide a methodological framework for the analysis of reaching motions for selection and manipulation with novel 3D interaction metaphors and to successfully characterize visuo-haptic versus non-visuo-haptic physical reaches in virtual and real-world situations.

Researchers at the University of Queensland, Australia, have developed and refined an optical feedback technique for measuring surface roughness. We spoke to one of the authors, Jeremy Herbert, to find out more. Optical feedback interferometry is a non-contact sensing technique that uses a laser as not only a coherent light source, but also as a receiver of information. The beam from the laser is reflected off a target where it is modulated by some property of the material. This light is then reinjected back into the laser cavity where it mixes with the standing wave and disturbs the normal operation of the laser. By monitoring this disturbance through either the output power or the terminal voltage of the laser, information about the target can be determined. Schematic of the roughness measurement experiment showing the direction of translation as parallel to the lasing axis The primary advantage of optical feedback is that a sensor can detect changes in optical path length on the order of nanometres, and/or minute changes in target reflectivity using compact and low cost components; in some configurations, a laser diode is the only optical component. This allows extremely sensitive measurement of quantities such as displacement, velocity and refractive index of transparent media. Another more recent application is the imaging of targets using exotic wavelengths (such as terahertz radiation) for which simple detectors are not widely available. The sensitivity of instrumentation based on optical feedback is a double-edged sword. While this type of sensor can measure minute changes in the properties of the target, it is also very sensitive to changes in environmental parameters such as temperature and vibration. A common approach to minimise the effects of these unwanted influences on the output signal is to perform repeated measurements and average them. Unfortunately in some circumstances, such as the one outlined in the article, it is not possible to perform repeated measurements, and so an alternative strategy is required. Roughness target locations where a focused translating beam would experience higher (green) and lower reflectivity (red). The black arrow shows the direction of scanning. In the article, a method of determining information about the roughness of a surface using optical feedback was described and demonstrated. The method does not rely on repeated measurement, but instead uses the windowed standard deviation of the optical feedback signal to track the effective reflectivity of the surface as it moves. A method to compute the windowed standard deviation in a real-time manner was presented in an earlier article also published in Electronics Letters (Herbert, J, Wilson, S, Rakic, A and Taimre, T, ‘FPGA implementation of a high-speed, real-time, windowed standard deviation filter’, Electronics Letters, 2016, 52, (1), pp. 22–23). This work is significant because it demonstrates a viable method for the use of optical feedback signals to measure surface roughness without necessitating repeated measurements of the target. It provides evidence for the idea that an online surface roughness sensor could be developed, with the small size and low cost advantages of laser feedback, for use in the manufacturing process of objects with specific roughness tolerances. More broadly, this approach could be applicable to other scenarios in which repeating a measurement is impossible, such as in determining information about a particle randomly passing through the beam. Possible applications of this work in the short term could be in the area of tribology, the measurement of mechanical wear, and in manufacturing processes that require tight control of surface roughness. Longer-term applications are also possible in areas where the measurement quantity of interest can be modelled as the problem of surface roughness, such as in the measurement of particle size. Since this work was completed, further efforts have been directed toward applying the methods in the article to other scenarios in which repeated measurements are not possible. Examples of these situations include the characterisation of uniquely shaped particles passing through the beam, and in the online control of manufacturing processes such as polishing. Optical feedback sensors are unusual in that they are able to measure nanometre-sized changes in optical path length, and yet can be constructed using a small number of compact and relatively inexpensive components. As manufacturers compete to have the most effective quality control and manufacturing processes, sensing techniques such as the one outlined in this article become more and more relevant. Over the coming years, further work in this area could result in low cost sensors that could improve the yield of manufacturing processes through the increased availability of information about process parameters.

A seismic simulator based on a 3-degrees-of-freedom (3-DOF) motion base is presented in an immersive augmented reality (AR) environment to provide effective earthquake response training. To mimic an environment similar to that of an actual earthquake, the proposed system provides users with precisely synchronized visual and motion haptic feedback. The motion haptic feedback is designed using a 3-DOF motor-based actuator to generate a momentum similar to an actual earthquake. In addition, scenario-based seismic content was designed for effective visualization in an AR setting to create a more immersive environment that provides a response training program for earthquakes beyond simple experience. A seismic simulator based on a 3-degrees-of-freedom (3-DOF) motion base is presented in an immersive augmented reality (AR) environment to provide practical earthquake response training. To mimic an environment similar to that of an actual earthquake, the proposed system provides users with precisely synchronized visual and motion haptic feedback. The motion haptic feedback is designed using a 3-DOF motor-based actuator to generate a momentum similar to an actual earthquake. In addition, scenario-based seismic content was designed for effective visualization in an AR setting to create a more immersive environment that provides a response training program for earthquakes beyond simple experience.

Humans can learn and store multiple visuomotor mappings (dual-adaptation) when feedback for each is provided alternately. Moreover, learned context cues associated with each mapping can be used to switch between the stored mappings. However, little is known about the associative learning between cue and required visuomotor mapping, and how learning generalises to novel but similar conditions. To investigate these questions, participants performed a rapid target-pointing task while we manipulated the offset between visual feedback and movement end-points. The visual feedback was presented with horizontal offsets of different amounts, dependent on the targets shape. Participants thus needed to use different visuomotor mappings between target location and required motor response depending on the target shape in order to “hit” it. The target shapes were taken from a continuous set of shapes, morphed between spiky and circular shapes. After training we tested participants performance, without feedback, on different target shapes that had not been learned previously. We compared two hypotheses. First, we hypothesised that participants could (explicitly) extract the linear relationship between target shape and visuomotor mapping and generalise accordingly. Second, using previous findings of visuomotor learning, we developed a (implicit) Bayesian learning model that predicts generalisation that is more consistent with categorisation (i.e. use one mapping or the other). The experimental results show that, although learning the associations requires explicit awareness of the cues’ role, participants apply the mapping corresponding to the trained shape that is most similar to the current one, consistent with the Bayesian learning model. Furthermore, the Bayesian learning model predicts that learning should slow down with increased numbers of training pairs, which was confirmed by the present results. In short, we found a good correspondence between the Bayesian learning model and the empirical results indicating that this model poses a possible mechanism for simultaneously learning multiple visuomotor mappings.