A two visual systems approach to understanding voice and gestural interaction

The two visual systems hypothesis in neuroscience suggests that pointing without visual feedback may be less affected by spatial visual illusions than cognitive interactions such as judged target location. Our study examined predictions of this theory for target localization on a large-screen display. We contrasted pointing interactions under varying levels of visual feedback with location judgments of targets that were surrounded by an offset frame. As predicted by the theory, the frame led to systematic errors in verbal report of target location but not in pointing without visual feedback for some participants. We also found that pointing with visual feedback produced a similar level of error as location judgments, while temporally lagged visual feedback appeared to reduce these errors somewhat. This suggests that pointing without visual feedback may be a useful interaction technique in situations described by the two visual systems literature, especially with large-screen displays and immersive environments.

Visual illusions affect both movement planning and on-line control: A multiple cue position on bias and goal-directed action

This paper concerns local yet systematic problems of contrastive underdetermination of model choice in cognitive neuroscience debates about the so-called two visual systems hypothesis. The underdetermination problem is systematically generated by the way certain assumptions about the representationalist nature of computation are translated into experimental practice. The problem is that behavioural data underdetermine the choice between competing representational models. In this paper, I diagnose how these assumptions generate underdetermination problems in the choice between competing functional models of perception–action. Using the tools of philosophy of science, I describe the type of underdetermination and sketch a possible cure.

In environments where animals have a high probability of encountering temperatures close to their thermal limits, animals with the capacity to minimize the chances of encountering these temperatures will be advantaged. We investigated the physiological and behavioral mechanisms used by flat rock spiders,Morebilus plagusius, to avoid encountering lethal or sub‐lethal temperatures on sandstone outcrops by measuring their critical thermal maximum (CTMax) in spring and summer and quantifying their retreat rock use.Morebilus plagusiushas a high CTMax, which acclimatizes seasonally (spring CTMax: 48.3° ± 0.2°C; summer CTMax: 49.6° ± 0.3°C). By measuring the operative temperatures available to spiders underneath rocks in spring and summer and quantifying retreat rock use by adult spiders we found that random use of rocks by spiders would frequently expose them to temperatures above their thermal tolerances, especially in summer. Spiders non‐randomly used rocks as diurnal retreats, with rocks with a large area and a rock substratum being positively correlated withM. plagusiusoccupancy. Rocks large in area provided cooler daytime shelters than exfoliated rocks with smaller area.Morebilus plagusiususes both physiological and behavioral mechanisms for avoiding lethal temperatures, and neither physiological nor behavioral mechanisms alone are enough to mitigate the increased probability of encountering lethal temperatures in summer. Our results highlight the importance of an integrative approach, incorporating behavioral, ecological and physiological mechanisms in accessing how animals avoid lethal temperatures in thermally extreme environments.

Neuropsychological findings used to motivate the “two visual systems” hypothesis have been taken to endanger a pair of widely accepted claims about spatial representation in conscious visual experience. The first is the claim that visual experience represents 3‐D space around the perceiver using an egocentric frame of reference. The second is the claim that there is a constitutive link between the spatial contents of visual experience and the perceiver’s bodily actions. In this paper, I review and assess three main sources of evidence for the two visual systems hypothesis. I argue that the best interpretation of the evidence is in fact consistent with both claims. I conclude with some brief remarks on the relation between visual consciousness and rational agency.

For efficient collaboration between participants, eye gaze is seen as being critical for interaction. Teleconferencing systems such as the AcessGrid allow users to meet across geographically disparate rooms but as of now there seems no substitute for face to face meetings. This paper gives an overview of some preliminary work that looks towards integrating eye gaze into an immersive Collaborative Virtual Environment and assessing the impact that this would have on interaction between the users of such a system. An experiment was conducted to assess the difference between users abilities to judge what objects an avatar is looking at with only head gaze being viewed and also with eye and head gaze data being displayed. The results from the experiment show that eye gaze is of vital importance to the subjects correctly identifying what a person is looking at in an immersive virtual environment. This is followed by a description of how the eye tracking system has been integrated into an immersive collaborative virtual environment and some preliminary results from the use of such a system.

With the proliferation of low-cost, consumer level, head-mounted displays (HMDs) such as Oculus VR or Sony's Morpheus, we are witnessing a reappearance of virtual reality. However, there are still important stumbling blocks that hinder the development of applications and reduce the visual quality of the results. Knowledge of human perception in virtual environments can help overcome these limitations. In this paper, within the much-studied area of perception in virtual environments, we chose to look into the less explored area of crossmodal perception, that is, the interaction of different senses when perceiving the environment. In particular, we looked at the influence of sound on visual motion perception in a virtual reality scenario. We first replicated a well-known crossmodal perception experiment, carried out on a conventional 2D display, and then extended it to a 3D head-mounted display (HMD). Next, we performed an additional experiment in which we increased the complexity of the stimuli of the previous experiment, to test whether the effects observed would hold in more realistic scenes. We found that the trend which was previously observed in 2D displays is maintained in HMDs, but with an observed reduction of the crossmodal effect. With more complex stimuli the trend holds, and the crossmodal effect is further reduced, possibly due to the presence of additional visual cues.

Visual illusions serve as a window into the altered perceptual experiences of individuals with psychiatric disorders. One such is the induced Roelofs effect, where the perceived position of a small stimulus is shifted toward the midline of a distractor frame surrounding the stimulus, as the shift is more pronounced in individuals with schizophrenia. The present work extends the induced Roelofs effect assessment to bipolar disorder. Healthy controls, patients with bipolar disorder, and patients with schizophrenia were shown a dot stimulus placed inside a frame whose horizontal position could vary, and they were asked to determine whether the dot position was to their left or to their right. The induced Roelofs effect was minimal in healthy controls, large in schizophrenia, and intermediate in bipolar disorder. No association between the effect size and psychotic symptomatology or cognitive abilities was found. Yet, a nonlinear, concave association was found between the total pharmacological load and the illusion size. The results suggest that the illusion is related to alterations in sensory processing, which are common to both bipolar disorder and schizophrenia, hence showing their potential utility in clinical evaluations, which currently tend to overlook the sensory domain.

When people collaborate with multiple large screens, gesture interactions will be used widely. However, in conventional methods of gesture interaction, when there are multiple users, simultaneous interaction is difficult. In this study we have proposed a method using a wearable mobile device which enables multi-user and hand gestures only interactions. In our system, the user wears a camera-equipped mobile device like a pendant, and interacts with a large screen.

Gamepads and 3D controllers are the main controllers used in most Virtual Environments. Despite being simple to use, these input devices have a number of limitations as fixed layout and difficulty to remember the mapping between buttons and functions. Mobile devices present interesting characteristics that might be valuable in immersive environments: more flexible interfaces, touchscreen combined with onboard sensors that allow new interaction and easy acceptance since these devices are used daily by most users. The work described in this article proposes a solution that uses mobile devices to interact with Immersive Virtual Environments for selection and navigation tasks. The proposed solution uses the mobile device camera to track the Head-Mounted-Display position and present a virtual representation of the mobile device screen; it was tested using an Immersive Virtual Museum as use case. Based on this prototype, a study was performed to compare controller based and mobile based interaction for navigation and selection showing that using mobile devices is viable in this context and offers interesting interaction opportunities.

Immersive environments are known for fostering empathy via their technological properties that offer users the opportunity to get immersed in a virtual environment and take other perspectives. As empathy and perspective-taking are considered to be the basis of moral development, the present study examines the potential of historical immersive media applications for the individual's moral development according to Kohlberg's stage model (1958). An integrative approach was used to examine the role of technological immersion as well as the recipients’ immersion (presence, transportation) in eliciting empathy and promoting moral development and behavior. Therefore, an online experiment with one factorial between-subject design was conducted in which participants (N = 289) were exposed to historical media that differed only in their extent of technological immersion; participants either read a text, or watched a video, or a 360° video. In line with previous research, results showed that technological immersion positively affects presence, while transportation was not influenced by technological aspects. Furthermore, results revealed positive effects of transportation and presence on empathy which, in turn, was positively related to moral orientation and behavior. The study indicates that immersive historical environments can promote empathy and morality due to their immersive characteristics.

Based on the analysis of sources on the problem of research, the article defines types of electronic educational resources (EER), general requirements for them, focuses on didactic and methodological requirements and existing holistic systems of psychological and pedagogical requirements for EER, presents the main types of parameters that can be used to evaluate the quality of EER, analyzes the typical structure of EER and characterizes its components. The analysis of approaches to defining the concepts of “immersiveness”, “immersive educational environment”, “immersive approach in education”, “immersive educational resources” (IER) was carried out, on the basis of which the relationship between the virtual environment and immersive learning environment was determined, the pedagogical aspects of learning in immersive environments were considered, the classification of IER was made. The main components of IER design and implementation are analyzed, and the essence of IER design and IER design methodology is determined. In the main part of the proposed general model of the methodology of IOR design and analyzed the goals, subjects of methods, stages, and content of training, forms of organization of the educational process, methods, teaching tools, and predicted results of the methodology. In the conclusion the elements of the methodology of IЕR design are presented, in particular, a complex IЕR in the form of a distance learning course, examples of designing prototypes of immersive teaching materials, etc. are considered.

In this paper, a method to link CAD models to an immersive virtual environment is proposed. CAD models cannot be viewed directly in a real-time visualization environment. CAD models have to be adapted to be viewed in an immersive environment with high quality rendering. The proposed method allows design review in application requesting high quality complex scene visualization in immersive virtual environment. Our application is dedicated to an immersive room called the MoVE (Mobile Virtual Environment). This display offers a particular place to the user. User is inside the virtual world. This position allows us to take care of the peripheral.

The primary task of human-computer interaction is to point to graphical elements. Selecting the targets precisely is an important part of interactive tasks such as virtual simulation and virtual assembly. In actual application conditions, especially in virtual reality(VR) environment, how to accurately and effectively select target objects has already been a popular research topic in recent years. As one of the most crucial interaction technologies in human-computer interaction, gesture interaction is widely used in VR environment. Based on the gesture interaction with ray-casting feedback, this paper conducted a multi-factor experiment in VR environment to explore the effects on the performance of target selection tasks under different depths and different perspectives. Three depth levels of 1 m, 2.5 m, and 6 m were set in the experiment. The positions of 9 circular target objects were determined at 20° intervals in the horizontal and vertical viewing angle, and the vertical plane in front of the participants was divided into nine areas. The size of the target at different depth levels remains the same. Participants need to click on circular target objects in different viewing angle areas at different depth levels. We used the HTC-Vive head-mounted display and Noitom HI5 gloves as the experimental equipment and recruited 14 participants with normal vision in the experiment. The results show that the higher the depth level, the higher the pointing accuracy of gesture interaction with ray-casting feedback, and the closer the target object is to the visual center, the higher the pointing accuracy. In addition, the results show that the spatial distribution of the ray-casting interactive pointing deviation, the participants’ actual point position is lower than the target point position, and the deviation of the actual point position on the right side of the viewing angle is significantly higher than that on the left and middle positions. The research results have reference value for the spatial distribution of targets for gesture interactive tasks in VR environment.

Virtual reality (VR) is one of the building information modeling (BIM) use cases that can benefit a project from the early stages of design to the construction and operation phases. VR enables designers, contractors, and owners to experience interacting with a facility before it is constructed. Currently, interaction with VR environments takes place through controllers. The equipment helps users to navigate virtual facilities and interact with the surrounding objects. Although controllers address the need for creating an interactive virtual environment, they make the experience less realistic, since real-life interactions take place through body movements, not by/through pressing buttons. This paper reports a project in which vision-based artificial intelligence (AI) technology is used to create a contactless connection between users and virtual environments. In this project, a series of body movements and gestures are mapped to the input commands of a BIM-based VR software. The paper overviews the state of the art in remote human-computer interaction (HCI) technologies and discusses both the potential and challenges of applying HCI technologies to immersive VR environments. It then introduces the architecture of the developed, contactless virtual environment and reports a case study of using this technology in an immersive setting.