The relationship between covert and overt attention in endogenous cuing

PASSIVE VISION AND ACTIVE VISION 1.1 Introduction 1.2 Passive vision 1.3 Visual attention 1.4 Active vision 1.5 Active vision and vision for action 1.6 Outline of the book BACKGROUND TO ACTIVE VISION 2.1 Introduction 2.2 The inhomogeneity of the visual projections 2.3 Parallel visual pathways 2.4 The oculomotor system 2.5 Saccadic eye movements 2.6 Summary VISUAL SELECTION, COVERT ATTENTION AND EYE MOVEMENTS 3.1 Covert and overt attention 3.2 Covert spatial attention 3.3 The relationship between covert and overt attention 3.4 Speed of attention 3.5 Neurophysiology of attention 3.6 Non-spatial attention 3.7 Active vision and attention 3.8 Summary VISUAL ORIENTING 4.1 Introduction 4.2 What determines the latency of orienting saccades? 4.3 Physiology of saccade initiation 4.4 What determines the landing position of orienting saccades? 4.5 Physiology of the WHERE system 4.6 The Findlay and Walker model 4.7 Development and plasticity VISUAL SAMPLING DURING TEXT READING 5.1 Introduction 5.2 Basic patterns of visual sampling during reading 5.3 Perception during fixations in reading 5.4 Language processing 5.5 Control of fixation duration 5.6 Control of landing position 5.7 Theories of eye control during reading 5.8 Practical aspects of eye control in reading 5.9 Overview VISUAL SEARCH 6.1 Visual search tasks 6.2 Theories of visual search 6.3 The need for eye movements in visual search 6.4 Eye movements in visual search 6.5 Ocular capture in visual search 6.6 Saccades in visual search: scanpaths 6.7 Physiology of visual search 6.8 Summary NATURAL SCENES AND ACTIVITIES 7.1 Introduction 7.2 Analytic studies of scene and object perception 7.3 Dynamic scenes and situations 7.4 Summary HUMAN NEUROPSYCHOLOGY 8.1 Blindsight 8.2 Neglect 8.3 Balint's syndrome and dorsal simultanagnosia 8.4 Frontal lobe damage 8.5 Orienting without eye movements 8.6 Summary SPACE CONSTANCY AND TRANS-SACCADIC INTEGRATION 9.1 The traditional approach: 'compensatory taking into account' 9.2 Trans-saccadic integration 9.3 Resolution of the conflicting results 9.4 Conclusion: The Active Vision Cycle 9.5 Future directions

Humans can voluntarily deploy attention to locations in visual space without moving their eyes. It has been shown that the direction of this so-called 'covert attention' can be decoded on a single-trial basis from EEG and MEG data. In this paper we explore the possibility of using covert attention as a paradigm for subject-independent brain-computer interfacing (BCI). We find that although there is an average decrease of classification performance by moving to subject-independent learning, good subjects tend to remain good, indicating that performance of our BCI system depends on the subject individual performance level rather than whether or not the BCI system is trained using data of the same subject. This suggests that a calibration session can be omitted, which cuts down preparation time and thereby increases the practical relevance of covert attention based BCIs.

Visual attention can be allocated to a location in visual space and/or to a representation of an object in the visual field, independently of their spatial location. In Posner's cueing paradigm, it is assumed that attention is moved to and then engaged at a cued location. If the target appears at an uncued location, attention first has to be disengaged from the cued location before moving to and engaging the target location. On the basis of this paradigm, we designed an experiment to measure the disengaging of attention from objects, independently of location. For this purpose we used the bistable Necker cube that can be perceived as two different object configurations, depending on the position of the front side of the cube (lower left or upper right). The subject was instructed to react when he perceived the Necker cube as a previously presented model configuration, ie a stable cube. Each condition started with a bistable cube (with equal luminance of the ribs) that gradually evolved into the model configuration, by manipulating the luminance of the ribs of the cube. Prior to this the subject was cued by a cube similar to the model in the valid condition and cued by a dissimilar cube in the invalid condition. The results showed a significant difference (F1,10=7.35; p<0.05) between the valid and invalid cue condition, indicating a significant cost for the invalid cue condition. This effect is in accord with the well-known set effect found previously with bistable figures. The difference between the valid and the invalid cue conditions will be interpreted as disengaging attention from an object. Thus, object-based components of attention can be examined by paradigms similar to Posner's paradigm for location-based attention.

The superior colliculus is an evolutionarily conserved midbrain region that is thought to mediate spatial orienting, including saccadic eye movements and covert spatial attention. Here, we reveal a role for the superior colliculus in higher-order cognition, independent of its role in spatial orienting. We trained rhesus macaques to perform an abstract visual categorization task that involved neither instructed eye movements nor differences in covert attention. We compared neural activity in the superior colliculus and the posterior parietal cortex, a region previously shown to causally contribute to abstract category decisions. The superior colliculus exhibits robust encoding of learned visual categories, which is stronger than in the posterior parietal cortex and arises at a similar latency in the two areas. Moreover, inactivation of the superior colliculus markedly impaired animals' category decisions. These results demonstrate that the primate superior colliculus mediates abstract, higher-order cognitive processes that have traditionally been attributed to the neocortex.

Traditionally, functional representations in early visual areas are conceived as retinotopic maps preserving ego-centric spatial location information while ensuring that other stimulus features are uniformly represented for all locations in space. Recent results challenge this framework of relatively independent encoding of location and features in the early visual system, emphasizing location-dependent feature sensitivities that reflect specialization of cortical circuits for different locations in visual space. Here we review the evidence for such location-specific encoding including: (1) systematic variation of functional properties within conventional retinotopic maps in the cortex; (2) novel periodic retinotopic transforms that dramatically illustrate the tight linkage of feature sensitivity, spatial location, and cortical circuitry; and (3) retinotopic biases in cortical areas, and groups of areas, that have been defined by their functional specializations. We propose that location-dependent feature sensitivity is a fundamental organizing principle of the visual system that achieves efficient representation of positional regularities in visual experience, and reflects the evolutionary selection of sensory and motor circuits to optimally represent behaviorally relevant information. Future studies are necessary to discover mechanisms underlying joint encoding of location and functional information, how this relates to behavior, emerges during development, and varies across species.

Saccade trajectory deviations and inhibition-of-return: Measuring the amount of attentional processing

While aiming and shooting, we make tiny eye movements called microsaccades that shift gaze between task-relevant objects within a small region of the visual field. However, in the brief period before pressing the trigger, microsaccades are suppressed. This might be due to the lack of a requirement to shift gaze as the retinal images of the two objects begin to overlap on the fovea. Alternatively, we might actively suppress microsaccades to prevent any disturbances in visual perception caused by microsaccades around the time of their occurrence and their subsequent effect on shooting performance. In this study we looked at microsaccade rates while participants performed a simulated shooting task under two conditions: a normal condition in which they moved their eyes freely, and an eccentric condition in which they maintained gaze on a fixed target while performing the shooting task at 5° eccentricity. As expected, microsaccade rate dropped near the end of the task in the normal viewing condition. However, we also found the same decrease for the eccentric condition in which microsaccades did not shift gaze between the task objects. Microsaccades are also produced in response to shifts in covert attention. To test whether disengagement of covert attention from the eccentric shooting location caused the drop in microsaccade rate, we monitored the location of participants’ spatial attention by using a Rapid Serial Visual Presentation (RSVP) task simultaneously at a location opposite to the shooting task. Target letter detection at the RSVP location did not improve during the drop in microsaccade rate, suggesting that covert attention was maintained at the shooting task location. We conclude that in addition to their usual gaze-shifting function, microsaccades during fine-acuity tasks might be modulated by cognitive processes other than spatial attention.

The ability to coordinate attention to events or objects between interactive social partners, referred to as social attention, is of great significance for adaptive social behaviors and nonverbal communications in our daily life, helping us to infer other person’s inner state (e.g., intentions, goals) and learn about where important events (e.g., food, danger) occur in the environment. In recent years, many studies have demonstrated that social cues (e.g., eye gaze, head orientation and walking direction of biological motion) can trigger reflexive attentional orienting effects using adapted central cueing paradigm originally designed by Posner. However, not all of us are equally adept at directing attention to where others are focusing on, and this ability is strongly impaired in individuals with autism spectrum disorder (ASD), a highly genetic neurodevelopmental disorder marked by striking social deficits and repetitive behaviors. Here we systematically reviewed recent work on abnormal social attention behaviors and its underlying neural mechanisms in ASD. We first expatiated a typical social attention behaviors indexed by covert and overt attention in ASD. It has been documented that ASD individuals tend to show reduced reflexive orienting effect manifesting itself in both covert attention and overt eye movement compared to typically developing individuals (TD). Yet some controversies concerning the malfunction of social attention in ASD remain to be resolved, based on some evidence demonstrating comparable orienting effect between ASD and TD group. Then we summarized the development course of social attention in ASD. Crucially, atypical orienting to eye gaze is more likely to be observed in younger children but not older children or adults with ASD. It is reasonable to postulate that ASD individuals may acquire this ability through overlearning the association between social cues and targets in everyday life as they grow older. Furthermore, we discussed the neural basis of abnormal social attention behaviors in ASD. Using a combination of psychophysical paradigms and neuroimaging techniques, researchers have reported atypical neural activities in superior temporal gyrus and prefrontal cortex under the supraliminal condition as well as abnormal activation in amygdala under the subliminal condition in the brain of ASD. Moreover, the ASD group showed much less difference in activation of frontoparietal attention networks between social and nonsocial attention task than the TD group, implying disruptive social attention in ASD. Finally, several perspectives on further investigations were put forward given the controversies and insufficient evidence concerning the malfunction of social attention in ASD. Future studies should employ multiple types of social cues (e.g., eye gaze and walking direction of biological motion) in conjunction with more ecological paradigms to investigate conscious and non-conscious social attention behaviors from a developmental approach. More importantly, more neuroimaging studies are needed to explore the functional connections among several key cortical regions and subcortical regions underlying atypical social attention behaviors in ASD. Such efforts will help to facilitate the early diagnosis and intervention of ASD.

Directing attention to a location in space results in retinotopic activation in primary visual cortex

Social information rapidly prioritizes overt but not covert attention in a joint spatial cueing task

Fatigue resulting from strenuous exercise can impair cognition and oculomotor control. These impairments can be prevented by administering psychostimulants such as caffeine. This study used two experiments to explore the influence of caffeine administered at rest and during fatiguing physical exercise on spatial attention—a cognitive function that is crucial for task-based visually guided behavior. In independent placebo-controlled studies, cohorts of 12 healthy participants consumed caffeine and rested or completed 180 min of stationary cycling. Covert attentional orienting was measured in both experiments using a spatial cueing paradigm. We observed no alterations in attentional facilitation toward spatial cues suggesting that covert attentional orienting is not influenced by exercise fatigue or caffeine supplementation. Response times were increased (impaired) after exercise and this deterioration was prevented by caffeine supplementation. In the resting experiment, response times across all conditions and cues were decreased (improved) with caffeine. Covert spatial attention was not influenced by caffeine. Together, the results of these experiments suggest that covert attentional orienting is robust to the effects of fatiguing exercise and not influenced by caffeine. However, exercise fatigue impairs response times, which can be prevented by caffeine, suggesting that pre-motor planning and execution of the motor responses required for performance of the cueing task are sensitive to central nervous system fatigue. Caffeine improves response time in both fatigued and fresh conditions, most likely through action on networks controlling motor function.

It is well established that neural responses to visual stimuli are enhanced at select locations in the visual field. Although spatial selectivity and the effects of spatial attention are well understood for discrete tasks (e.g. visual cueing), little is known for naturalistic experience that involves continuous dynamic visual stimuli (e.g. driving). Here, we assess the strength of neural responses across the visual space during a kart‐race game. Given the varying relevance of visual location in this task, we hypothesized that the strength of neural responses to movement will vary across the visual field, and it would differ between active play and passive viewing. To test this, we measure the correlation strength of scalp‐evoked potentials with optical flow magnitude at individual locations on the screen. We find that neural responses are strongly correlated at task‐relevant locations in visual space, extending beyond the focus of overt attention. Although the driver's gaze is directed upon the heading direction at the centre of the screen, neural responses were robust at the peripheral areas (e.g. roads and surrounding buildings). Importantly, neural responses to visual movement are broadly distributed across the scalp, with visual spatial selectivity differing across electrode locations. Moreover, during active gameplay, neural responses are enhanced at select locations in the visual space. Conventionally, spatial selectivity of neural response has been interpreted as an attentional gain mechanism. In the present study, the data suggest that different brain areas focus attention on different portions of the visual field that are task‐relevant, beyond the focus of overt attention.

Covert attention modulates visual perception in early infancy.

Visuospatial neglect is a frequent and disabling disorder, mostly after stroke, that presents in impaired awareness to stimuli on one side of space. Neglect causes disability and functional dependence, even long after the injury. Improving measurements of the core attentional deficit might hold the key for better understanding of the condition and development of treatment. We present a rapid, pupillometry-based method that assesses automatic biases in (covert) attention, without requiring behavioral responses. We exploit the phenomenon that pupil light responses scale with the degree of covert attention to stimuli, and thereby reveal what draws (no) attention. Participants with left-sided neglect after right-sided lesions following stroke (n = 5), participants with hemianopia/quadrantanopia following stroke (n = 11), and controls (n = 22) were presented with two vertical bars, one of which was white and one of which was black, while fixating the center. We varied which brightness was left and right, respectively across trials. In line with the hypotheses, participants with neglect demonstrated biased pupil light responses to the brightness on the right side. Participants with hemianopia showed similar biases to intact parts of the visual field, whilst controls exhibited no bias. Together, this demonstrates that the pupil light response can reveal not only visual, but also attentional deficits. Strikingly, our pupillometry-based bias estimates were not in agreement with neuropsychological paper-and-pencil assessments conducted on the same day, but were with those administered in an earlier phase post-stroke. Potentially, we pick up on persistent biases in the covert attentional system that participants increasingly compensate for in classical neuropsychological tasks and everyday life. The here proposed method may not only find clinical application, but also advance theory and aid the development of successful restoration therapies by introducing a precise, longitudinally valid, and objective measurement that might not be affected by compensation.

Our perception of an event is often modulated by our past experience and expectations [1Ramachandran V.S. Armel C. Foster C. Stoddard R. Object recognition can drive motion perception.Nature. 1998; 395: 852-853Crossref PubMed Scopus (35) Google Scholar, 2O'Regan J.K. Noe A. A sensorimotor account of vision and visual consciousness.Behav. Brain Sci. 2001; 24: 939-973Crossref PubMed Scopus (1863) Google Scholar]. Here we used a magic trick [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar] to demonstrate how magicians can distort our subjective perception and we investigate the mechanisms behind this deception. We found that when a magician performed an illusion in which a ball was seen to vanish in the air, 63% of observers perceived the ball leave his hand, move upwards, and disappear even though the ball did not leave the magician's hand. Moreover, observers' illusory perception of the ball was determined by cues that indicated the ball's location, namely the magician's head direction, rather than the percept itself. Furthermore, eye movement records revealed participants' strategic use of social cues prior to looking at the ball. Surprisingly, however, when the ball was not physically present, observers did not look at the area where they claimed to have seen the ball vanish, suggesting that the oculomotor system was not fooled by the illusion. These results show that although people's subjective percept is determined by expectations, the oculomotor system is largely driven by accurate bottom-up information, which is consistent with the suggestion that there are separate mechanisms for perception and visuomotor control [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar]. A striking example of magicians' deception is the vanishing ball illusion, in which a ball apparently disappears in mid air. This illusion is created by pretending to throw a ball up in the air, when in fact it remains secretly palmed in the magician's hand [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar]. If done convincingly, the observer should perceive the ball as moving up in the air even though it is no longer physically present. Similarly, when observers view a moving object that suddenly disappears, the final position of the object is usually perceived as being further along the path of motion than its actual final position [5Freyd J.J. Finke R.A. Representational Momentum.J. Exp. Psychol. Learn. Mem. Cogn. 1984; 10: 126-132Crossref Scopus (464) Google Scholar], a phenomenon known as representational momentum [6Assad J.A. Maunsell J.H.R. Neuronal correlates of inferred motion in primate posterior parietal carter.Nature. 1995; 373: 518-521Crossref PubMed Scopus (181) Google Scholar]. Much of the magician's deception relies on social cuing, in terms of his head and gaze direction, and we therefore predicted that the effectiveness of the vanishing ball illusion would be influenced by the magician's social cuing. Two different versions of the trick were created that addressed the extent to which the magician's social cues are responsible for the illusion. In the social cues pro-illusion condition, on the final 'fake' throw, the magician's eyes and head followed an imaginary ball moving upwards (Figure 1 and Supplemental data available on-line with this issue). In the social cues anti-illusion condition, the magic trick was identical except that on the final 'fake' throw, the magician looked at the hand concealing the ball, rather than following the imaginary ball (see Figure 2F). We also measured people's eye movements while they watched the video of the trick, to gain an online measure of the information obtained by the visual system. Immediately after having seen the magic trick, participants were questioned as to whether they perceived the 'illusory ball' moving towards the top of the screen on the final throw.Figure 2Participants' fixation points for the social cues pro-illusion (top row) and the social cues anti-illusion (bottom row) conditions.Show full captionThe dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002).View Large Image Figure ViewerDownload Hi-res image Download (PPT) The dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002). Participants who perceived the illusory ball had a vivid recollection of seeing the ball leave the screen at the top, and typically claimed that the illusion was created by someone catching it beyond the top of the screen. 68% of the participants in the social cues pro-illusion condition experienced the illusion, significantly more than in the social cues anti-illusion condition (32%, χ2 = 5.16, p = 0.025), thus demonstrating that the illusion was mediated by the social cuing. Most of the participants claimed that they spent their entire time looking at the ball. However, these reports deviated strongly from where they were actually looking. Figure 1 shows a timeline indicating where participants were looking during the magic trick. We found a very high consistency in the eye movement patterns between the participants, suggesting that people employed similar eye movement strategies. On throws where the ball was visible, participants generally looked at the ball once it reached the apex. But rather than merely tracking the ball, most of the participants glanced at the magician's face before looking at the ball, indicating that the visual system uses information about where the magician is looking as a way of predicting the location of the ball. Our results show that an observer's percept was driven by the magician's social cuing, and that participants utilized the magician's social cues. However, there was a surprising difference in the susceptibility of these two systems to the illusion. On the final illusory throw, most participants claimed to have seen the ball at the top of the screen. The oculomotor system on the other hand was not fooled by the illusion. Figure 2 shows that participants only looked at the top of the screen when the ball was physically present. These results illustrate a remarkable dissociation between what participants claimed to have seen and the way in which their eyes behaved. Whilst their percept was predominantly based on expectations, their eyes were largely driven by the actual visual input. Our results offer a striking example of how the dissociation between vision for perception and for action, well documented in neurological patients [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar, 7Goodale M.A. Haffenden A. Frames of reference for perception and action in the human visual system.Neurosci. Biobehav. Rev. 1998; 22: 161-172Crossref PubMed Scopus (110) Google Scholar, 8Milner A.D. Goodale M.A. The Visual Brain in Action. Oxford University Press, Oxford1995Google Scholar] and neurologically intact subjects [9Aglioti S. Desouza J.F.X. Goodale M.A. Size-contrast illusions deceive the eye but not the hand.Curr. Biol. 1995; 5: 679-685Abstract Full Text Full Text PDF PubMed Scopus (721) Google Scholar], also translates to more real world situations. We would like to thank the reviewers and Cristiana Cavina-Pratesi for their valuable comments on the manuscript, and Tim Smith for his help with a control experiment. Download .pdf (.1 MB) Help with pdf files Document S1. Supplemental Experimental Procedures Download .mpg (1.09 MB) Help with mpg files Movie 1 Vanishing ball trick pro-illusion condition Download .mpg (1.12 MB) Help with mpg files Movie 2 Vanishing ball trick anti-illusion condition