To bind or not to bind: Addressing the question of object representation in visual short-term memory

Visual working memory temporarily represents a continuous stream of task-relevant objects as we move through our environment performing tasks. Previous work has identified candidate neural mechanisms of visual working memory storage; however, we do not know which of these mechanisms enable the storage of objects as we sequentially encounter them in our environment. Here, we measured the contralateral delay activity (CDA) and lateralized alpha oscillations as human subjects were shown a series of objects that they needed to remember. The amplitude of CDA increased following the presentation of each to-be-remembered object, reaching asymptote at about three to four objects. In contrast, the concurrently measured lateralized alpha power remained constant with each additional object. Our results suggest that the CDA indexes the storage of objects in visual working memory, whereas lateralized alpha suppression indexes the focusing of attention on the to-be-remembered objects.

Contralateral delay activity does not reflect behavioral feature load in visual working memory

Our previous work suggests that 2 colors can be consolidated into visual short-term memory (VSTM) in parallel without a loss of memory precision, whereas consolidation of 2 orientations is performed in a strictly serial manner. Those experiments compared VSTM performance for simultaneously and sequentially presented stimuli. However, there is still controversy about whether the bandwidth for consolidation is determined by the type of information. To further investigate this issue, here we measured electroencephalography while participants attempted to consolidate 1, 2 or 4 simultaneously presented colors (Experiment 1) or orientations (Experiment 2) under limited presentation times. We used the contralateral delay activity (CDA) as an electrophysiological marker of the number of items that were consolidated. For colored stimuli, the CDA amplitude increased between set-size 1 and 2 but did not further increase for set size 4. By contrast, for orientation, the CDA amplitude remained at the set size 1 amplitude as set size increased to 2 or 4 items. Furthermore, in a long exposure duration (300 ms) condition that did not limit the consolidation process, the CDA amplitude pattern indicated that VSTM capacity is limited to about 3 colored items and about 2 orientation items in our paradigm. Thus, the CDA effects observed in the short presentation time was not limited by VSTM storage, but rather by consolidation. These results are consistent with our previous behavioral research and suggest that the bandwidth of VSTM consolidation is determined by the stimulus feature. (PsycINFO Database Record

Working memory (WM) is limited in capacity, but it is controversial whether these capacity limitations are domain-general or are generated independently within separate modality-specific memory systems. These alternative accounts were tested in bimodal visual/tactile WM tasks. In Experiment 1, participants memorized the locations of simultaneously presented task-relevant visual and tactile stimuli. Visual and tactile WM load was manipulated independently (one, two, or three items per modality), and one modality was unpredictably tested after each trial. To track the activation of visual and tactile WM representations during the retention interval, the visual contralateral delay activity (CDA) and tactile CDA (tCDA) were measured over visual and somatosensory cortex, respectively. CDA and tCDA amplitudes were selectively affected by WM load in the corresponding (tactile or visual) modality. The CDA parametrically increased when visual load increased from one to two and to three items. The tCDA was enhanced when tactile load increased from one to two items and showed no further enhancement for three tactile items. Critically, these load effects were strictly modality-specific, as substantiated by Bayesian statistics. Increasing tactile load did not affect the visual CDA, and increasing visual load did not modulate the tCDA. Task performance at memory test was also unaffected by WM load in the other (untested) modality. This was confirmed in a second behavioral experiment where tactile and visual loads were either two or four items, unimodal baseline conditions were included, and participants performed a color change detection task in the visual modality. These results show that WM capacity is not limited by a domain-general mechanism that operates across sensory modalities. They suggest instead that WM storage is mediated by distributed modality-specific control mechanisms that are activated independently and in parallel during multisensory WM.

The limits of human visual short-term memory (VSTM) have been well documented, and recent neuroscientific studies suggest that VSTM performance is associated with activity in the posterior parietal cortex. Here we show that artificially elevating parietal activity via positively charged electric current through the skull can rapidly and effortlessly improve people's VSTM performance. This artificial improvement, however, comes with an interesting twist: it interacts with people's natural VSTM capability such that low performers who tend to remember less information benefitted from the stimulation, whereas high performers did not. This behavioral dichotomy is explained by event-related potentials around the parietal regions: low performers showed increased waveforms in N2pc and contralateral delay activity (CDA), which implies improvement in attention deployment and memory access in the current paradigm, respectively. Interestingly, these components are found during the presentation of the test array instead of the retention interval, from the parietal sites ipsilateral to the target location, thus suggesting that transcranial direct current stimulation (tDCS) was mainly improving one's ability to suppress no-change distractors located on the irrelevant side of the display during the comparison stage. The high performers, however, did not benefit from tDCS as they showed equally large waveforms in N2pc and CDA, or SPCN (sustained parietal contralateral negativity), before and after the stimulation such that electrical stimulation could not help any further, which also accurately accounts for our behavioral observations. Together, these results suggest that there is indeed a fixed upper limit in VSTM, but the low performers can benefit from neurostimulation to reach that maximum via enhanced comparison processes, and such behavioral improvement can be directly quantified and visualized by the magnitude of its associated electrophysiological waveforms.

Previous research has demonstrated greater visual working memory (VWM) performance for real-world objects compared with simple features. Greater amplitudes of the contralateral delay activity (CDA)-a sustained event-related potential measured during the delay period of a VWM task-havealso been noted for meaningful stimuli, despite being thought of as a neural marker of a fixed working memory capacity. The current study aimed to elucidate the factors underlying improved memory performance for real-world objects by isolating the relative contributions of perceptual complexity (i.e., number of visual features) and conceptual meaning (i.e., availability of semantic, meaningful features). Participants (N = 22) performed a lateralized VWM task to test their memory of intact real-world objects, scrambled real-world objects and colours. The CDA was measured during both encoding and WM retention intervals (600-1000 ms and 1300-1700 ms poststimulus onset, respectively), and behavioural performance was estimated by using d' (memory strength in a two-alternative forced choice task). Behavioural results revealed significantly better performance within-subjects for real-world objects relative to scrambled objects and colours, with no difference between colours and scrambled objects. The amplitude of the CDA wasalso largest for intact real-world objects, with no difference in magnitude for scrambled objects and colours, during working memory maintenance. However, during memory encoding, both the colours and intact real-world objects had significantly greater amplitudes than scrambled objects and were comparable in magnitude. Overall, findings suggest that conceptual meaning (semantics) supports the memory benefit for real-world objects.

Visual short-term memory (VSTM) provides an on-line mental space for incoming sensory information to be temporally maintained to carry out complex behavioral tasks. Despite its essential functions, the capacity at which VSTM could maintain sensory information is limited (i.e., VSTM can hold only about three to four visual items at once). Moreover, the quality of sensory representation (i.e., precision) degrades as more information has to be maintained in VSTM. Correlational evidence suggests that the level and the pattern of neural activity measured in the posterior parietal cortex (PPC) track both VSTM capacity and precision. However, the causal contributions of the PPC to these different VSTM operations are unclear. Here, we tested whether stimulating the PPC with transcranial direct current stimulation (tDCS) could increase VSTM capacity or precision. We found that stimulating the PPC in male and female human participants selectively enhanced VSTM capacity when the number of memory items exceeded capacity limit, without significant effects on VSTM precision. Moreover, this enhancement of VSTM capacity is region specific as stimulating the prefrontal cortex did not change VSTM capacity or precision. Null stimulation effects in the sensory memory condition confirmed that the tDCS-induced enhancement of VSTM capacity was not simply due to changes in sensory or attentional processes. Altogether, these results provide causal evidence suggesting that the PPC has a more dominant role in supporting the storage capacity of VSTM compared with maintaining the quality of sensory representations. Furthermore, tDCS could be used as a promising noninvasive method to enhance this PPC VSTM-related function.SIGNIFICANCE STATEMENT Correlational evidence from neuroimaging and electrophysiology suggests that the posterior parietal cortex (PPC) supports the storage capacity of visual short-term memory (VSTM) and the precision of sensory representations maintained in VSTM. However, the causal contributions of the PPC to these different VSTM functions were unclear. Here, we found that electrical stimulation over the PPC selectively enhanced VSTM capacity without changing VSTM precision. Overall, our findings suggest that the PPC has a dominant and causal role in supporting the storage capacity of VSTM.

Precision requirements do not affect the allocation of visual working memory capacity

Time-resolved neuroimaging of visual short term memory consolidation by post-perceptual attention shifts

Efficient performance in an environment filled with complex objects is often achieved through the temporal maintenance of conjunctions of features from multiple dimensions. The most striking finding in the study of binding in visual short-term memory (VSTM) is equal memory performance for single features and for integrated multi-feature objects, a finding that has been central to several theories of VSTM. Nevertheless, research on binding in VSTM focused almost exclusively on low-level features, and little is known about how items from low- and high-level visual dimensions (e.g., colored manmade objects) are maintained simultaneously in VSTM. The present study tested memory for combinations of low-level features and high-level representations. In agreement with previous findings, Experiments 1 and 2 showed decrements in memory performance when non-integrated low- and high-level stimuli were maintained simultaneously compared to maintaining each dimension in isolation. However, contrary to previous findings the results of Experiments 3 and 4 showed decrements in memory performance even when integrated objects of low- and high-level stimuli were maintained in memory, compared to maintaining single-dimension objects. Overall, the results demonstrate that low- and high-level visual dimensions compete for the same limited memory capacity, and offer a more comprehensive view of VSTM.

The binding of object identity (color) and location in visual short-term memory (VSTM) was examined in 6.5- to 12.5-month-old infants (N= 144). Although we previously found that by age 6.5 months, infants can represent both color and location in VSTM, in the present study we observed that 6.5-month-old infants could not remember trivially simple color-location combinations across a 300-ms delay. However, 7.5-month-old infants could bind color and location as effectively as 12.5-month-old infants. Control conditions confirmed that the failure of 6.5-month-old infants was not a result of perceptual or attentional limitations. This rapid development of VSTM binding between 6.5 and 7.5 months occurs during a period of rapid increase in VSTM storage capacity and just after a period of dramatic neuroanatomical changes in parietal cortex. Thus, the ability to bind features and the ability to store multiple objects may both depend on a process that is mediated by posterior parietal cortex and is perhaps related to focused attention.

Efficient functioning of visual working memory (VWM) relies on the ability to filter out irrelevant information. Recent studies suggest that negative emotional states impair this filtering process; however, the underlying mechanisms-specifically whether this impairment stems from deficits in target selection or distractor suppression-remain insufficiently understood. The present study employed ERPs in combination with a lateralized change detection task to systematically examine how negative emotional state affects information filtering in VWM and its neural underpinnings. Behavioral results confirmed that negative emotional states marginally significantly reduced VWM accuracy. ERP analyses further revealed that whereas target processing (target-elicited N2pc and contralateral delay activity) remained intact across conditions, distractor processing was significantly altered by negative emotional states. Specifically, under negative emotional states, irrelevant information elicited a more negative N2pc and a less positive distractor positivity amplitude, indicating that irrelevant information exerted stronger attentional capture and weaker active suppression during the encoding stage. During the maintenance stage, the contralateral delay activity component was observed, suggesting that irrelevant information entered and occupied memory resources. Correlation analyses further demonstrated that stronger subjective negative affect was associated with weaker neural inhibition (distractor positivity), which predicted the extent of distractor storage. In summary, this study indicates that negative emotional state impairs the ability to filter distractors from working memory primarily by disrupting the active suppression of irrelevant information during the encoding stage, rather than by hindering target selection. This failure in inhibitory control allows distractors to intrude into the limited capacity of VWM, thereby degrading overall performance.

How do verbal descriptions affect visual memory over the short and long term? Here we show for the first time that verbal labeling can boost visual memories, but the source of this benefit depends on whether representations are maintained over the short term in visual working memory or over the long term in visual long-term memory. Across three experiments, we contrasted color memory of randomly colored objects when participants labeled (a) the color, (b) the object, or (c) the color-object binding, to memory under an articulatory suppression condition inhibiting labeling. Memory was tested at two time points: after three objects (visual working memory) and at the end of the experiment (visual long-term memory). In Experiment 1, color labeling improved, whereas object labeling impaired, visual working memory in comparison to suppression. Visual long-term memory remained unchanged across conditions. Experiment 2 tested whether this was attributable to poor overall long-term learning by repeating the colored objects over three successive working memory trials. This increased performance over the short and long term, yet labeling did not change learning rate over repetitions or delayed memory performance, showing no long-term memory benefit. In Experiment 3, a labeling benefit was observed when the color-object binding was labeled both over the short and long term. Mixture modeling indicated that color-labeling benefits in visual working memory resulted from an increase of detailed visual memory, whereas long-term memory benefits accrued from categorical representations. Our findings point to dissociations on the role of language in visual working memory and visual long-term memory. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Memory representations can be stored in a passive state in a visual working memory (VWM) task. However, it remains unclear whether the representations stored in the passive state are prone to interference and decay. To explore this issue, we asked participants to successively remember two sets of memory items (M1 and M2) in three test manners: a combined test (both M1 and M2 are probed simultaneously), a backward test (probe M2 first and M1 second), or a forward test (probe M1 first and M2 second). We found that the contralateral delay activity (CDA) amplitude after the onset of M2 only tracked M2 independently of M1 in the two separate tests (Experiments 1-3), and the accuracy of M1 was well above chance. These results implied that the M1 representations had been transferred from the online state into the passive state after the onset of M2. Furthermore, the accuracy of M1 (two representations were transferred from the online state into the passive state and retrieved later) in the backward test was worse than M2 (2 representations in the online state throughout) in the backward test (Experiments 1-2), but was comparable to M1 (two representations were transferred from the online state into the passive state and retrieved first) in the forward test (Experiment 2). These results demonstrated that the memory representations were impaired during state switching. Importantly, once the representations had been stored in the passive state, they were robust with little memory loss during latent retention. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Visual working memory is the ability to hold visual information temporarily in mind. A key feature of working memory is its starkly limited capacity, such that only a few simple items can be remembered at once. Prior work has shown that this capacity limit cannot be circumvented by providing additional encoding time, whether providing just 200 ms or up to 1300 ms, capacity is still limited to only three to four items. In contrast, Brady et al. (2016) hypothesized that real-world objects, but not simple items used in prior research, benefit from additional encoding time and are not subject to traditional capacity limits. They supported this hypothesis with results from both behavior and the contralateral delay activity (CDA), an EEG marker of working memory storage, and concluded that familiar, complex stimuli are necessary to observe encoding time effects. Here, we conducted three replications of Brady et al.’s key manipulation with a larger number of human participants and more trials per condition. We failed to replicate their primary behavioral result (objects benefit more than colors from additional encoding time) and failed to observe an object-specific increase in the CDA. Instead, we found that encoding time benefitted both simple color items and real-world objects, in contrast to both the findings by Brady et al., and some prior work on this topic. Overall, we observed no support for the hypothesis that real-world objects have a different capacity than colored squares. We discuss the implications of our findings for theories of visual working memory (VWM).