ReMOT: A model-agnostic refinement for multiple object tracking

Multiple Object Tracking(MOT) developed by Pylyshyn and Storm(1988) has been widely used in the study of capacity-limited and object-based attention. Many researchers interested in finding out whether or not the tracking processes in MOT occupy attentional resource and what type of resource is used. The typical paradigm used in this line of research is dual-task paradigm. Participants were asked to perform a visual or auditory task and the MOT task simultaneously. MOT performance was interfered by both the visual and the auditory task. However, the interference of visual task and auditory task with MOT occurred at different levels. The previous studies demonstrated that the MOT task and visual task occupy visual attention resources. Although the MOT task and auditory task don't occupy visual attention resources, they share more central attention resources(such as executive function). Multiple Identity Tracking(MIT) is a variant of MOT, in which each object carries a unique identity. The previous studies also included both visual or auditory task and MIT task, but those studies have not examined how the visual/auditory task affects the MIT task when the two tasks shared the same properties. The current study included 3 experiments and aimed to investigate the influence of a visual or auditory task on either MOT or MIT task. The first two experiments manipulated participants' eye movement to compare the different effects of visual task and auditory task on MOT performance. The result of experiment 1A showed that the auditory task interfered more with MOT than did the visual task when eyes were fixated at the center of the screen. However, the auditory task yielded less interference with MOT when there was no eye movement control in experiment 1B. The results indicated that the tracking processes in MOT not only occupy visual attention resources, but also occupy central attention resources(such as executive function). The experiment 2 applied visual and auditory digit judgment task and MIT task. When the object identity(marked by a digit) was identical to the number in the digit judgment, the MIT performance was facilitated by both the visual and auditory task. It is possible that the process of digit activated the target identities(the same digits) that was stored in visual working memory during tracking.

Walking while simultaneously engaged in another activity becomes more difficult as one grows older. Here, we address the issue of changes in dual-task behavior at different stages of life, particularly in the latter stages. We developed a dual task that combined walking along an 8-m walkway with a multiple object tracking (MOT) task of increasing difficulty. This secondary cognitive task imitates visuospatial daily activities and provides reliable quantitative measurements. Our dual-task paradigm was tested on 27 young adults (23.85 ± 2.09 years old) and two groups of older adults (18 young-old and 18 old-old adults, aged 63.89 ± 3.32 and 80.83 ± 3.84 years, respectively). Significant decrease in tracking performance with increasing complexity of the MOT task was found in all three groups. An age-related decrease in MOT and gait performance was also found. However, young-old adults performed as well as young adults under low attentional load conditions (in the MOT task and simple walking), whereas their performance was as impaired as those of old-old adults under high attentional load conditions (in the MOT task and walking under dual-task condition). These different profiles between the two groups of older participants could be explained in terms of compensation strategies and risk of falling.

Simultaneous tracking of the position of the ball and player locations and activities places high demands on visual attention in team ball sport athletes. Owing to their extensive sports training, these athletes may be expected to exhibit visual attention skills superior to non-athletes; however, the results of studies examining this are inconsistent. Thus, the first aim of this study was to assess the impact of participating in a team ball sport on visual attention. There is limited empirical evidence indicating a sex difference in visual attention, and few studies have reported on visual attention in male and female athletes. Thus, the second aim of this study was to determine whether team ball sport experience affected any sex differences in visual attention. In total, 44 highly skilled basketball players—22 men (mean age: 21.86 ± 2.15 years) with a mean (SD) of 8.46 (2.92) years training experience and 22 women (mean age: 21.32 ± 1.58 years) with a mean (SD) of 8.22 (2.44) years of training experience—and 44 non-athletic undergraduate college students—22 men (mean age: 21.62 ± 1.88 years) and 22 women (mean age: 21.55 ± 1.72 years)—were recruited and completed this study. Visual attention was measured by using the multiple object tracking (MOT) task. Skilled basketball players showed superior tracking accuracy to non-athletic college students on the MOT task. A significant sex difference was found only among the non-athletic college students, with better tracking accuracy for men than for women on the MOT task. By contrast, no significant sex difference was observed among the skilled basketball players for tracking accuracy on the MOT task. These findings indicated that team ball sport training may enhance visual attention as assessed by tracking accuracy. Given that the male and female basketball players in this study had similar training experience and game performance demands, long-term team ball sport experience appeared to minimize the sex difference in visual attention found among non-athletic students.

BackgroundThis study aimed to investigate whether performance in a multiple object tracking (MOT) task could be improved incrementally with sports expertise, and whether differences between experienced and less experienced athletes, or non-athletes, were modulated by load.MethodsWe asked 22 elite and 20 intermediate basketball players, and 23 non-athletes, to perform an MOT task under three attentional load conditions (two, three, and four targets). Accuracies were analyzed to examine whether different levels of sports expertise influence MOT task performance.ResultsThe elite athletes displayed better tracking performance compared with the intermediate or non-athletes when tracking three or four targets. However, no significant difference was found between the intermediate athletes and the non-athletes. Further, no differences were observed among the three groups when tracking two targets.DiscussionThe results suggest that the effects of expertise in team ball sports could transfer to a non-sports-specific attention task. These transfer effects to general cognitive functions occur only in elite athletes with extensive training under higher attentional load.

Previous research has revealed the uniqueness-facilitation effect in the multiple object tracking (MOT) task: simple distinct identities and surface features of moving targets could facilitate attentional tracking. By adapting compound stimuli, the present study investigated whether the global or local properties played the main role in the uniqueness-facilitation effect in the MOT task. The uniqueness of local properties, of global properties or of both local and global properties were considered. Observers’ tracking performance in alternative conditions were compared with that in the homogeneous condition wherein all stimuli have identical local and global properties. Results from two experiments suggest that the global properties played the key role in facilitating tracking. The distinctiveness of local properties can also facilitate tracking with global properties being homogeneous. However, when the stimuli’s global properties are distinct from each other—whether the local properties being unique or not—observers’ tracking performance can achieve the same level as that in the unitary-uniqueness condition wherein the moving objects were distinct unitary letters. These results revealed a global superiority effect in the MOT task. Finally, the facilitation effects of the global and local properties might depend on the stimulus sparsity. When the compound stimuli had fewer local elements, the uniqueness facilitation effect on tracking decreased.

This study aimed to assess the impact of manipulating postural stability during the execution of the multiple objects tracking (MOT) task on gaze behaviour. Twenty-eight males performed the MOT task under three levels of postural stability while eye movements were recorded. Postural stability had a significant impact on MOT scores and gaze behaviour. Participants exhibited worse MOT performance (p < 0.001, η² = 0.20) in less stable conditions, showing a reduced fixation duration (p = 0.020, η² = 0.09) and increased saccade frequency (p = 0.013, η² = 0.10). As stability conditions decreased, participants shifted their gaze strategies by an average of 12.34% of the time distribution from maintaining fixation on a central location to adopting the target-looking strategy. Additionally, the decline in MOT scores with decreasing stability was correlated with a reduction in fixation duration (r = 0.409, p = 0.031). The findings evidence that gaze behaviour during MOT is dependent on postural stability and MOT performance is influenced by the chosen gaze strategy. Key points Reduced postural stability significantly impairs the ability to track multiple objects. Changes in postural stability during a multiple object tracking task affect gaze behaviour by alterations in fixation duration and saccadic frequency. Decrease in multiple object tracking task scores is associated with alterations in gaze behaviour, specifically a reduction in fixation duration. Participants shifted their gaze strategies from maintaining fixation on a central location to adopting a saccadic target-looking strategy as stability conditions decreased.

Speech is often degraded by environmental noise or hearing impairment. People can compensate for degradation, but this requires cognitive effort. Previous research has identified frontotemporal networks involved in effortful perception, but materials in these works were also less intelligible, and so it is not clear whether activity reflected effort or intelligibility differences. We used functional magnetic resonance imaging to assess the degree to which spoken sentences were processed under distraction and whether this depended on speech quality even when intelligibility of degraded speech was matched to that of clear speech (close to 100%). On each trial, male and female human participants either attended to a sentence or to a concurrent multiple object tracking (MOT) task that imposed parametric cognitive load. Activity in bilateral anterior insula reflected task demands; during the MOT task, activity increased as cognitive load increased, and during speech listening, activity increased as speech became more degraded. In marked contrast, activity in bilateral anterior temporal cortex was speech selective and gated by attention when speech was degraded. In this region, performance of the MOT task with a trivial load blocked processing of degraded speech, whereas processing of clear speech was unaffected. As load increased, responses to clear speech in these areas declined, consistent with reduced capacity to process it. This result dissociates cognitive control from speech processing; substantially less cognitive control is required to process clear speech than is required to understand even very mildly degraded, 100% intelligible speech. Perceptual and control systems clearly interact dynamically during real-world speech comprehension.SIGNIFICANCE STATEMENT Speech is often perfectly intelligible even when degraded, for example, by background sound, phone transmission, or hearing loss. How does degradation alter cognitive demands? Here, we use fMRI to demonstrate a novel and critical role for cognitive control in the processing of mildly degraded but perfectly intelligible speech. We compare speech that is matched for intelligibility but differs in putative control demands, dissociating cognitive control from speech processing. We also impose a parametric cognitive load during perception, dissociating processes that depend on tasks from those that depend on available capacity. Our findings distinguish between frontal and temporal contributions to speech perception and reveal a hidden cost to processing mildly degraded speech, underscoring the importance of cognitive control for everyday speech comprehension.

With the increased sophistication of technology, humans have the possibility to offload a variety of tasks to algorithms. Here, we investigated whether the extent to which people are willing to offload an attentionally demanding task to an algorithm is modulated by the availability of a bonus task and by the knowledge about the algorithm’s capacity. Participants performed a multiple object tracking (MOT) task which required them to visually track targets on a screen. Participants could offload an unlimited number of targets to a “computer partner”. If participants decided to offload the entire task to the computer, they could instead perform a bonus task which resulted in additional financial gain—however, this gain was conditional on a high performance accuracy in the MOT task. Thus, participants should only offload the entire task if they trusted the computer to perform accurately. We found that participants were significantly more willing to completely offload the task if they were informed beforehand that the computer’s accuracy was flawless (Experiment 1 vs. 2). Participants’ offloading behavior was not significantly affected by whether the bonus task was incentivized or not (Experiment 2 vs. 3). These results combined with those from our previous study (Wahn et al. in PLoS ONE 18:e0286102, 2023), which did not include a bonus task but was identical otherwise, show that the human willingness to offload an attentionally demanding task to an algorithm is considerably boosted by the availability of a bonus task—even if not incentivized—and by the knowledge about the algorithm’s capacity.

Despite considerable similarities between multiple object tracking (MOT) and single object tracking (SOT) tasks, modern MOT methods have not benefited from the development of SOT ones to achieve satisfactory performance. The major reason for this situation is that it is inappropriate and inefficient to apply multiple SOT models directly to the MOT task, although advanced SOT methods are of the strong discriminative power and can run at fast speeds.In this paper, we propose a novel and end-to-end trainable MOT architecture that extends CenterNet by adding an SOT branch for tracking objects in parallel with the existing branch for object detection, allowing the MOT task to benefit from the strong discriminative power of SOT methods in an effective and efficient way. Unlike most existing SOT methods which learn to distinguish the target object from its local backgrounds, the added SOT branch trains a separate SOT model per target online to distinguish the target from its surrounding targets, assigning SOT models the novel discrimination. Moreover, similar to the detection branch, the SOT branch treats objects as points, making its online learning efficient even if multiple targets are processed simultaneously. Without tricks, the proposed tracker achieves MOTAs of 0.710 and 0.686, IDF1s of 0.719 and 0.714, on MOT17 and MOT20 benchmarks, respectively, while running at 16 FPS on MOT17.

Introduction The ability to simultaneously track numerous moving objects in the presence of irrelevant stimuli is essential for successfully carrying out a variety of tasks. Sleep loss impairs neurocognitive functioning and, as a result, attentional processing capacity is reduced. The objective of the current study was to determine if performance on the multiple object tracking (MOT) task was adversely impacted by a week of chronic sleep restriction (CSR). Methods Twelve healthy participants (6 males, 6 females) kept a fixed sleep-wake schedule, with a constant waketime, at home for four weeks (actigraphy confirmed compliance). During weeks one and three, participants maintained 9 hours in bed. During weeks two and four, participants were randomly assigned to 5 and 9 hours of sleep. Following weeks two and four, participants completed a 13-hour laboratory visit under dim light (&amp;lt; 15 lux), where they maintained a constant posture and were provided with hourly isocaloric snacks. MOT was presented at approximately 6 and 8 hours after waking. Participants were required to track four, five, or six moving targets in the presence of identical distractors (always 12 total objects). Results Participants slept significantly less when assigned to 5 (M = 4.43 hours, SD = 0.33 hours), compared to 9 hours of sleep (M = 7.42 hours, SD = 0.42 hours; F (1, 22) = 206.89, p = 0.00). The proportion of correct MOT responses was significantly lower following 5 (M = 0.70, SD = 0.15) compared to 9 hours of sleep (M = 0.77, SD = 0.12; F (1, 22) = 10.29, p &amp;lt; .05). Conclusion A week of CSR adversely impacted MOT performance compared to a week of sleep satiation. These findings have implications for individuals, such as air traffic controllers and truck drivers, who must visually track multiple moving objects, often while chronically sleep deprived. Support Supported by the Force Health Protection Program of the Office of Naval Research (SAA2402925-1, Contract Award no. N0001418IP00050).

We previously reported that in the Multiple Object Tracking (MOT) task, which requires tracking several identical targets moving unpredictably among identical nontargets, the nontargets appear to be inhibited, as measured by a probe-dot detection method. The inhibition appears to be local to nontargets and does not extend to the space between objects—dropping off very rapidly away from targets and nontargets. In the present three experiments we show that (1) nontargets that are identical to targets but remain in a fixed location are not inhibited and (2) moving objects that have a different shape from targets are inhibited as much as same-shape nontargets, and (3) nontargets that are on a different depth plane and so are easily filtered out are not inhibited. This is consistent with a task-dependent view of item inhibition wherein nontargets are inhibited if (and only if) they are likely to be mistaken for targets.

The ability to move our gaze to locations of interest facilitates interactions in everyday life. Where do participants direct gaze when multiple locations are of interest simultaneously? We previously demonstrated that, when tracking several moving targets amidst distractors in a multiple object tracking (MOT) task, participants primarily looked at a central point in between the targets (H. M. Fehd & A. E. Seiffert, 2008). This strategy of center-looking is in contrast to a target-looking strategy where participants would saccade from target to target. Here we investigated what factors influence the use of center-looking as well as its effectiveness. By decreasing object speed, we determined that center-looking is not a result of avoiding costly eye movements during tracking. Decreasing object size showed that peripheral visibility is necessary for tracking, but that center-looking continues up to the limits of peripheral visibility. Further analysis revealed that participants often engaged in both target-looking and center-looking by switching gaze from the center to targets and back again. Directly comparing participants' performance when they either did or did not include center-looking along with target-looking revealed that center-looking facilitates tracking performance. These results suggest that there is value in looking at the center that relates directly to the process of tracking multiple objects.

Multiple object tracking (MOT) is generally regarded as a pure attention-consuming task that draws heavily on attention resources. In the present study, we adopted a cross-channel visual-audio dual-task paradigm, i.e., the MOT task combined with the concurrent auditory N-back working memory task, to test whether working memory indeed plays a necessary role in the process of multiple tracking, as well as to further identify the specific types of working memory components involved in this process. Experiments 1a and 1b examined the relationship between the MOT task and nonspatial object working memory (OWM) processing by manipulatingthe tracking load and working memory load, respectively. Results in both experiments indicated that the concurrent nonspatial OWM task did not have a significant effect on the tracking capacity of the MOT task. In contrast, Experiments 2a and 2b examined the relationship between the MOT task and spatial working memory (SWM) processing by a similar approach. Results in both experiments indicated that the concurrent SWM task significantly impaired the tracking capacity of the MOT task, showing a gradual decrease with increasing SWM load. Overall, our study provides empirical evidence that multiple object tracking does involve working memory, primarily related to spatial working memory rather than nonspatial object working memory, which sheds more light on the mechanisms of multiple object tracking.

In the near future, humans will increasingly be required to offload tasks to artificial systems to facilitate daily as well as professional activities. Yet, research has shown that humans are often averse to offloading tasks to algorithms (so-called "algorithmic aversion"). In the present study, we asked whether this aversion is also present when humans act under high cognitive load. Participants performed an attentionally demanding task (a multiple object tracking (MOT) task), which required them to track a subset of moving targets among distractors on a computer screen. Participants first performed the MOT task alone (Solo condition) and were then given the option to offload an unlimited number of targets to a computer partner (Joint condition). We found that participants significantly offloaded some (but not all) targets to the computer partner, thereby improving their individual tracking accuracy (Experiment 1). A similar tendency for offloading was observed when participants were informed beforehand that the computer partner's tracking accuracy was flawless (Experiment 2). The present findings show that humans are willing to (partially) offload task demands to an algorithm to reduce their own cognitive load. We suggest that the cognitive load of a task is an important factor to consider when evaluating human tendencies for offloading cognition onto artificial systems.

Feedback is beneficial for learning. Nevertheless, it remains unclear whether (i) feedback draws attentional resources when integrated and (ii) the benefits of feedback for learning can be demonstrated using an attention-based task. We therefore (i) isolated feedback-specific load from task-specific load via individual differences in attention resource capacity and (ii) examined the effect of trial-by-trial feedback (i.e., present vs. absent) on learning a multiple object-tracking (MOT) paradigm. We chose MOT because it is a robust measure of attention resource capacity. In Study 1 participants tracked one (i.e., lowest attentional load condition) through four target items (i.e., highest load condition) among eight total items. One group (n = 32) received trial-by-trial feedback whereas the other group (n = 32) did not. The absence of feedback resulted in better MOT performance compared with the presence of feedback. Moreover, the difference in MOT capability between groups increased as the task-specific attentional load increased. These findings suggest that feedback integration requires attentional resources. Study 2 examined whether the absence (n = 19) or presence (n = 19) of feedback affects learning on the same MOT task across four testing days. When holding task-specific load constant, improvement in MOT was greater with feedback than without. Although this study is the first to isolate feedback-specific load in attention with MOT, more evidence is needed to demonstrate how the benefits of feedback translate to improvement on an attention-based task. These findings encourage future research to further explore the interaction between feedback, attention and learning.