Eye and mouth movements extraction for driver cognitive distraction detection

Driver distraction has become an important and growing safety concern as the use of in-vehicle information systems (IVISs), such as cell phones and navigation systems, continues to increase. One approach to allowing people to benefit from IVISs without compromising safety is to create adaptive IVISs that adjust their functions according to driver and roadway state. A critical element of adaptive IVISs involves monitoring driver distraction in real time; with such a monitoring function it is possible to mitigate that distraction. This study applied Bayesian networks (BNs), a data mining method, to develop a real-time approach to detecting cognitive distraction using drivers’ eye movements and driving performance. Data were collected in a simulator experiment involving 10 participants who interacted with an IVIS system while driving. BN models were trained and tested to investigate the influence of three model characteristics on distraction detection: time history of driver behavior, inclusion of hidden nodes in the model structure, and how data are summarized and the length of training sequences. Results showed that BNs could identify driver distraction reliably with an average accuracy of 80.1%. Dynamic BNs (DBNs) that consider time dependencies of driver behavior produced more sensitive models than static BNs (SBNs). Longer training sequences improved DBN model performance. Blink frequency and fixation measures were particularly indicative of distraction. These results demonstrate that BNs, especially DBNs, can detect driver cognitive distraction by extracting information from complex behavioral data. Potential applications include the design of adaptive in-vehicle systems and the evaluation of driver distraction.

Concurrence of mouthing movement and rapid eye movement/non-rapid eye movement phases with advance in gestation of the human fetus

Cognitive distraction can impair drivers’ situation awareness and control performance in driving. An on-road study was conducted to examine the efficacy in the detection of driver cognitive distraction based on the driver monitoring system developed by Seeing Machines. Participants completed a 25-km test drive on the local public roads whilst engaging in a series of secondary tasks that were designed to trigger different types of cognitive distraction, such as conversation, comprehension, N-back, and route-planning tasks. The findings showed that percent road center (PRC), one of the promising gaze metrics, increased significantly with cognitive distraction when compared to baseline, but failed to distinguish between different forms of cognitive distraction Moreover, PRC’s sensitivity to cognitive distraction was found to be affected by the chosen radius of road center area. These findings of driver cognitive distraction measurement provide data-driven suggestions for the development of real-time driver monitoring systems in the wild.

Patients with Parkinson's disease (PD patients) have been shown to exhibit abnormally low levels of synergy in their posture control. The goal of this study was to determine how synergic interactions between vision and posture are affected in PD patients. These synergic interactions were expected to be impaired because PD affects the basal ganglia, which are involved in the modulation of both types of movement. Twenty patients (mean age: 60) on levodopa and 20 age-matched-controls (mean age: 61) performed a precise visual task (searching for targets in an image) and an unprecise control task (randomly looking at an image) in which images were projected onto a large panoramic display. Lower back, upper back, head and eye movements were recorded simultaneously. To test behavioural synergies, Pearson correlations between eye and postural movements were analysed. The relationships between eye movements and upper and lower back movements were impaired in the patients. The age-matched controls did not show any significant correlations between eye and postural movements. Overall, our results showed that the PD patients failed to adjust and control their postural stability for success in the visual task. The impaired synergy between eye and postural movements was not related to clinical variables-probably because our patients had early-stage PD. Our results showed that impairments in synergy can occur very early in PD. Hence, the analysis of this synergy might provide a better understanding of postural instability, visual task performance in the upright stance, and perhaps the risk of falls in PD patients.

Detection of driver cognitive distraction is critical for active safety systems of road vehicles. Compared with visual distraction, cognitive distraction is more challenging for detection due to the lack of apparent exterior features. This paper presents a novel real-time detection algorithm for driver cognitive distraction by using support vector machine (SVM). Data are collected from 26 subjects, driving in typical urban and highway scenarios in a simulator. The chosen urban scenario is the stop-controlled intersection and the highway scenario is the speed-limited highway. Driver cognitive distraction while driving is induced by clock tasks which compete with the main driving tasks for visuospatial short working memory. For each subject, distracted driving instances and the equal number of non-distracted driving instances were collected (24 for urban scenario and 20 for highway scenario in total). Features concerning both driving performance and eye movement are used for training and validation. The proposed algorithm have correct rate of 93.0% and 98.5% for highway and urban scenarios respectively. Results also show that driver distraction can be recognized 6.5 s to 9.0 s after its happening, indicating good performance of the detection algorithm.

Eating and drinking is an essential part of every-day life. And yet, there are many people in the world today who rely on others to feed them. In this work, we present a prototype robot-assisted self-feeding system for individuals with movement disorders. The system is capable of perceiving, localizing, grasping, and delivering non-compliant food items to an individual. We trained an object recognition network to detect specific food items, and we compute the grasp pose for each item. Human input is obtained through an interface consisting of an eye-tracker and a display screen. The human selects options on the monitor with their eye and head movements and triggers responses with mouth movements. We performed a pilot study with four able-bodied participants and one participant with a spinal cord injury (SCI) to evaluate the performance of our prototype system. Participants selected food items with their eye movements, which were then delivered by the robot. We observed an average overall feeding success rate of 89.1% and an average overall task time of $31.4 \pm 2.4$ seconds per food item. The SCI participant gave scores of 90.0 and 8.3 on the System Usability Scale and NASA Task Load Index, respectively. We also conducted a custom, post-study interview to gather participant feedback to drive future design decisions. The quantitative results and qualitative user feedback demonstrate the feasibility of robot-assisted self-feeding and justify continued research into mealtime-related assistive devices.

Bell's palsy rarely recurs or arises bilaterally. We describe unique oral motor sequelae that manifest with bilateral facial palsy. Two rare patients with alternating bilateral Bell's palsy experienced compromised mouth movement, affecting eating, speaking, and air exchange, and reduced quality of life. Rehabilitation programs. Electromyography and electroneuronography. On electromyography study, little activity of the orbicularis oris muscle was seen in any mouth or eye movement in Case 1. In Case 2, the electromyography activity of the depressor anguli oris and depressor labii inferioris muscles was stronger than that of the orbicularis oris muscle. The 2 cases demonstrated different patterns of recovery after the same subsequent treatment. Different patterns of reinnervation occur in bilateral Bell's palsy. Mouth movement disturbances after bilateral Bell's palsy are most disabling when incomplete reinnervation (as suggested by electroneuronography) of the orbicularis oris muscle occurs. Misdirection of regenerating nerve fibers produces lesser levels of oral motor impairment.

Driver states adaptive drive supporting system is highly expected in use for the reduction of the number of traffic accidents. This study aimed at creating a constituent technology for detecting driver's cognitive distraction, which may be one of major factors of driver's psychosomatic states just before a traffic accident. We reproduced driver's cognitive distraction by means of imposing cognitive loads such as arithmetic and conversation to a subject on a driving simulator. Besides gaze angle, head rotation angle, and interval between heart R-waves (hereafter, heart rate RRI) from an ECG (electrocardiogram), we added pupil diameter of a subject as recognition features for pattern recognition. We established high accuracy and rapid detection methodology for driver's cognitive distraction by adopting the AdaBoost.

Our perception of an event is often modulated by our past experience and expectations [1Ramachandran V.S. Armel C. Foster C. Stoddard R. Object recognition can drive motion perception.Nature. 1998; 395: 852-853Crossref PubMed Scopus (35) Google Scholar, 2O'Regan J.K. Noe A. A sensorimotor account of vision and visual consciousness.Behav. Brain Sci. 2001; 24: 939-973Crossref PubMed Scopus (1863) Google Scholar]. Here we used a magic trick [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar] to demonstrate how magicians can distort our subjective perception and we investigate the mechanisms behind this deception. We found that when a magician performed an illusion in which a ball was seen to vanish in the air, 63% of observers perceived the ball leave his hand, move upwards, and disappear even though the ball did not leave the magician's hand. Moreover, observers' illusory perception of the ball was determined by cues that indicated the ball's location, namely the magician's head direction, rather than the percept itself. Furthermore, eye movement records revealed participants' strategic use of social cues prior to looking at the ball. Surprisingly, however, when the ball was not physically present, observers did not look at the area where they claimed to have seen the ball vanish, suggesting that the oculomotor system was not fooled by the illusion. These results show that although people's subjective percept is determined by expectations, the oculomotor system is largely driven by accurate bottom-up information, which is consistent with the suggestion that there are separate mechanisms for perception and visuomotor control [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar]. A striking example of magicians' deception is the vanishing ball illusion, in which a ball apparently disappears in mid air. This illusion is created by pretending to throw a ball up in the air, when in fact it remains secretly palmed in the magician's hand [3Triplett N. The psychology of conjuring deceptions.Am. J. Psychol. 1900; 6: 439-510Crossref Google Scholar]. If done convincingly, the observer should perceive the ball as moving up in the air even though it is no longer physically present. Similarly, when observers view a moving object that suddenly disappears, the final position of the object is usually perceived as being further along the path of motion than its actual final position [5Freyd J.J. Finke R.A. Representational Momentum.J. Exp. Psychol. Learn. Mem. Cogn. 1984; 10: 126-132Crossref Scopus (464) Google Scholar], a phenomenon known as representational momentum [6Assad J.A. Maunsell J.H.R. Neuronal correlates of inferred motion in primate posterior parietal carter.Nature. 1995; 373: 518-521Crossref PubMed Scopus (181) Google Scholar]. Much of the magician's deception relies on social cuing, in terms of his head and gaze direction, and we therefore predicted that the effectiveness of the vanishing ball illusion would be influenced by the magician's social cuing. Two different versions of the trick were created that addressed the extent to which the magician's social cues are responsible for the illusion. In the social cues pro-illusion condition, on the final 'fake' throw, the magician's eyes and head followed an imaginary ball moving upwards (Figure 1 and Supplemental data available on-line with this issue). In the social cues anti-illusion condition, the magic trick was identical except that on the final 'fake' throw, the magician looked at the hand concealing the ball, rather than following the imaginary ball (see Figure 2F). We also measured people's eye movements while they watched the video of the trick, to gain an online measure of the information obtained by the visual system. Immediately after having seen the magic trick, participants were questioned as to whether they perceived the 'illusory ball' moving towards the top of the screen on the final throw.Figure 2Participants' fixation points for the social cues pro-illusion (top row) and the social cues anti-illusion (bottom row) conditions.Show full captionThe dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002).View Large Image Figure ViewerDownload Hi-res image Download (PPT) The dots represent the fixation points for the participants who experienced the illusion and the 'X' marks the fixation points for participants who did not experience the illusion. (A,D) At the point when the ball reached its zenith on the first throw. (B,E) at the point when the ball reached its zenith on the second throw. (C,F) At the point when the ball disappeared. For each participant the height of the terminal eye position on the monitor was calculated (distance between the fixation point and the bottom of the monitor, monitor height = 25°); each panel shows the means (M) and standard deviations (SD) for participants who experienced the illusion and those who did not. On the final throw (C,F) there was no significant difference between the final eye position of the participants who perceived the illusion compared to those who did not perceive it (t(36) < 1), which shows that participants' eye movements were not correlated with the perception of the illusion. On the final throw there was no significant difference in the terminal eye positions between the anti- and the pro-illusion condition (t(36) < 1), illustrating that participants' eye movements were not driven by the social cues. 19 participants experienced the illusion. For these participants the terminal eye position on the final throw was significantly lower than on the first (t(18) = 3.23, p = 0.005) and the second throw (t(18) = 3.57, p = 0.002). Participants who perceived the illusory ball had a vivid recollection of seeing the ball leave the screen at the top, and typically claimed that the illusion was created by someone catching it beyond the top of the screen. 68% of the participants in the social cues pro-illusion condition experienced the illusion, significantly more than in the social cues anti-illusion condition (32%, χ2 = 5.16, p = 0.025), thus demonstrating that the illusion was mediated by the social cuing. Most of the participants claimed that they spent their entire time looking at the ball. However, these reports deviated strongly from where they were actually looking. Figure 1 shows a timeline indicating where participants were looking during the magic trick. We found a very high consistency in the eye movement patterns between the participants, suggesting that people employed similar eye movement strategies. On throws where the ball was visible, participants generally looked at the ball once it reached the apex. But rather than merely tracking the ball, most of the participants glanced at the magician's face before looking at the ball, indicating that the visual system uses information about where the magician is looking as a way of predicting the location of the ball. Our results show that an observer's percept was driven by the magician's social cuing, and that participants utilized the magician's social cues. However, there was a surprising difference in the susceptibility of these two systems to the illusion. On the final illusory throw, most participants claimed to have seen the ball at the top of the screen. The oculomotor system on the other hand was not fooled by the illusion. Figure 2 shows that participants only looked at the top of the screen when the ball was physically present. These results illustrate a remarkable dissociation between what participants claimed to have seen and the way in which their eyes behaved. Whilst their percept was predominantly based on expectations, their eyes were largely driven by the actual visual input. Our results offer a striking example of how the dissociation between vision for perception and for action, well documented in neurological patients [4Goodale M.A. Milner A.D. Separate visual pathways for perception and action.Trends Neurosci. 1992; 15: 20-25Abstract Full Text PDF PubMed Scopus (3924) Google Scholar, 7Goodale M.A. Haffenden A. Frames of reference for perception and action in the human visual system.Neurosci. Biobehav. Rev. 1998; 22: 161-172Crossref PubMed Scopus (110) Google Scholar, 8Milner A.D. Goodale M.A. The Visual Brain in Action. Oxford University Press, Oxford1995Google Scholar] and neurologically intact subjects [9Aglioti S. Desouza J.F.X. Goodale M.A. Size-contrast illusions deceive the eye but not the hand.Curr. Biol. 1995; 5: 679-685Abstract Full Text Full Text PDF PubMed Scopus (721) Google Scholar], also translates to more real world situations. We would like to thank the reviewers and Cristiana Cavina-Pratesi for their valuable comments on the manuscript, and Tim Smith for his help with a control experiment. Download .pdf (.1 MB) Help with pdf files Document S1. Supplemental Experimental Procedures Download .mpg (1.09 MB) Help with mpg files Movie 1 Vanishing ball trick pro-illusion condition Download .mpg (1.12 MB) Help with mpg files Movie 2 Vanishing ball trick anti-illusion condition

Objective. Cultures have essential influences on emotions. However, most studies on cultural influences on emotions are in the areas of psychology and neuroscience, while the existing affective models are mostly built with data from the same culture. In this paper, we identify the similarities and differences among Chinese, German, and French individuals in emotion recognition with electroencephalogram (EEG) and eye movements from an affective computing perspective. Approach. Three experimental settings were designed: intraculture subject dependent, intraculture subject independent, and cross-culture subject independent. EEG and eye movements are acquired simultaneously from Chinese, German, and French subjects while watching positive, neutral, and negative movie clips. The affective models for Chinese, German, and French subjects are constructed by using machine learning algorithms. A systematic analysis is performed from four aspects: affective model performance, neural patterns, complementary information from different modalities, and cross-cultural emotion recognition. Main results. From emotion recognition accuracies, we find that EEG and eye movements can adapt to Chinese, German, and French cultural diversities and that a cultural in-group advantage phenomenon does exist in emotion recognition with EEG. From the topomaps of EEG, we find that the γ and β bands exhibit decreasing activities for Chinese, while for German and French, θ and α bands exhibit increasing activities. From confusion matrices and attentional weights, we find that EEG and eye movements have complementary characteristics. From a cross-cultural emotion recognition perspective, we observe that German and French people share more similarities in topographical patterns and attentional weight distributions than Chinese people while the data from Chinese are a good fit for test data but not suitable for training data for the other two cultures. Significance. Our experimental results provide concrete evidence of the in-group advantage phenomenon, cultural influences on emotion recognition, and different neural patterns among Chinese, German, and French individuals.

Correlation between regular mouthing movements and heart rate patterns during non-rapid eye movement periods in normal human fetuses between 32 and 40 weeks of gestation

Event Abstract Back to Event Influences of social cognition on visuospatial attention: A neurophysiological investigation Grace Wei1*, Jacqueline A. Rushby1, Christopher Sufani1 and Frances De Blasio1 1 University of New South Wales, Australia Aims: The reflexive orienting response triggered by non-predictive gaze cues is thought to be driven by a dedicated social neural network responsible for directing attention towards socially salient information. Atypical processing of eye gaze using concomitant perceptual features such as motion transients may underlie reflexive attentional orienting in groups with impairments in social cognition. Thus, reflecting the employment of qualitatively different cognitive mechanisms used to access social information. The present study examined the neurophysiological correlates of gaze processing, considering individual variability in social cognition and the relative salience of social and perceptual cues on visuospatial attention. Method: Electroencephalographic (EEG) activity and behavioural response time (RT) was recorded from 26 healthy adult participants during a spatial cueing task. Participants responded to presentations of a lateralised target on valid (congruently cued) or invalid (incongruently cued) trials across 3 conditions; social gaze, mouth motion and eye motion. Scores on the AQ, LSAS and TAS-20 were used to assess social cognitive functioning. ERP components were obtained with separate PCAs for each cue type and condition. Results: Behavioural analyses revealed cueing effects for the social gaze and mouth motion conditions. Responses did not differ significantly in the eye motion condition. In the social gaze condition, higher scores on the TAS-20 correlated with a reduction of the N2b frontal enhancement to invalid trials (r=.439, p<.05). In the mouth motion condition, higher LSAS and TAS scores were correlated with a greater overall N2b to invalid trials (r=.403, p<.05 and r=.429, p<.05 respectively). Conclusions: Findings suggest the social gaze cue is less salient for those with greater alexithymic traits. An attentional bias towards perceptual changes (mouth motion) parallel impairments in social cognitive functioning, and thus may reflect reduced sensitivity to social stimuli. Such paradigms may benefit investigations in clinical populations. Keywords: Event Related Potentials (ERPs), spatial attention, social cognition, Gaze Processing, Autism Spectrum Disorders, social anxiety, alexithymia, Principal Components Analysis (PCA), N2b Conference: ASP2017: 27th Annual Meeting for the Australasian Society for Psychophysiology, Parramatta, Australia, 29 Nov - 1 Dec, 2017. Presentation Type: Oral Presentation Topic: Abstract (Student Award) Citation: Wei G, Rushby JA, Sufani C and De Blasio F (2019). Influences of social cognition on visuospatial attention: A neurophysiological investigation. Conference Abstract: ASP2017: 27th Annual Meeting for the Australasian Society for Psychophysiology. doi: 10.3389/conf.fnhum.2017.224.00008 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Oct 2017; Published Online: 25 Jan 2019. * Correspondence: Ms. Grace Wei, University of New South Wales, Kensington, Australia, grace.wei@sydney.edu.au Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Grace Wei Jacqueline A Rushby Christopher Sufani Frances De Blasio Google Grace Wei Jacqueline A Rushby Christopher Sufani Frances De Blasio Google Scholar Grace Wei Jacqueline A Rushby Christopher Sufani Frances De Blasio PubMed Grace Wei Jacqueline A Rushby Christopher Sufani Frances De Blasio Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

<title>Abstract</title> Meta Quest Pro-based eye and facial tracking technology plays an essential role in studies related to visual attention, cognitive processes, user interactions, and human behavior analysis. However, differences in eye anatomy, such as outer canthal distance (OCD), interpupillary distance (IPD), and intercanthal distance (ICD), may impact the accuracy &amp; reliability of eye movement data. This study aims to evaluate the impact of inter-eye distance variations on eye-movement-based feature extraction in VR environments using the head-mounted device Meta Quest Pro. The data collection process involved eye measurements (OCD, IPD, and ICD), an eye calibration test before the experiment, and three experimental tasks: fixed gaze task, regular eye movement task, and irregular eye movement task. The Meta Quest Pro recorded participants’ eye movements while they performed VR-based tasks, followed by the extraction of eye-related features from the device. The dataset underwent Shapiro–Wilk, and D’Agostino–Pearson normality tests, followed by statistical analysis (Spearman Correlation, Kruskal–Wallis, and Mann–Whitney U test) to assess differences in eye-movement correlations across wide, narrow, and average eye distance groups. The results of statistical tests did not indicate a significant influence of inter-eye distance on eye movement tracking accuracy. This study is important for research, including learning disabilities research, child and adult behavior analysis, and human–computer interaction, ensuring that eye data remains accurate and reliable across diverse users.

Psychological literature reflects little information about neurolinguistic programming. There are even fewer reports of research o n basic tenets. This research investigated one facet, namely, the eye-movement hypothesis of Bandler and Grinder (1975, 1979) who posited that eye movements should indicate the representational system in use by an individual, e.g., visual, auditory, or kinesthetic. Eye movements have been descr~bed elsewhere (Thomason, Arbuckle, & Cady, 1980; Dorn, Atwater, Jereb, & Russell, 1983) . In t h ~ s research 22 undergraduates, whose mean age was 21.6 yr., were asked to respond to a 30-item questionnaire of 1 0 each, visual, auditory, and kinesthetic questions. Subjects' eye movements were videotaped through a one-way mirror and later rated by five naive undergraduates. Values of chi squared, computed individually for each subject, were not significant for eye movements. One analysis indicated that 50% of the rime no movements were discernable and rhat frequencies of eye movements to questions in other modalities were 2 2 % visual, 2 2 9 6 auditory, and 696 kinesthetic. Visual responses were as frequent as auditory responses which is conrrary to the current thinking that visual responses predominate. These results are not consistent with previous research (Thomason, et al., 1980) , except rhat neither project supported the eye-movement hypothesis of neurolinguistic programming. It is questionable whether the chi squared, reported by Thomason, et al. ( 1980) , is appropriate given the assumption of independence of data. Although the developers of neurolinguistic programming reject empirical research (Brady & Biedermann, 1985) , it appears that other controlled studies are warranted.

Oculomotor behavior and perceptual strategies in complex tasks