Affective State Recognition in Livestock-Artificial Intelligence Approaches.

Affective state recognition is a key component of any system equipped with emotional awareness and intelligence. The ability of recognizing emotions allows the machine to better understand the user requirements, and guide its decision and response, thus establishing a more connected relationship with the human. It is usually assumed that emotion recognition is mainly determined by face features from visual data, which impose the dilemma of invading the privacy of the user and capturing their identity, which is unacceptable by many people, especially in public human-machine interaction (HMI) setups. On the other hand, bodily reactions and background context can provide enough emotional clues visually and are less susceptible to contextual influences compared to facial expression. Consequently, this paper investigate the recognition of affective state from visual data captured during a naturalistic conversation with similar perspective to HMI. The faces were masked to conceal the identity of the users. A deep learning recognition model based on a combined Convolutional Neural Network and Long Short-Term Memory (CNN-LSTM) architecture is employed to classify the user’s affective state into two levels of arousal and valence, as well as their quadrant combinations. The experiments were conducted using two different labeling schemes mimicking the self and conversation partner perspectives. The results shows that affective state recognition from masked data using the proposed model can achieve comparable performances (up to 96.82%, 95.91%, and 91.52% for arousal, valence, and quad classes recognition, respectively) in comparison to the use of raw data with facial expressions. This paves the way for privacy aware emotion recognition systems that could be widely accepted by the users.

Genetically optimized Fuzzy C-means data clustering of IoMT-based biomarkers for fast affective state recognition in intelligent edge analytics

Some research shows that affective state of learners is not less important for didactic effectiveness than their cognitive state. This paper describes an attempt to stimulate some affective states in order to boost a didactic process quality by using augmented reality systems as a stimuli. Results of some research about AR effectiveness in didactic process were analyzed, and they show that augmented reality as it is isn’t sufficient to entail and preserve adequate affective states in pupils. Subsequently, there were shown examples AR applications that stimulate specific affective states crucial for optimal didactic process. Some Augmented Reality application mechanisms were suggested, that corresponds with those affective states beneficial for didactics optimization. Finally, users of those applications were examined, and the results were analyzed in order to suggest a guidelines for designing AR applications, which by using affective stimulation are best suited for didactic process.KeywordsAugmented realityAffective statesAR effectiveness

Simple SummaryIn the past, there was a strong focus on avoiding or reducing negative animal welfare in animal experimentation. Recently, the importance of promoting positive animal welfare in laboratory animals has been highlighted. To ensure and promote positive animal welfare, reliable methods to evaluate the animal’s emotional state are important. Important achievements have been made to assess pain and other negative states in animals in the last decades, and only recently have positive emotions been gaining more interest. Therefore, more methods allowing the assessment of emotional states in animals have been introduced. In this overview article, we present common and emerging methods to assess emotions in laboratory rodents. We focus on the use of these methods in applied refinement research to identify achieved progress as well as the potential of these tools to improve animal welfare in animal-based research and animal experimentation.An animal’s capacity to suffer is a prerequisite for any animal welfare concern, and the minimization of suffering is a key aim of refinement research. In contrast to the traditional focus on avoiding or reducing negative welfare states, modern animal welfare concepts highlight the importance of promoting positive welfare states in laboratory animals. Reliable assessments of affective states, as well as the knowledge of how to elicit positive affective states, are central to this concept. Important achievements have been made to assess pain and other negative affective states in animals in the last decades, but it is only recently that the neurobiology of positive emotions in humans and animals has been gaining more interest. Thereby, the need for promotion of positive affective states for laboratory animals is gaining more acceptance, and methods allowing the assessment of affective states in animals have been increasingly introduced. In this overview article, we present common and emerging methods to assess affective states in laboratory rodents. We focus on the implementation of these methods into applied refinement research to identify achieved progress as well as the future potential of these tools to improve animal welfare in animal-based research.

The information about affective states in individuals with autism spectrum disorder (ASD) is difficult to obtain as they usually suffer from deficits in facial expression. Affective state conditions of individuals with ASD were associated with impaired regulation of speech, communication, and social skills leading towards poor socio-emotion interaction. It is conceivable that the advance of technology could offer a psychophysiological alternative modality, particularly useful in persons who cannot verbally communicate their emotions as affective states such as individuals with ASD. The study is focusing on the investigation of technology-assisted approach and its relationship to affective states recognition. A systematic review was executed to summarize relevant research that involved technology-assisted implementation to identify the affective states of individuals with ASD using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) approach. The output from the online search process obtained from six publication databases on relevant studies published up to 31 July 2020 was analyzed. Out of 391 publications retrieved, 20 papers met the inclusion and exclusion criteria set in prior. Data were synthesized narratively despite methodological and heterogeneity variations. In this review, some research methods, systems, equipment and models to address all the related issues to the technology-assisted and affective states concerned were presented. As for the consequence, it can be assumed that the emotion recognition with assisted by technology, for evaluating and classifying affective states could help to improve efficacy in therapy sessions between therapists and individuals with ASD. This review will serve as a concise reference for providing general overviews of the current state-of-the-art studies in this area for practitioners, as well as for experienced researchers who are searching for a new direction for future works.

Artificial intelligence (AI) models can sense subjective affective states from facial images. Although recent psychological studies have indicated that dimensional affective states of valence and arousal are systematically associated with facial expressions, no AI models have been developed to estimate these affective states from facial images based on empirical data. We developed a recurrent neural network-based AI model to estimate subjective valence and arousal states from facial images. We trained our model using a database containing participant valence/arousal states and facial images. Leave-one-out cross-validation supported the validity of the model for predicting subjective valence and arousal states. We further validated the effectiveness of the model by analyzing a dataset containing participant valence/arousal ratings and facial videos. The model predicted second-by-second valence and arousal states, with prediction performance comparable to that of FaceReader, a commercial AI model that estimates dimensional affective states based on a different approach. We constructed a graphical user interface to show real-time affective valence and arousal states by analyzing facial video data. Our model is the first distributable AI model for sensing affective valence and arousal from facial images/videos to be developed based on an empirical database; we anticipate that it will have many practical uses, such as in mental health monitoring and marketing research.

Simple SummaryIt is widely recognized that the assessment of animal welfare should include measures of positive emotional (affective) state. Existing behavioral and physiological indicators of a positive affective state frequently lack sensitivity, objectivity or are unsuitable in a production environment. Therefore, there is a need to develop new approaches to accurately and objectively measure a positive emotional state in animals, including novel molecular markers such a miRNA. These biomarkers must be measurable in the peripheral circulation and provide an accurate account of the physiological and molecular activity in regions of the brain associated with emotional processing. Further, such markers require validation against established behavioral and physiological indices. Here we investigated the efficacy of circulating miRNA as biomarkers of emotional state in the pig.The ability to assess the welfare of animals is dependent on our ability to accurately determine their emotional (affective) state, with particular emphasis being placed on the identification of positive emotions. The challenge remains that current physiological and behavioral indices are either unable to distinguish between positive and negative emotional states, or they are simply not suitable for a production environment. Therefore, the development of novel measures of animal emotion is a necessity. Here we investigated the efficacy of microRNA (miRNA) in the brain and blood as biomarkers of emotional state in the pig. Female Large White × Landrace pigs (n = 24) were selected at weaning and trained to perform a judgment bias test (JBT), before being exposed for 5 weeks to either enriched (n = 12) or barren housing (n = 12) conditions. Pigs were tested on the JBT once prior to treatment, and immediately following treatment. MiRNA and neurotransmitters were analyzed in blood and brain tissue after euthanasia. Treatment had no effect on the outcomes of the JBT. There was also no effect of treatment on miRNA expression in blood or the brain (FDR p > 0.05). However, pigs exposed to enriched housing had elevated dopamine within the striatum compared to pigs in barren housing (p = 0.02). The results imply that either (a) miRNAs are not likely to be valid biomarkers of a positive affective state, at least under the type of conditions employed in this study, or (b) that the study design used to modify affective state was not able to create differential affective states, and therefore establish the validity of miRNA as biomarkers.

It is argued that a robotic system that is capable of implicit communication with a human and can modify its behavior appropriately based on such communication could be useful. This paper presents a closed loop human-robot interaction framework where the robot can recognize the affective state of the human implicitly and adapt to it appropriately. Affective cues are inferred by a robot in real-time using psychophysiological analysis where the physiological signals are measured through wearable biofeedback sensors. A robot-based basketball game is designed where a robotic coach monitors the participant's anxiety to alter the difficulty level of the game in a real-time closed loop manner. The results are compared with situations when anxiety is not monitored and the game is adapted only according to the performance. Results show that monitoring and responding to affective cues led to higher performance improvement of the participants under lower anxiety. This is the first time, to our knowledge, that the impact of such implicit communication between a robot and a human has been demonstrated experimentally.

Simple SummaryOur research team has developed an automated computer system that uses convolutional neural networks (CNNs) to monitor and analyse the sleep patterns of dogs. Traditional methods of recording animal behaviour, such as direct observations (of sleep) of either live behaviour or recorded behaviour, can be time-consuming and error-prone, making it difficult to replicate studies. Sleep may be a crucial indicator of an animal’s well-being, but it has been overlooked in animal welfare research due to the time-consuming nature of measuring sleep. Compared to direct behavioural observations from the same videos, our system achieved an 89% similarity score in automatically detecting and quantifying sleep duration and fragmentation in dogs. Although there were no significant differences in the time percentage of sleep observed, the system recorded more total sleep time than human observers making direct observations on the same data sources. The automated system used could become a valuable tool for animal behaviour and welfare research.Although direct behavioural observations are widely used, they are time-consuming, prone to error, require knowledge of the observed species, and depend on intra/inter-observer consistency. As a result, they pose challenges to the reliability and repeatability of studies. Automated video analysis is becoming popular for behavioural observations. Sleep is a biological metric that has the potential to become a reliable broad-spectrum metric that can indicate the quality of life and understanding sleep patterns can contribute to identifying and addressing potential welfare concerns, such as stress, discomfort, or health issues, thus promoting the overall welfare of animals; however, due to the laborious process of quantifying sleep patterns, it has been overlooked in animal welfare research. This study presents a system comparing convolutional neural networks (CNNs) with direct behavioural observation methods for the same data to detect and quantify dogs’ sleeping patterns. A total of 13,688 videos were used to develop and train the model to quantify sleep duration and sleep fragmentation in dogs. To evaluate its similarity to the direct behavioural observations made by a single human observer, 6000 previously unseen frames were used. The system successfully classified 5430 frames, scoring a similarity rate of 89% when compared to the manually recorded observations. There was no significant difference in the percentage of time observed between the system and the human observer (p > 0.05). However, a significant difference was found in total sleep time recorded, where the automated system captured more hours than the observer (p < 0.05). This highlights the potential of using a CNN-based system to study animal welfare and behaviour research.

Simple SummaryFor at least the last four decades, the focus of animal welfare research, quality assurance, and policy initiatives has been on measuring behavioural and physiological stress responses in animals. In the last decade, however, this focus of animal welfare research has shifted to the consequences of these behavioural and physiological stress responses rather than only the responses per se. Modern-day farming, even with the intensification and automation requires regular monitoring and interactions by stockpeople. Research conducted in both experimental and commercial settings has shown widespread effects of the human-animal interactions on behaviour, physiology, and reproductive performance in farm animals. In this paper, we review the implications of human-animal interactions on reproduction in farm animals.A negative human-animal relationship (HAR) from the perspective of the animal is a limiting factor affecting farm animal welfare, as well as farm animal productivity. Research in farm animals has elucidated sequential relationships between stockperson attitudes, stockperson behaviour, farm animal fear behaviour, farm animal stress physiology, and farm animal productivity. In situations where stockperson attitudes to and interactions with farm animals are sub-optimal, through animal fear and stress, both animal welfare and productivity, including reproductive performance, can be compromised. There is a growing body of evidence that farm animals often seek and enjoy interacting with humans, but our understanding of the effects of a positive HAR on stress resilience and productivity in farm animals is limited. In this review, we explore the pathways by which stress induced by human-animal interactions can negatively affect farm animal reproduction, in particular, via inhibitory effects on the secretion of gonadotrophins. We also review the current knowledge of the stockperson characteristics and the nature of stockperson interactions that affect fear and physiological stress in farm animals. The contents of this review provide an insight into the importance of the HAR on farm animal welfare and reproduction while highlighting the gap in knowledge regarding the effects of a positive HAR on farm animals.

Digital twins are virtual models of physical artefacts that may or may not be synchronously connected, and that can be used to simulate their behavior. They are widely used in several domains such as manufacturing and automotive to enable achieving specific quality goals. In the health domain, so-called digital patient twins have been understood as virtual models of patients generated from population data and/or patient data, including, for example, real-time feedback from wearables. Along with the growing impact of data science technologies like artificial intelligence, novel health data ecosystems centered around digital patient twins could be developed. This paves the way for improved health monitoring and facilitation of personalized therapeutics based on management, analysis, and interpretation of medical data via digital patient twins. The utility and feasibility of digital patient twins in routine medical processes are still limited, despite practical endeavors to create digital twins of physiological functions, single organs, or holistic models. Moreover, reliable simulations for the prediction of individual drug responses are still missing. However, these simulations would be one important milestone for truly personalized therapeutics. Another prerequisite for this would be individualized pharmaceutical manufacturing with subsequent obstacles, such as low automation, scalability, and therefore high costs. Additionally, regulatory challenges must be met thus calling for more digitalization in this area. Therefore, this narrative mini-review provides a discussion on the potentials and limitations of digital patient twins, focusing on their potential bridging function for personalized therapeutics and an individualized pharmaceutical manufacturing while also looking at the regulatory impacts.

Biopharma Is Going Digital … Bit by Bit

Simple SummaryActions of human caretakers influence the experience of animals under their care, in zoos and elsewhere. These animals often learn to associate stimuli—sights, smells, sounds—with desirable outcomes such as feedings, training sessions, or other positive experiences. Here, we propose that a conscientious approach to providing reliable cues about daily events and observing animal behavior in response to both reliable and uncertain cues can help caretakers support and assess animal welfare.The actions of human caretakers strongly influence animals living under human care. Here, we consider how intentional and unintentional signals provided by caretakers can inform our assessment of animals’ well-being as well as help to support it. Our aim is to assist in further developing techniques to learn animals’ affective state from their behavior and to provide simple suggestions for how animal caretakers’ behavior can support animal welfare. We suggest that anticipatory behavior towards expected rewards is related to decision-making behavior as viewed through the cognitive bias lens. By considering the predictions of the theories associated with anticipatory behavior and cognitive bias, we propose to use specific cues to probe the cumulative affective state of animals. Additionally, our commentary draws on the logic of reward sensitivity and judgement bias theories to develop a framework that suggests how reliable and equivocal signals may influence animals’ affective states. Application of this framework may be useful in supporting the welfare of animals in human care.

In modern medicine, novel technologies play a key role in improving the diagnosis, treatment, and management of patient health. One of the most promising innovations is Digital Twin technology, which enables the creation of virtual models of real-world objects, including human organs. This technology opens new possibilities for personalised medicine, disease prediction, and the optimisation of medical processes. Digital Twins are already actively used in various industries, such as manufacturing, aviation, and urban planning, but their potential in medicine is only beginning to unfold. They allow for creating detailed models of an individual patient’s body, contributing to more accurate diagnostics, simulation of treatment responses, and enhanced efficiency of medical interventions. This article explores the key aspects of Digital Twin medical applications, their advantages, challenges, and development prospects. Special attention is given to the technical foundations of this technology, its use in personalised medicine, health monitoring, disease prediction, and the optimisation of healthcare facilities. For the practical part, the study shows how to create a Digital Twin of a heart based on real patient data. Additionally, the study investigates a model of the heart’s Digital Twin, demonstrates its use, and illustrates how to replicate heart functionality using the Digital Twin. The study describes how a part of a Digital Twin was created using the suggested architecture and outlines how to use and extend this functionality. More than that, the study shows how to integrate artificial intelligence into the Digital Twin architecture and how artificial intelligence algorithms help create Digital Twins. The research shows how to build the Digital Twin using machine learning and artificial intelligence technologies, and how to choose the mathematical model for the Digital Twin. To conclude, the study defines further work, problems, and difficulties in creating the Digital Twin of the heart. The study shows the perspectives on using Digital Twins, their practicality, and real-life cases where Digital Twins can save lives. Also, the study defines the next steps for the Digital Twin creation, testing, implementation, and usage in the real-life healthcare industry.

Facial behaviour forms an important cue for understanding human affect. While a large number of existing approaches successfully recognize affect from facial behaviours occurring in daily life, facial behaviours displayed in work settings have not been investigated to a great extent. This paper presents the first study that systematically investigates the influence of spatial and temporal facial behaviours on human affect recognition in work-like settings. We first introduce a new multi-site data collection protocol for acquiring human behavioural data under various simulated working conditions. Then, we propose a deep learning-based framework that leverages both spatio-temporal facial behavioural cues and background information for workers’ affect recognition. We conduct extensive experiments to evaluate the impact of spatial, temporal and contextual information for models that learn to recognize affect in work-like settings. Our experimental results show that (i) workers’ affective states can be inferred from their facial behaviours; (ii) models pre-trained on naturalistic datasets prove useful for predicting affect from facial behaviours in work-like settings; and (iii) task type and task setting influence the affect recognition performance.