Establishing reference points for artificial social agent evaluation: the ASAQ representative set 2025

Abstract: Research links a progressive decrement in episodic memory to deficits in judgment and decision making in aging. In social decision making, decrements in episodic memory contribute to suboptimal decisions among older adults. As artificial agents – such as humanoid robots, artificial intelligence (AI)-powered deepfakes, and chatbots – become increasingly present in human society, they can both benefit (e.g., combat loneliness) and harm (e.g., defraud) older adults. Understanding how older adults make social judgments and decisions involving artificial agents is crucial for research and policy making. Nevertheless, it remains poorly understood how episodic memory deficits influence social judgments and decisions involving artificial agents, and how this relation changes with age. This review will bridge this gap by applying a developmental model to explain how memory influences social judgments and decisions involving artificial agents in late adulthood. Evidence suggests that older adults increasingly rely on gist-based processing, which may explain their greater preference for humanoid robots compared with younger adults. We will also discuss the distinct challenges deepfakes pose for older adults’ trust-related decision making. In closing, we will discuss the implications of age-related memory changes for social judgments and decisions involving disembodied artificial agents, such as chatbots.

A model for simulating reputation dynamics in industrial districts

We present the ASA Questionnaire, an instrument for evaluating human interaction with an artificial social agent (ASA), resulting from multi-year efforts involving more than 100 Intelligent Virtual Agent (IVA) researchers worldwide. It has 19 measurement constructs constituted by 90 items, which capture more than 80% of the constructs identified in empirical studies published in the IVA conference 2013--2018. This paper reports on construct validity analysis, specifically convergent and discriminant validity of initial 131 instrument items that involved 532 crowd-workers who were asked to rate human interaction with 14 different ASAs. The analysis included several factor analysis models and resulted in the selection of 90 items for inclusion in the long version of the ASA questionnaire. In addition, a representative item of each construct or dimension was selected to create a 24-item short version of the ASA questionnaire. Whereas the long version is suitable for a comprehensive evaluation of human-ASA interaction, the short version allows quick analysis and description of the interaction with the ASA. To support reporting ASA questionnaire results, we also put forward an ASA chart. The chart provides a quick overview of the agent profile.

<p>In this paper, we report on the multi-year Intelligent Virtual Agents (IVA) community effort, involving more than 90 researchers worldwide, researching the IVA community interests and practice in evaluating human interaction with an artificial social agent (ASA). The joint efforts have previously generated a unified set of 19 constructs that capture more than 80% of constructs used in empirical studies published in the IVA conference between 2013 to 2018. In this paper, we present expert-content-validated 131 questionnaire items for the constructs and their dimensions, and investigate the level of reliability. We establish this in three phases. Firstly, eight experts generated 431 potential construct items. Secondly, 20 experts rated whether items measure (only) their intended construct, resulting in 207 content-validated items. Next, a reliability analysis was conducted, involving 192 crowd-workers who were asked to rate a human interaction with an ASA, which resulted in 131 items (about 5 items per measurement, with Cronbach's alpha ranged [.60 - .87]). These are the starting points for the questionnaire instrument of human-ASA interaction.</p>

Mature social evaluations privilege agents' intentions over the outcomes of their actions, but young children often privilege outcomes over intentions in verbal tasks probing their social evaluations. In three experiments (N =118), we probed the development of intention-based social evaluation and mental state reasoning using nonverbal methods with 15-month-old toddlers. Toddlers viewed scenarios depicting a protagonist who sought to obtain one of two toys, each inside a different box, as two other agents observed. Then, the boxes' contents were switched in the absence of the protagonist and either in the presence or the absence of the other agents. When the protagonist returned, one agent opened the box containing the protagonist's desired toy (a positive outcome), and the other opened the other box (a neutral outcome). When both agents had observed the toys move to their current locations, the toddlers preferred the agent who opened the box containing the desired toy. In contrast, when the agents had not seen the toys move and therefore should have expected the desired toy's location to be unchanged, the toddlers preferred the agent who opened the box that no longer contained the desired toy. Thus, the toddlers preferred the agent who intended to make the protagonist's desired toy accessible, even when its action, guided by a false belief concerning that toy's location, did not produce a positive outcome. Well before children connect beliefs to social behavior in verbal tasks, toddlers engage in intention-based evaluations of social agents with false beliefs.

This study empirically evaluates the perceived realism of six micromovement strategies implemented in two agent embodiments (Humanoid and Dummy).Thirty participants (ages 18-55) recruited via the Gorilla platform viewed randomised 5-second video clips (six micro-movement strategies two embodiments).After each clip, participants rated five dimensions-Resemblance, Daily-Life Likelihood, Efficiency, Movement Realism, and Overall Agent Realism-on a continuous 0-100 scale.Descriptive statistics for a representative movement (TurnBackward) revealed that the Dummy variant scored lower in Overall Agent Realism than the Humanoid, despite similar Movement Realism ratings.Two-way ANOVAs showed significant main effects of Movement Type on Resemblance, Day-to-day, Efficiency, and Movement Realism but no significant Movement Embodiment interactions.Agent type significantly influenced only Overall Agent Realism.Mixed Linear Models corroborated that CurveForward and Forward movements positively impacted Resemblance and Day-to-day, while Backward and Strafe had negative effects.Correlation analysis revealed strong positive associations among Resemblance, Day-to-day, Efficiency, and Movement Realism, but weaker links to Overall Agent Realism.These findings validate the proposed micro-movement taxonomy and indicate that movement kinematics drive perceived realism more than agent appearance, guiding future ASA controller design.

Humans interpret and predict others’ behaviors by ascribing intentions or beliefs, or in other words, by adopting the intentional stance. Since artificial agents are increasingly populating our daily environments, the question arises whether (and under which conditions) humans would apply the “human model” to understand the behaviors of these new social agents. Thus, in a series of three experiments, we tested whether embedding humans in a social interaction with a humanoid robot either displaying a human-like or machine-like behavior would modulate their initial tendency to adopt the intentional stance. Results showed that indeed humans are more prone to adopt the intentional stance after having interacted with a more socially available and human-like robot, while no modulation of the adoption of the intentional stance emerged toward a mechanistic robot. We conclude that short experiences with humanoid robots presumably inducing a “like-me” impression and social bonding increase the likelihood of adopting the intentional stance.

Anthropomorphic Artificial Social Agent with Simulated Emotions and its Implementation

Este artigo propõe reflexões acerca da avaliação médico-social da pessoa com deficiência no acesso ao Benefício de Prestação Continuada da Lei Orgânica da Assistência Social. A abordagem qualitativa da pesquisa possibilitou a análise do movimento do real, a partir do processo de trabalho de assistentes sociais e médicos peritos do INSS. Foram realizadas entrevistas com os referidos profissionais de agências da previdência social de Fortaleza-CE. Os interlocutores revelaram uma realidade, com relação ao modus operandi desse processo avaliativo, que referenda a ausência de uniformidade no entendimento do modo de avaliar a pessoa com deficiência que pleiteia o BPC.

Evaluating others’ prosocial tendencies can benefit individuals by allowing them to interact with prosocial individuals and avoid antisocial ones. The ontogeny of humans’ strong prosocial preference has been widely investigated using the hill paradigm. Infants’ preference for helper over hinderer agents was measured after they watched a scene in which the helper agent pushed a climber up a hill while the hinderer agent pushed the climber down the hill. Bonobos tested with the hill paradigm preferred the hinderer over the helper, contrasting previous findings for other nonhuman primates. In this study, we explored whether another primate species would exhibit a hinderer preference using the same procedure as the one used with bonobos. Tonkean macaques (Macaca tonkeana) did not prefer the helper over the hinderer (or vice versa). While the small sample size (n = 12) and low attentional level observed in our subjects limit interpretation, this finding contributes to a broader critical thinking on the relevance of the hill paradigm to investigate prosocial preferences and on methodological limitations when testing nonhuman animals. Studies using various experimental paradigms with conspecifics or human actors as social agents are needed to further investigate the social evaluation of prosocial behaviours in primates.

In this paper, we are developing the CIVS (Civilization-Inspired Vying Societies) system, which is a novel evolutionary learning multi-agent system loosely inspired by the history of human civilization. The main objective of the CIVS system is to develop a bottom-up artificial-life approach to produce artificial agents that are inherently social in how they think, learn, adapt, and operate, so as to be more adaptable and generally intelligent in a complex, challenging environment. To this end, the CHARISMA (Context Hierarchy-based Adaptive ReasonIng Self-Motivated Agent) cognitive architecture is proposed for the agents within the CIVS system. By using intrinsic motivations and dynamic knowledge representation, the CHARISMA cognitive architecture can provide autonomous mental development capability for the agent to develop its knowledge and skills through its own interactions within a dynamic challenging environment as well as social interactions with other agents.