Conclusion

Researchers in the fields of AI and Communication both study human communication, but despite the opportunities for collaboration, they rarely interact. Red Hen Lab is dedicated to bringing them together for research on multimodal communication, using multidisciplinary teams working on vast ecologically-valid datasets. This article introduces Red Hen Lab with some possibilities for collaboration, demonstrating the utility of a variety of machine learning and AI-based tools and methods to fundamental research questions in multimodal human communication. Supplemental materials are at http://babylon.library.ucla.edu/redhen/KI .

Humans and other primates communicate in multiple sensory modalities, and language itself is usually a multimodal form of communication. Discussion of the multimodal nature of primate communication goes back at least as far as the publications of Charles Darwin and has recently seen renewed interest. Here, I review key topics in the study of multimodal communication in nonhuman primates and humans, including issues of definitional complexity as well as classification systems and empirical approaches. I argue that multimodal communication is ubiquitous, ingrained, and advantageous. I discuss sensory aging and how multimodal communication can offer alternative routes to comprehension when sensory systems become impaired. To conclude, I consider future avenues of research that seem likely to prove productive.

Language evolution refers to the origins and development of the human system of communication. Research on language evolution compares aspects of human communication with those of other animals, especially nonhuman primates, and analyzes gestures, multimodal communication, pidgins and creoles, child development, the physiology of the vocal tract, genes and neurons, the brain and brain damage, as well as the fossil record. Theories of language evolution discuss whether human language emerged gradually or abruptly and through which evolutionary adaptations or exaptations, as well as whether there are any key features of human language. Controversies remain but recent research has contributed to a more nuanced picture of differences between humans and nonhumans that may prove to not be qualitative but quantitative.

Turn-taking is essential to the structure of human teamwork. Humans are typically aware of team members' intention to keep or relinquish their turn before a turn switch, where the responsibility of working on a shared task is shifted. Future co-robots are also expected to provide such competence. To that end, this paper proposes the Cognitive Turn-taking Model (CTTM), which leverages cognitive models (i.e., Spiking Neural Network) to achieve early turn-taking prediction. The CTTM framework can process multimodal human communication cues (both implicit and explicit) and predict human turn-taking intentions in an early stage. The proposed framework is tested on a simulated surgical procedure, where a robotic scrub nurse predicts the surgeon's turn-taking intention. It was found that the proposed CTTM framework outperforms the state-of-the-art turn-taking prediction algorithms by a large margin. It also outperforms humans when presented with partial observations of communication cues (i.e., less than 40 % of full actions). This early prediction capability enables robots to initiate turn-taking actions at an early stage, which facilitates collaboration and increases overall efficiency.

Drawing insight from Toyin Falola’s call for African scholars to Africanize knowledge, this article argues for reviewing the digital technological tools used for African Studies research to process and present African data properly. To achieve this, the inadequacies of digital humanities (DH) for specific areas of African Studies will be highlighted, especially in the deployment of digital humanities tools. The major challenge is the distortion and constraint experienced in processing and presenting research through DH means of translation and communication. The article argues that such technological limitation has its root in the incompatibility of the epistemological frameworks within which those digital tools were developed. The article discusses Ojú lòrówà (‘discussion is in the eye’), that is, ‘communication takes place when we see physically’ – a theory of communication in African society used as a model to highlight the importance of African context to African scholars in their exploration of African history, technology, culture, philosophy and tradition. The indigenous theory is an appropriate model for developing digital and virtual software for African scholars in human communication. The article concludes by urging scholars in African Studies to ensure that the digital tools employed in African Studies can collect data and process and present data adequately without losing the original meaning or sense.

Motivated by the combination of knowing and doing in humanities curricula, this piece gives audiences a concrete sense of teaching and learning multimodal communications at the graduate level. Together with seven example prompts and multiple student responses to each, the piece also includes student analysis, reflection, and commentary on the processes of learning, composing, and circulating digital scholarship. In so doing, it demonstrates specific ways of using the authoring and publishing platform, Scalar, for collaborative research and writing in and beyond the classroom. The materials provided are drawn from a graduate seminar in the Department of English at the University of Victoria: “Digital Literary Studies: History and Principles.” Intended audiences for the piece include writing studies instructors, literary critics, cultural studies scholars, digital humanities and media studies practitioners, transmedia artists, and anyone who supports and facilitates university curricula and related infrastr...

Two experiments were performed to investigate the feasibility for robot-to-human communication of a tactile language using a lexicon of standardized tactons (tactile icons) within a sentence. Improvements in autonomous systems technology and a growing demand within military operations are spurring interest in communication via vibrotactile displays. Tactile communication may become an important element of human-robot interaction (HRI), but it requires the development of messaging capabilities approaching the communication power of the speech and visual signals used in the military. In Experiment 1 (N = 38), we trained participants to identify sets of directional, dynamic, and static tactons and tested performance and workload following training. In Experiment 2 (N = 76), we introduced an extended training procedure and tested participants' ability to correctly identify two-tacton phrases. We also investigated the impact of multitasking on performance and workload. Individual difference factors were assessed. Experiment 1 showed that participants found dynamic and static tactons difficult to learn, but the enhanced training procedure in Experiment 2 produced competency in performance for all tacton categories. Participants in the latter study also performed well on two-tacton phrases and when multitasking. However, some deficits in performance and elevation of workload were observed. Spatial ability predicted some aspects of performance in both studies. Participants may be trained to identify both single tactons and tacton phrases, demonstrating the feasibility of developing a tactile language for HRI. Tactile communication may be incorporated into multi-modal communication systems for HRI. It also has potential for human-human communication in challenging environments.

Spoken dialogs represent a considerable challenge for language technology for at least two reasons: (a) due to the frequent incompleteness of syntactic structure a given sequence of words cannot often undergo satisfactory parsing, (b) as a result, the associated fragmented prosody may not be used to substitute for missing elements. However, spoken dialogs are essentially multimodal with nonverbal means having the potential of offering some particular structural cues as well as functional-pragmatic ones. An incomplete syntax should in principle break down automatic syntactic analysis, however, its combination with the output of other; non-verbal modalities may generate cues to some possible interpretation and decision making. An important challenge is to identify a proper complex of such aligning cues for a suitable decision making. In this context relying on larger amounts of data gleaned from longer stretches of communicative events may contribute to a more successful representation of the cognitive content behind similar instances of fragmented communication if these sets of data are based on a theoretically founded understanding of multimodal communication. The HuComTech multimodal corpus of 60 hours of spoken dialogues of Hungarian [1] consists of a multi-level annotation of both physical and interpretative (functional, pragmatic) aspects of communication. Annotation is not purely descriptive: it is based on a novel generative model of communication allowing for the capturing of the two-way, top down and bottom up character of communication with the purpose to identify the individual minimal building blocks of communication and to get a deeper understanding of how these components make up a complex communicative event as a whole. Based on these insights, a novel scheme of annotation for syntax, prosody as well as uni- and multimodal pragmatics is introduced aiming at describing and understanding the multimodal nature of human communication, cognitive info-communication in general, and, on the longer run, contributing to the further development of human-computer interaction systems.

One of the most important motivations for many humanoid robot projects is that robots with a human-like body and human-like senses could in principle be capable of intuitive multimodal communication with people. The general idea is that by mimicking the way humans interact with each other, it will be possible to transfer the efficient and robust communication strategies that humans use in their interactions to the man-machine interface. This includes the use of multiple modalities, such as speech, facial expressions, gestures, body language, etc. If successful, this approach yields a user interface that leverages the evolution of human communication and that is intuitive to naive users, as they have practiced it since early childhood. We work towards intuitive multimodal communication in the domain of a museum guide robot. This application requires interacting with multiple unknown persons. The testing of communication robots in science museums and on science fairs is popular, because the robots encounter there many new interaction partners, which have a general interest in science and technology. Here, we present the humanoid communication robot Fritz that we developed as successor to the communication robot Alpha (Bennewitz et al., 2005). Fritz uses speech, facial expressions, eye-gaze, and gestures to interact with people. Depending on the audio-visual input, our robot shifts its attention between different persons in order to involve them into an interaction. He performs human-like arm gestures during the conversation and also uses pointing gestures generated with eyes, its head, and arms to direct the attention of its communication partners towards the explained exhibits. To express its emotional state, the robot generates facial expressions and adapts the speech synthesis. The remainder of the chapter is organized as follows. The next section reviews some of the related work. The mechanical and electrical design of Fritz is covered in Sec. 3. Sec. 4 details the perception of the human communication partners. Sec. 5 explains the robot's attentional system. The generation of arm gestures and of facial expressions is presented in Sec. 6 and 7, respectively. Finally, in the experimental section, we discuss experiences made during public demonstrations of our robot.

Engaged Humanities and the Future of Spanish Programs Ann Abbott Keywords curricular reform/reforma curricular, engaged humanities/humanidades comprometidas, foreign language study/estudio de lengua extranjera, Hispanism/hispanismo, reductionist models/modelos reduccionistas, Service-Learning/aprendizaje del servicio, Spanish for the Professions/español para profesiones Response to "Apologia No More: On Strong Foundations and the Future of Hispanism" More than just finally articulating the value of what we do in Spanish programs, we must actually change a significant portion of what we do—and how we do it—in order to offer the kind of value the authors wisely propose. To provide solutions to "the possibility of non-Hispanic American cultural resistance to the growth of Hispanic communities" (Bayliss and Rossomondo 2017) we need to focus much of our curricula on US Latinos, commit to social justice education and engage with our local Latino communities. Few departments do this, though, because few departments have truly seen this as their mission. To prepare students to be civically-engaged, savvy cultural critics who are equipped to combat highly-charged, racialized discourses will require integrating into our curriculum topics such as human migrations (historical and global), economics, policy studies, communication, media studies and more. Yet the vast majority of departments continue to mostly teach students how to analyze cultural products (literary studies, predominantly) and language (linguistics). To bridge what Spanish programs actually do and what the authors claim we are preparing students to do requires a more radical—and uncomfortable—shift than just finding the right words. This curricular shift must occur swiftly. A decade has already passed since the MLA's special report on foreign languages (cited by the authors) as well as Carlos Alonso's declaration in Profession that Spanish is now "a second national language and culture in this country" (220). Yet most Spanish departments in the United States still operate as foreign language programs, perennially privileging Spain and reproducing value systems and power structures the authors suggest we can dismantle in the United States. They astutely propose that Spanish departments embrace the local and transnational nature of Spanish, but our current approach of scattered heritage speaker sections and service learning courses is insufficient. Departments that adopt the engaged humanities model and a mission to see and seek connections among traditional scholarly projects of inquiry and the issues surrounding them imbue their programs with an urgency and relevance that our profession as a whole currently lacks but that students and other stakeholders seek. The demographic trends listed by the authors will consolidate by 2067, but we can change today. Take the concrete example of campuses located in new-growth communities—places where immigrants have not traditionally settled and which lack infrastructure to build linguistic and transcultural competencies. Cultural studies scholars and students can work with the community to analyze depictions of these new encounters and suggest more accurate, helpful representations of the challenges and opportunities within their changing communities. Second language acquisition experts and schools can partner to create positive responses to [End Page 33] multilingualism, inside and outside the K–12 classrooms. Literary scholars and libraries can collaborate to build collections, attract Latino patrons and design relevant programming. Language program directors can integrate service learning across the curriculum to provide targeted assistance that meets community-identified needs. Conceived as the department's mission, these activities can be integrated into regular research, teaching and service obligations, not heaped upon them. Nonetheless, a quick look at the dissertations produced in Spanish departments ("Open Access Dissertation Lists") reveals that our focus on literary analysis and linguistics has barely budged, constantly reproducing frameworks that do not actually address the societal needs the authors foreground. Lastly, our profession should embrace both civic and career connections to Spanish studies. After all, the important civic project the authors outline plays out in workplaces and among colleagues and clients, not just in voting booths and neighborhoods. We perpetuate our own reductionism by presenting a focus on careers as a caricature of vocational training. Instead, it is an opportunity to engage with the complexities and creative challenges of professionals who must not just develop but also deploy translingual and transcultural competence in real time, with real people, not...