Point & Click - Interaction in Smart Environments

A service robot should be able to detect and identify the user in order to personalize its services and guarantee security, it should recognize the user’s emotions to allow affective interaction, and it should be able to communicate easily with the user and understand given commands by recognizing speech and gestures. Our research is motivated, for example, by the emerging needs of elderly care, health care, safety, and logistics. We have developed a distributed system for affective human–robot interaction which combines all the basic sensory elements and can collaborate with a smart environment and obtain further knowledge from the Internet. The realized HRI robot, Minotaurus, runs in real time and is capable of interacting with human beings. Minotaurus forms a rather generic platform for experimenting with human–robot collaboration in different applications and environments.

It is now a common belief that the truths of our lives are hidden in the databases streamed from our interactions in smart environments. In this current hype of big data, the Internet of Things has been suggested as the idea to embed small sensors and actuators everywhere to unfold the truths beneath the surfaces of everything. However, remaining the technology that promises more than it can provide thus far, more important for the IoT’s actual expansion to various social domains than the actual discovery of hidden truths has been people’s speculations about the unknown problems, such as hidden security issues or lifestyle concerns, beyond the narrow human knowability but assumed to leave their traces in the IoT-collected big data. This paper discuss this speculation as the concealed cognitive labor of IoT users that projects some fictitious values to the big data IoT companies accumulate. By the term pan-kinetics, the systemic operation of smart actuators is analyzed as the process through which fictitious values of data are converted to the real values as these actuators draw some profitable correlations from physical domains of the IoT. Analyzing smart electroencephalogram (EEG) headsets as the unique IoT devices operating on human brains, it argues how the IoT translates this speculative realism of unknown problems into its big data, which the IoT developers believe to be full of machine-learnable correlations that would lead to the smart solutions of the problems.

The ongoing utilization of computer technologies in all areas of life leads to the development of smart environments comprising numerous networked devices and resources. Interacting in and with such environments requires new interaction paradigms, abstracting from single interaction devices to utilize the environment as interaction space. Using a networked set of interaction resources allows supporting multiple modalities and new interaction techniques, but also requires the consideration of the set of devices and the adaptation to this set at runtime. While the generation of user interfaces based on UI models, although still challenging, has been widely researched, the runtime processing and delivery of the derivable user interfaces has gained less attention. Delivering distributed user interfaces while maintaining their interdependencies and keeping them synchronized is not a trivial problem. In this paper we present an approach to realize a runtime environment, capable of distributing user interfaces to a varying set of devices to support multimodal interaction based on a user interface model and the management of interaction resources.Keywordsmultimodal distributed user interfacesmodel-based user interfaceshuman-computer interactionsmart environments

The automatic monitoring of specific Activities of Daily Living (ADL) can be a useful tool for Human-Robot Interaction in smart environments and Assistive Robotics applications. The qualitative definition that is given for most ADL and the lack of well-defined benchmarks, however, are obstacles toward the identification of the most effective monitoring approaches for different tasks. The contribution of the article is two-fold: (i) we propose a taxonomy of ADL allowing for their categorization with respect to the most suitable monitoring approach; (ii) we present a freely available dataset of acceleration data, coming from a wrist-worn wearable device, targeting the recognition of 14 different human activities.

Perspectives in human-media interaction We are now in the third decade of the 21st Century, and, especially in the last years, the achievements made by scientists have been exceptional, leading to major advancements in the fast-growing field of Human-Media Interaction. This article collection is aimed at promoting a critical reflection on the research conducted so far, as well as the future perspectives on all aspects of human interaction with computers and related media. This Research Topic has solicited perspective articles that describe the state of the art, outlining recent developments and major accomplishments that have been achieved and that need to occur to move the field forward. Authors have been encouraged to identify the greatest challenges in the sub-disciplines, and how to address those challenges. The goal of this special edition Research Topic is to shed light on the progress made in the past decade in the Human-Media Interaction field, and on its future challenges to provide a thorough overview of the field. This article collection is meant to inspire, inform, and provide direction and guidance to researchers in the field. Although not limited to perspective articles, original research and review papers were welcomed as well, we focussed on receiving articles that cover all aspects of human-media interaction. Therefore, topics of interest included but were not limited to human-computer interaction and interaction design, multi-modal and multisensory interaction, verbal and non-verbal behavior, affective and physiological computing, augmented and virtual reality, human-food interaction, embodied interaction, tangible interfaces and wearables, conversational agents, interaction in smart environments, and social media interaction. Many of these topics were covered by the articles submitted to this Research Topic. In total, 20 articles were submitted, and 12 articles, including desk rejects, were rejected. In this editorial, we discuss the articles that were accepted. We can distinguish between original research, perspective, and review articles. Two "Original Research" articles, four "Perspective" articles, and two "Review" articles were accepted. We introduce and summarize these contributions in the next section. In section 3 we have some concluding remarks on these articles and how they fit in the development of human-media interaction research.

This paper presents ArmNet, a mobile sensing system for capturing the trajectory of the wrist using measurements from wrist-worn IMU devices, specifically targeting robust interaction within Internet of Things (IoT) ecosystems. Unlike existing solutions that directly map IMU data to joint positions, ArmNet integrates physical kinematic constraints with neural network modeling. This hybrid approach is crucial for resource-constrained IoT devices where computational overhead and sensor limitations are primary concerns. Specifically, kinematic priors capture spatial dependencies between the elbow and wrist, generating physics-guided intermediate features that reduce the solution space. These features, together with raw IMU data, are fed into a recurrent neural network to learn joint displacement vectors, which are then integrated into continuous trajectories. Extensive experiments show that ArmNet achieves robust and accurate arm tracking, generalizing well across users and motion patterns, thereby enabling a new modality for seamless human-computer interaction in smart environments.

The way we communicate with speech goes far beyond the words we use and nonverbal cues play a pivotal role in, e.g., conveying emphasis or expressing emotion. Furthermore, speech can also serve as a biomarker for a range of health conditions, e.g., Alzheimer’s. With a strong evolution of speech technologies, in recent years, speech has been increasingly adopted for interaction with machines and environments, e.g., our homes. While strong advances are being made in capturing the different verbal and nonverbal aspects of speech, the resulting features are often made available in standalone applications and/or for very specific scenarios. Given their potential to inform adaptability and support eHealth, it’s desirable to increase their consideration as an integral part of interactive ecosystems taking profit of the rising role of speech as an ubiquitous form of interaction. In this regard, our aim is to propose how this integration can be performed in a modular and expandable manner. To this end, this work presents a first reflection on how these different dimensions may be considered in the scope of a smart environment, through a seamless and expandable integration around speech as an interaction modality by proposing a first iteration of an architecture to support this vision and a first implementation to show its feasibility and potential.