Humidity-Based Human-Machine Interaction System for Healthcare Applications.

Flexible piezoresistive pressure sensor based on wrinkled layers with fast response for wearable applications

Flexible piezoresistive pressure sensors may exhibit excellent sensing performances to be applied in wearable electronics, medical diagnosis, and electronic skin. Herein, we report a multi-layer and phased-responsive reduced graphene oxide/MXene-based piezoresistive pressure sensor with hierarchical microspines constructed by sandpaper as the template. Thanks to the multi-level and multi-layer structure, the obtained sensor realized phased response and showed wide detection range (up to 70 kPa), fast response (response/recovery time of 40/80 ms), and excellent working stability (1000 fatigue cycles). Furthermore, the sensor was successfully applied for detecting various human motions including pulse beats, cheek bulging, nodding, finger bending, speech recognition, handwriting, and other pressure signals. Besides, a 6 × 6 sensing matrix integrated by the sensors was able to sensitively perceive the distribution of plane pressure. The findings in this work conceivably stand out as a new strategy to fabricate high-performance piezoresistive pressure sensors in the fields of intelligent healthcare and medical diagnosis, wearable electronic devices, electronic skin, and human-machine interaction.

Cation functionalized nylon composite nanofibrous mat as a highly positive friction layer for robust, high output triboelectric nanogenerators and self-powered sensors

This paper proposes a risk analysis method for human-machine interaction system (HMIS). There are five correlated risk factors being analyzed in the HMIS: human conditions, human decision processes, human behaviors, machine and environment. With the considering of the system hazards in three types of risk scenario, we put forward an evaluation method on human-machine interaction risk for the HMIS, which provides a formalized table and contains the interaction interfaces risk evaluation procedure. In the method, we use a fuzzy method to quantify the human-machine interaction risk. Finally, a case study of the unmanned aerial vehicle (UAV) control system is conducted to illustrate the effectiveness of the method, and some advices for human-machine design are given.

Conventional Human–Machine Interaction (HMI) interfaces have predominantly relied on GUI and voice commands. However, natural human communication also consists of non-verbal communication, including hand gestures like pointing. Thus, recent works in HMI systems have tried to incorporate pointing gestures as an input, making significant progress in recognizing and integrating them with voice commands. However, existing approaches often treat these input modalities independently, limiting their capacity to handle complex multimodal instructions requiring intricate reasoning of language and gestures. On the other hand, multimodal tasks requiring complex reasoning are being challenged in the language and vision domain, but these typically do not include gestures like pointing. To bridge this gap, we explore one of the challenging multimodal tasks, called Referring Expression Comprehension (REC), within multimodal HMI systems incorporating pointing gestures. We present a virtual setup in which a robot shares an environment with a user and is tasked with identifying objects based on the user’s language and gestural instructions. Furthermore, to address this challenge, we propose a hybrid neuro-symbolic model combining deep learning’s versatility with symbolic reasoning’s interpretability. Our contributions include a challenging multimodal REC dataset for HMI systems, an interpretable neuro-symbolic model, and an assessment of its ability to generalize the reasoning to unseen environments, complemented by an in-depth qualitative analysis of the model’s inner workings.

Advanced technologies such as robotics, 5G mobile communications, IoT, cloud computing and wireless sensor networks have had a huge impact and influence on manufacturing, with an increased collaboration between humans and smart systems. As the manufacturing process becomes more automated using real-time data, communication systems, Artificial Intelligence (AI) techniques and robotics feed data back into the manufacturing process. This enables the design of products that are more customized and personal, and leads to a more competitive, efficient and value-added production process by reacting more quickly to technical or human errors to avoid product and system damage while increasing workplace safety, and reducing waste, pollution, and associated costs. This edited book covers challenges, concepts, systems, architectures, technologies, and design characteristics of human-machine cooperation and interaction systems in smart manufacturing environments using state of the art technologies including AI, 5G, IoTs, Blockchains, CPS, sensing, automation and robotics. The book is aimed at researchers and engineers working on the applications of robotics and automation, HMI, HCI, CPS, sensing, information and communications technology, data science, ML/DL/AI, AR/VR, cybersecurity and electronics. It is also a useful reference for advanced students and lecturers in these fields, and will appeal to manufacturers and automation system developers.

Human-computer interaction for virtual-real fusion

Robust, high-performance tactile sensors are critical for developing soft intelligent robotics, human-machine interfaces (HMI), and wearable electronics. However, it is challenging to fabricate soft tactile sensors capable of multifunctional sensing under both fast and slow response conditions in a simple and low-cost manner. In this talk, I will present our recent efforts in developing novel self-powered sensors and HMIs for wearables and intelligent controls. I will first introduce a textile-based tactile sensor capable of multifunctional sensing (covering both dynamic and static measurements) for personalized healthcare monitoring and soft robotic control. Inspired by human skin, the tactile sensor consists of a triboelectric nanogenerator sensing layer to mimic the function of fast-adapting (FA) mechanoreceptors and a piezoresistive sensing layer to achieve the functionality of slow-adapting (SA) mechanoreceptors. Then, I will demonstrate a self-powered breath sensor to monitor the health conditions of patients and provide daily assistance for severely disabled people to control electrical household appliances and other smart systems. The new system provides the advantages of high sensitivity, good stability, low cost, and ease of use.

Antifreeze protein-based ultrastretchable and highly sensitive hydrogel for human-machine interaction

Skin-like thermoelectric (TE) films with temperature- and strain-sensing functions are highly desirable for human-machine interaction systems and wearable devices. However, current TE films still face challenges in achieving high flexibility and excellent sensing performance simultaneously. Herein, for the first time, a facile roll-to-roll strategy is proposed to fabricate an ultraflexible chalcogenide glass-polytetrafluoroethylene composite film with superior temperature- and strain-sensing performance. The unique reticular network of the composite film endows it with efficient Seebeck effect and flexibility, leading to a high Seebeck coefficient (731 µV/K), rapid temperature response (≈0.7 s), and excellent strain sensitivity (gauge factor = 836). Based on this high-performance composite film, an intelligent robotic hand for action feedback and temperature alarm is fabricated, demonstrating its great potential in human-machine interaction. Such TE film fabrication strategy not only brings new inspiration for wearable inorganic TE devices, but also sets the stage for a wide implementation of multifunctional human-machine interaction systems.

Automatically estimating 3D human poses in video and inferring their meanings play an essential role in many human-centered automation systems. Existing researches made remarkable progresses by first estimating 2D human joints in video and then reconstructing 3D human pose from the 2D joints. However, mono-directionally reconstructing 3D pose from 2D joints ignores the interaction between information in 3D space and 2D space, losses rich information of original video, therefore limits the ceiling of estimation accuracy. To this end, this paper proposes a bidirectional 2D-3D transformation framework that bidirectionally exchanges 2D and 3D information and utilizes video information to estimate an offset for refining 3D human pose. In addition, a bone-length stability loss is utilized for the purpose of exploring human body structure to make the estimated 3D pose more natural and to further increase the overall accuracy. By evaluation, estimation error of the proposed method, measured by the mean per joint position error (MPJPE), is only 46.5 mm, which is much lower than state-of-the-art methods under the same experimental condition. The improvement on accuracy will make machines to better understand human poses for building superior human-centered automation systems. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper was motivated by the demand of human-centered automation systems needing to accurately understand human poses. Existing approaches mainly focus on inferring 3D human pose from 2D joints mono-directionally. Although they made remarkable contributions to estimating 3D human pose in such a mono-directional way, we found that they ignore the 2D-3D interaction and do not use original video when inferring 3D pose from 2D joints. This paper therefore suggests a bidirectional 2D-3D transformation that exchanges 2D and 3D information and utilizes video information to estimate more accurate 3D human pose for human-centered automation systems. This work is a pioneering attempt of interactively using 2D and 3D information for more accurate estimation of human pose. Benefited from the state-of-the-art accuracy, the proposed approach is expected to make significant contributions to many human-centered automation systems, such as human-machine interaction, biomimetic manipulation, and automatic surveillance systems.

The impact of immediate emotions on human decision-making has been extensively studied; however, their role within human–machine interactive systems remains underexplored. This study investigates how emotional states influence decision-making behaviors in such systems by introducing a novel two-tiered progressive inquiry model—environment–emotion and emotion–decision. Using Kuhn poker as an experimental context, we developed an intelligent decision support system based on Counterfactual Regret Minimization algorithms to provide real-time strategic advice to participants. Our findings reveal that positive emotional states lead individuals to be more risk-seeking and less inclined to collaborate with the AI-driven system, whereas negative emotional states result in risk-averse tendencies and higher compliance with system recommendations. By systematically uncovering the causal mechanisms through which environmental factors regulate emotions and subsequently affect decision-making, this research provides critical theoretical and empirical insights for optimizing human–machine interaction design.

Although in-vehicle interaction technologies offer assistances for drivers from multi-aspects, the distractions that may cause by these technologies would significantly challenge driving safety. This study examined the influences of three in-vehicle interactions on drivers' driving performance and eye movement behavior. The three in-vehicle interactions include phone pick-up (PPU), Bluetooth and audio $(\mathbf{BT}+\mathbf{AUD})$ and Bluetooth and head-up display $(\mathbf{BT}+\mathbf{HUD})$ . A control group (i.e., None, in which no human-machine interaction was included while driving) was designed to highlight the influences of the examined interactions. Fourteen licensed drivers participated in the simulation experiments. Their age ranged from 20 to 30 years old $(\mathbf{Mean}\pm \mathbf{SD}=25.1\pm 3.8)$ . They were required to complete the Location Based Service (LBS) tasks using the three interactions while driving. Both driving performance and eye movement data were collected for analysis. Results showed that, when using PPU or $\mathbf{BT}+\mathbf{HUD}$ , the minimum throttle opening, the steering wheel angle (SWA) and the Shannon entropy (SE) of SWA were all greater than those values when using $\mathbf{BT}+\mathbf{AUD}$ or None. As for the SE of throttle opening and all the examined eye movement features (frequency and duration of fixation/saccade), the values when using $\mathbf{BT}+\mathbf{AUD}$ or $\mathbf{BT}+\mathbf{HUD}$ were higher than the values when using PPU or None. The presented results in this study provide evidences for car manufactures to design better $\mathbf{BT} +\mathbf{AUD}$ human-machine interaction systems as a priority strategy.

By introducing autonomous or software-controlled systems, human operators are increasingly required to perform cognitive-intensive tasks in addition to existing labour-intensive tasks. As a result, it will be more difficult to identify human roles in future complex systems with traditional approaches such as hierarchical task analysis used in the conventional HRA. This paper proposes a novel systematic approach for a human reliability assessment to better understand human activities in complex systems. The proposed framework is a hybrid method combining the System Theoretic Process Analysis (STPA) and the Success Likelihood Index Method (SLIM) to assess the system reliability. The STPA is adopted to analyse the interaction relationship between different system components. The primary purpose of STPA is to find and analyse human activities that affect the risk in human-machine interaction systems. Then the identified human activities are evaluated and quantified by the SLIM as a probability of human error. The system reliability block diagram represents the derived human error probabilities to assess the entire system for a probabilistic risk assessment. Furthermore, the study proposed system alternations by comparing three different system configurations. Results demonstrate the importance of human performance in a complex system where humans, machines, and software interact.

3D hybrid porous Mxene-sponge network and its application in piezoresistive sensor