Towards Vertical and Horizontal Extension of Shared Control Concept

The present research aims to develop advanced driver assistance systems (ADASs) that achieve increase of driver's skill as well as increase of driver performance during using the ADAS. A haptic shared control (HSC) is a promising tool for such ADASs with skill learning function because it enables the human operator to interact and communicate continuously with the assist system through a haptic interface. We focused on the assist system by haptic shared control (HSC) for backward parking maneuvers, which are considered to be a demanding task for novice drivers. Some studies showed that backward parking assist by HSC achieves performance increase during assist control as well as increase of parking skill. However, no research shows the effect of the driver's skill and the method to use the HSC though it is thought to affect the effectiveness of the ADAS. The purpose of the present study was to investigate effect of the gain setting of our proposed parking assist system with HSC on the novice drivers' performance increase during use of the system as well as their skill increases where the drivers were instructed to drive in accordance with the haptic guidance. Driving simulator experiments of different feedback gains demonstrated that skill im-provements were achieved when a higher gain is employed.

A workload adaptive haptic shared control scheme for semi-autonomous driving

Advanced driver-assistance systems have successfully reduced drivers’ workloads and increased safety. On the other hand, the excessive use of such systems can impede the development of driving skills. However, there exist collaborative driver-assistance systems, including shared and cooperative controls, which can promote effective collaboration between an assistance system and a human operator under appropriate system settings. Given an effective collaboration setup, we address the goal of simultaneously developing or maintaining driving skills while reducing workload. As there has been a paucity of research on such systems and their methodologies, we discuss a methodology applying shared and cooperative controls by considering related concepts in the skill-training field. Reverse parking assisted by haptic shared control is presented as a means of increasing performance during assistance, while skill improvement following assistance is used to demonstrate the possibility of simultaneous achievement of driver assistance through the reduction of workload and skill improvement.

Motivation for a Neuromuscular Basis for Haptic Shared Control

Camera Holding Robotic Devices in Urology

This paper presents a haptic-based assistance system (AS) for power wheelchair users designed using the model-based shared control approach. The idea is to combine robust control with a high-level driving supervisor in order to successfully share control authority between the wheelchair user and the assistance system. This shared control strategy is composed of two parts, namely an operational part and a tactical part. Through the haptic joystick interface, this assistance system aims to reduce user’s effort when manipulating the joystick, guide the user to avoid any potential collisions, and maintain the active participation of the user in driving the wheelchair. In the operational part, an optimal Takagi-Sugeno fuzzy control approach is proposed to deal with the time-varying user’s intention represented by his desired longitudinal and angular position errors and velocities and hand torques. The control design is formulated as an LMI optimization problem which can be easily solved with numerical solvers. Two unknown input observers for Takagi-Sugeno fuzzy model have been designed to estimate the user’s intention in order to generate an assistance torque via a haptic force feedback joystick. The control supervisor in the tactical part, aims to provide a decision-making signal which allows for the conflict management based on the user hand torque estimation. A specific algorithm has been developed to solve the conflict between the user’s desired actions and the suggestions from the assistance system to ensure the user remains the final decision-maker. Experimental results show the effectiveness and the validity of the proposed assistance system.

This paper describes a set of shared control technical tools for intelligent teaming of a human operator and a robotic manipulator, and provides experimental evaluation of those tools. These technical tools can provide the human operator with an intuitive operational interface to command a multiple degree-of-freedom robotic manipulator via a general-purpose game controller and provide intelligent assistance to the human operator with human intent prediction and shared control. Two cameras with vision-feedback mounted on the robot end-effector are utilized to capture visual information and identify the location of the target object through which potential automatic control input for making adjustment to the robot motion is generated. A method to predict operator’s intent by comparing human input and automatic control input is introduced and employed to generate a probability with which the two inputs are dynamically combined to command the robot through human-machine shared control. An experimental platform consisting of a six degree-of-freedom industrial robot with a gripper and cameras is employed; results are provided with a human operator commanding the robot manipulator to execute an object inspection and handling task with the assistance of the proposed intelligent teaming algorithm.

Advanced driver assistance systems (ADASs) have been developed and have subsequently decreased opportunities for driver skill development because of degraded situational awareness and less activity in the control loop. Beginning drivers must therefore engage in training. The idea has arisen that haptic shared control, which encourages a driver to operate the vehicle properly and with correct timing, should serve as a training system for daily driving. Such a system, however, might degrade performance, comfort, or safety because training aspects might increase the driver workload as a result of error correction feedback. The possibility of lowered usability is not acceptable for ADASs. This study was conducted to develop an assist system that achieves workload reduction and skill improvement. A haptic assistance system that applies guidance torque on the steering wheel was proposed and investigated for backward parking, which is a difficult driving situation. Driving simulator experiments were conducted. Results showed that both support and after-use effects were realized. This study provides knowledge for the development of driving systems using haptic assistant systems for which initial skill influences performance.

Fully autonomous robot systems, though highly desired, face substantial theoretical and practical challenges when being deployed into a dynamic environment where humans co-exist. To tackle this challenge, this thesis investigates the concept of human-in-the-loop (HITL) systems, which incorporate human input to enhance robot functionality. HITL systems offer a pragmatic alternative, combining human versatility with robotic precision. This research aims to address critical questions in one particular HITL architecture which prioritizes the dominant role of the human within the system, positioning the robot primarily in an assistive capacity that adheres to human commands to facilitate the achievement of a shared goal. It explores two primary paradigms of shared assistance methods-Shared Control (SC) and Shared Autonomy (SA)-and discusses the system designs as well as specific algorithms to implement the three critical components in a HITL system: human intention estimation, modulation of human inputs and robot autonomy, and the human-robot communication channel. Due to the variety of use cases and their unique challenges, four distinct HITL systems are developed and analyzed to illustrate how shared assistance methods could be employed to assist human operators: an assistive wheelchair for indoor navigation, a human-centered robot system design for industrial tasks, a mobile bi-manual robot for tele-manipulation, and a virtual reality (VR)-based customizable shared control system for fine teleoperation. Although each system represents a comprehensive robotic solution, the research contributions for each work vary. In the assistive wheelchair navigation system, the focus was on human intent estimation via a low-throughput interface utilizing a recursive Bayesian filter, with significant efforts dedicated to developing a real-time user interface serving as the communication channel. In the human-robot collaboration system for industrial setting, the emphasis was on human state estimation through camera-based posture tracking and exploring the interplay between robot behavior and human ergonomics. For the two teleoperation systems, the primary focus was on the real-time modulation of human inputs and robot autonomy to aid in achieving dexterous manipulation tasks. A novel VR-based user interface was developed to enable users to customize the robot autonomy level. Each system was validated through a pilot study involving 10-20 human subjects, accompanied by extensive data analysis to provide insights into designing HITL systems for various applications. In conclusion, this thesis contributes to a deeper understanding of HITL systems, highlighting their potential to enhance human productivity, ergonomics, and quality of life in varied applications through concrete examples. The integration of human intent estimation and real-time shared control methods into robotic systems demonstrates the feasibility and benefits of HITL approaches. Our extensive experimental analysis underscores the critical role of human feedback in designing practical HITL systems that can be deployed in real-world scenarios. --Author's abstract

How operator admittance affects the response of a teleoperation system to assistive forces – A model analytic study and simulation

An overly high degree of complex automation can lead to human error in a human-machine-system. By means of a cognitive assistant system the operator of an Unmanned Aerial System (UAS) control station shall be supported in a way that reduces human error. The assistant system monitors the operator's behavior and the progress of the UAS mission. Using knowledge based on models of the operator and the mission, the assistant system can infer if problems arise and if the operator reacts appropriately. If he or she fails to do so, the assistant system intervenes adaptively to the operator's cognitive state. Early experiments confirm a significant reduction of human error.

In an unstructured environment where real-time human decision is essential, shared control allows collaboration between humans and robotic systems, combining advantages of both. However, existing control methods are challenged with precision loss, inconsistency and interference from unconscious human inputs. An adaptive anisotropic control frame is presented, enabling interaction both operational and tactical levels. Using predefined trajectory, a dynamic weight function is proposed to allow the human operator to override. Movement along preferred direction is encouraged and compensated, providing accurate real-time tracking performance. Haptic feedback during shared control is evaluated and optimised. Experiments validate that the raised method can achieve a tracking precision of under milling payload, with sensible feedback to the operator. The override manipulation can be rapidly made within 0.4s as the tactical level interaction. The proposed approach provides both stability and flexibility in interactive surgical manipulations, maintaining similar precision with autonomous execution.

A cascade steering shared controller with dual-level dynamic authority

Recently, the framework of shared control between a human operator and a robot system using a cooperative controller with human operator to decide control input to a robot system has been drawing much attention for dexterous telemanipulation. However, desiging a suitable controller for shared control scheme for the given task is difficult due to the difficulties in modeling of operator'’s behavior and environment. In this paper，we proposed a model-free approach using reinforcement learning to learn a shared control policy through interactions with the operator, robot and environment. To validate our method，we adopted a page turning task by telemanipulation and developed an experimental platform with a physical simulator．Experimental results suggest that our method is able to learn task-relevant shared control for flexible and enhanced dexterous manipulation．

This poster presents simulation of a tele-operated shared controlled robot task that is integrated with a generic simulator of RADOE (Robot Application Development and Operating Environment). A customized and extendable Rviz interface plugin is designed and applied to import models, do simulation, enable real robot operation, and communicate with other projects by clicking related buttons. In the simulation process, the robot model in the simulator is controlled and visualized by human operator using an Omega 7 haptic device and automatic method, i.e., the shared control. After the simulation is conducted and satisfied, the system will send back a signal to the real robot system to execute the operation task; otherwise, simulation of the shared control process will be continued until satisfaction. We provide a simulation of a drawing task on the surface of a sphere.