Object manipulation in 3d space by two cone-shaped finger robots based on finger-thumb opposability without object sensing

The problem of dexterous manipulation of rigid objects is discussed. A formalism suitable for representing the space of the contact forces allowing a grasped object to perform an assigned motion is presented. The advantages that can be gained using such a formalism when robust grasping planning problems are dealt with are highlighted. The question of how manipulation dexterity in performing grasping actions could be attained in cases where possible repetitions of the planned task are somehow allowed is discussed. This is done on the basis of results concerning the so-called iterative learning hybrid control theory, extended to address robotic manipulation problems. >

Quickly and accurately extracting information about a touched object’s orientation is a critical aspect of dexterous object manipulation. However, the speed and acuity of tactile edge orientation processing with respect to the fingertips as reported in previous perceptual studies appear inadequate in these respects. Here we directly establish the tactile system’s capacity to process edge-orientation information during dexterous manipulation. Participants extracted tactile information about edge orientation very quickly, using it within 200 ms of first touching the object. Participants were also strikingly accurate. With edges spanning the entire fingertip, edge-orientation resolution was better than 3° in our object manipulation task, which is several times better than reported in previous perceptual studies. Performance remained impressive even with edges as short as 2 mm, consistent with our ability to precisely manipulate very small objects. Taken together, our results radically redefine the spatial processing capacity of the tactile system.

In this thesis, new methods to grasp and to manipulate unknown objects with a robotic multi-fingered hand have been proposed. These methods have the goals to reduce the time of the execution of the grasp to be suitable with real-time applications and to exploit the redundancy in the ``multi-fingered hand + system in order to optimize other subtasks during manipulation. In particular, a visual system has been used to cope with the problem of unknown objects grasping. Then, it is straightforward to recognize that two main tasks have to be performed in such a context: object recognition/reconstruction and grasp planning. The motivation behind the proposed algorithm is that the visual reconstruction algorithm should guide the object's grasp planner in a coordinated manner. The same coordination aspects are the key-points of the proposed method about manipulation of objects: now, the fingers of the robotic hand have to cooperate in order to manipulate the object in the desired way with a good dexterity and this can be done exploiting the redundancy of the whole system.

Dexterous robotic manipulation of unknown objects can open the way to novel tasks and applications of robots in semi-structured and unstructured settings, from advanced industrial manufacturing to exploration of harsh environments. However, it is challenging for at least three reasons: the desired motion of the object might be too complex to be described analytically, precise models of the manipulated objects are not available, the controller should simultaneously ensure both a robust grasp and an effective in-hand motion. To solve these issues we propose to learn in-hand robotic manipulation tasks from human demonstrations, using Dynamical Movement Primitives (DMPs), and to reproduce them with a robust compliant controller based on the Virtual Springs Framework (VSF), that employs real-time feedback of the contact forces measured on the robot fingertips. With this solution, the generalization capabilities of DMPs can be transferred successfully to the dexterous in-hand manipulation problem: we demonstrate this by presenting real-world experiments of in-hand translation and rotation of unknown objects.

Grasping and manipulation are common robotic tasks which require to properly control the interaction forces and position of multiple robots end‐effectors. In this study, the authors focus on the problem of dexterous manipulation of rigid objects by means of a cooperative robotic system. Unlike several works, they consider the more complex case when the manipulated object is not mechanically attached to the robots' end‐effectors. In addition, they address the control problem of designing a control law with continuous and bounded input torques for cooperative robots in constrained motion. To guarantee a stable grasp and a fine manipulation a centralised control algorithm based on the Orthogonalisation principle and a dynamic sliding mode control is proposed. The control algorithm does not require the knowledge of the dynamic model of the robot manipulators and manipulated object for implementation. Experimental results are presented to show the good performance of the proposed controller. In addition, the stability analysis of the closed‐loop dynamics is developed.

Dexterous manipulation of unknown objects performed by robots equipped with mechanical hands represents a critical challenge. The difficulties arise from the absence of a precise model of the manipulated objects, unpredictable environments, and limited sensing capabilities of the mechanical hands compared to human hands. This paper introduces a data-driven approach that provides a learning-based planner for dexterous manipulation employing an Adaptive Neuro-Fuzzy Inference System (ANFIS) fed by data obtained from an analytical manipulation planner. ANFIS captures the complex relationships between inputs and optimal manipulation parameters. Moreover, during a training phase, it is able to fine-tune itself on the basis of its experiences. The proposed planner enables a robot to interact with objects of various shapes, sizes, and material properties while providing an adaptive solution for increasing robotic dexterity. The planner is validated in a real-world environment, applying an Allegro anthropomorphic robotic hand. A link to a video of the experiment is provided in the paper.

Embodied intelligent agents, which represent the future of robotics, demand precise perception and real-time decision-making capabilities to achieve natural environmental interactions. However current systems face inherent limitations in unimodal sensing and cross-modal coordination, which hinder their performance in dynamic contact-rich operations. Herein, we present a fabric-based event-driven tactile interface that features an innovative woven structure with cross-fiber electrodes. It achieves breakthroughs in sensitivity (246.3kPa-1), pressure detection (>450kPa), and waterproof robustness. This interface enables millisecond-level pressure/slip dual-mode feedback for self-adaptive grasping, thereby improving the dexterous manipulation of fragile or slippery objects. For underwater scenarios, a bio-inspired visual-tactile fusion (VTF) architecture leverages tactile perception to compensate for visual limitations, demonstrating a high accuracy of 97.7% in complex tasks, including underwater transparent object manipulation and recognition of similar objects. Event-driven tactile feedback is merged with visual semantics for decision-level optimization, thereby enhancing the autonomy and adaptation of humanoid machine intelligence. It creates an innovative closed-loop cross-modal perception-decision system that builds a direct link between environmental interaction and autonomous decision-making for intelligent agent development in open-world scenarios. The superior performance of the VTF architecture dynamic interaction tasks represents a crucial step toward robotic systems with advanced intelligence.

Detaching phenomena in the learning control of manipulation of rigid objects

Predicting instead of only reacting to the properties of objects we grasp is crucial to dexterous object manipulation. Although we normally plan our grasps according to well-learned associations, we rely on implicit sensorimotor memories when we learn to interact with novel or ambiguous objects. However, little is known about the influence of sensorimotor predictions on subsequent perception and action. Here, young and elderly subjects repeatedly lifted an object in which the center of mass (CoM) was randomly varied between trials straight upward with the aim of preventing object tilts. After each lift, subjects indicated the location of the perceived CoM and reported how heavy the object felt. Surprisingly, we found that sensorimotor torque memories eventually causing initial lifting errors had substantial effects on the perception of torques, weight, and the torque planning for the next lift. Whereas subjects tended to partly retain their previous erroneous sensorimotor memories (instead of solely relying on the previously encountered torque for the upcoming motor plan), they perceived encountered torques to be stronger when they erroneously predicted them. Additionally, we found that torque prediction errors, as well as the actual torques, made the object feel heavier. By contrast, perception did not influence upcoming motor control. There were no major differences observed between the age groups. The sensorimotor impact on torque perception can be explained by internal feedforward prediction highlighting task-relevant errors, while the partial retention and adaptation of sensorimotor torque memories is reconciled with the trial-to-trial learning rule for motor adaptation. NEW & NOTEWORTHY The current study is the first to demonstrate in an object manipulation task in uncertainty that errors in the sensorimotor prediction of torques influence the perception of both torques and weight, whereas sensorimotor torque memories are partly retained and partly adapted to planning errors. Our results provide novel insights into the predictive mechanisms underpinning the common everyday task of object manipulation and further support theories about the predictive modulation of perception established in other neuroscientific disciplines.

This paper proposes a novel object manipulation method to regulate the position and attitude of an object in the task space with dynamic stability by using a triple soft-fingered robotic hand system. In our previous works, a dynamic object grasping method without use of any external sensing, called "the Blind Grasping", has been proposed. Although stable grasping in a dynamic sense has been realized by the method, a simultaneous object position and attitude control has not yet been treated, so far. In this paper, instead of using any information of the real object position and attitude, virtual data of object position and attitude are introduced by defining a virtual frame. By using the virtual information, a control signal to regulate the virtual object position and attitude without use of any external sensing is designed. The usefulness of our proposed control method even under the existence of nonholonomic rolling constraints is illustrated through a numerical simulation result.

Humans have unrivalled abilities to perform dexterous object manipulation. This requires the sensorimotor system to quickly adapt to environmental changes and predictively counter act the external disturbances. Many studies have focused on the anticipatory control of digits with real-world experiments. However, examining manipulation using virtual reality with haptic devices expands the possibilities of investigation. In this work, participants grasped and lifted an inverted T-shaped object in a virtual reality setup. The graspable surface of the object was either constrained to a small area or unconstrained. The position of the object's center of mass changed between blocks, and the participants were asked to minimize the rotation of the object during the lift. Our results show that, consistent with the results of real-world experiments, participants gradually learn to adjust the digit positions and forces to predictively compensate for the torque due to the shifted center of mass prior to liftoff. The only major difference found was that the length of trials needed during the adaptation phase to each condition increased from 3 in real-world to 5 in virtual environment.

Describes a dexterous and robust manipulation system with a multi-fingered hand. The system is composed of a real-time pose estimation of an object manipulated by the hand, a precise manipulation using fingertip with rolling contact, motion primitives for mechanical assembly, and error-recovery strategies based on the task observation function. The system architecture and experimental results are shown.

We present a novel method for classifying and estimating the categories and poses of deformable objects, such as clothing, from a set of depth images. The framework presented here represents the recognition part of the entire pipeline of dexterous manipulation of deformable objects, which contains grasping, recognition, regrasping, placing flat, and folding. We first create an off-line simulation of the deformable objects and capture depth images from different view points as training data. Then by extracting features and applying sparse coding and dictionary learning, we build up a codebook for a set of different poses of a particular deformable object category. The whole framework contains two layers which yield a robust system that first classifies deformable objects on category level and then estimates the current pose from a group of predefined poses of a single deformable object. The system is tested on a variety of similar deformable objects and achieves a high output accuracy. By knowing the current pose of the garment, we can continue with further tasks such as regrasping and folding.

Virtual hand interactions play key roles in virtual environments. The recent addition of force feedback to virtual reality simulations has enhanced their realism, especially when dexterous manipulation of virtual objects is concerned. In the past decades, much effort has been made on virtual hand modeling from the perspectives of computer animation and human computer interaction. However, much less attention is paid on haptic modeling of flexible virtual hand. In this paper, we propose a 4-layer flexible virtual hand model for virtual hand haptic interaction. The skin layer, kinematics layer, collision detection layer and haptic layer are integrated into a sophisticated virtual hand to simulate the human hands natural anatomy in its appearance and motion, and to reflect the area contact feature of force feedback datagloves. The infrastructure and details of the flexible virtual hand model are discussed. Experimental results show that the proposed flexible virtual hand demonstrates good performance in virtual hand haptic applications.

Like their natural counterparts, soft bioinspired robots capable of actively tuning their mechanical rigidity can rapidly transition between a broad range of motor tasks-from lifting heavy loads to dexterous manipulation of delicate objects. Reversible rigidity tuning also enables soft robot actuators to reroute their internal loading and alter their mode of deformation in response to intrinsic activation. In this study, we demonstrate this principle with a three-fingered pneumatic gripper that contains "programmable" ligaments that change stiffness when activated with electrical current. The ligaments are composed of a conductive, thermoplastic elastomer composite that reversibly softens under resistive heating. Depending on which ligaments are activated, the gripper will bend inward to pick up an object, bend laterally to twist it, and bend outward to release it. All of the gripper motions are generated with a single pneumatic source of pressure. An activation-deactivation cycle can be completed within 15 s. The ability to incorporate electrically programmable ligaments in a pneumatic or hydraulic actuator has the potential to enhance versatility and reduce dependency on tubing and valves.