Adaptive tactile control for in-hand manipulation tasks of deformable objects

A tactile sensor is a device that can measure a given property of an object or contact event through physical contact between the sensor and the object. Traditionally, tactile sensors have been developed using measurements of strain produced in sensing materials that are detected using physical quantities such as electric resistance and capacity, magnetic intensity, voltage and light intensity (Nicholls, 1990). Research on tactile sensor is basically motivated by the tactile sensing system of the human skin. In humans, the skin’s structure provides a mechanism to simultaneously sense static and dynamic pressure with extremely high accuracy. Meanwhile in robotics, several tactile sensing principles are commonly used nowadays, such as capacitive, piezoelectrical, inductive, piezoresistive, and optoelectrical sensors (Schmidt et al., 2006, Lee & Nicholls, 1999). In our research lab, with the purpose to establish object manipulation ability in robotic fingers, we developed a hemispherical shaped optical three-axis tactile sensor capable of acquiring normal and shearing forces to mount on the fingertips of robot fingers. This tactile sensor uses an optical waveguide transduction method and applies image processing techniques. Such a sensing principle is expected to provide better sensing accuracy to realize contact phenomena by acquiring the three axial directions of the forces, so that normal and shearing forces can be measured simultaneously. This tactile sensor is designed in a hemispherical dome shape that consists of an array of sensing elements. This shape is to mimics the structure of human fingertips for easy compliance with various shapes of objects. For miniaturization of the tactile sensor, measurement devices are placed outside the sensor. The small size of the sensor makes it easy for installation at robotic fingers. The optical three-axis tactile sensor developed in this research is designed in hemispherical shape, and the sensing elements are distributed in 41-sub region. Due to this structure, the acquired images by CCD camera, except for sensing element at the sensor tip area, are not the actual image of contact pressure at the sensing elements. Therefore, to compensate with the sensor structure, it is necessary to conduct coordinate transformation calculations for each sensing element except for the element at the sensor tip area. In this chapter, we

Review on aspects of artificial tactile feedback in laparoscopic surgery

An edge detection algorithm for use with tactile sensors is presented in this paper. The algorithm is based on the physical properties of the tactile sensor and tactile data. In addition, the algorithm is computationally efficient, and is thus suitable for real-time data processing. Experimental results from the application of this algorithm to a Lord LTS-210 tactile array sensor are presented. Further, observations about the use of a tactile sensor in a system are discussed. The proposed algorithm is a part of a real-time controller, implemented on CMU DD Arm II, that uses a tactile sensor in the feedback loop.

Drawing inspiration from the human vision-touch interaction that demonstrates the ability of vision in assisting tactile manipulation tasks, this paper addresses the issue of 3D object recognition from tactile data whose acquisition is guided by visual information. An improved computational visual attention model is initially applied on images collected from multiple viewpoints over the surface of an object to identify regions that attract visual attention. Information about color, intensity, orientation, symmetry, curvature, contrast and entropy are advantageously combined for this purpose. Interest points are then extracted from these regions of interest using an innovative technique that takes into consideration the best viewpoint of the object. As the movement and positioning of the tactile sensor to probe the object surface at the identified interest points take generally a long time, the local data acquisition is first simulated to choose the most promising approach to interpret it. To enable the object recognition, the tactile images are analyzed with the help of various classifiers. A method based on similarity is employed to select the best candidate tactile images to train the classifier. Among the tested algorithms, the best performance is achieved using the k-nearest neighbor classifier both for simulated data (87.89% for 4 objects and 75.82% for 6 objects) and real tactile sensor data (72.25% for 4 objects and 70.47% for 6 objects).

In-hand recognition and manipulation of elastic objects using a servo-tactile control strategy

Electrical impedance tomography (EIT)-based tactile sensors offer cost-effective and scalable solutions for robotic sensing, especially promising for soft robots. However, a major issue of EIT-based tactile sensors when applied to highly deformable objects is their performance degradation due to surface deformations. This limitation stems from their inherent sensitivity to strain, which is particularly exacerbated in soft bodies, thus requiring dedicated data interpretation to disentangle the parameter being measured and the signal deriving from shape changes. This has largely limited their practical implementations. This article presents a machine learning-assisted tactile sensing approach to address this challenge by tracking surface deformations and segregating this contribution in the signal readout during tactile sensing. We first capture the deformations of the target object, followed by tactile reconstruction using a deep learning model specifically designed to process and fuse EIT data and deformation information. Validations using numerical simulations achieved high correlation coefficients (0.9660–0.9999), peak signal-to-noise ratios (SNRs) (28.7221–55.5264 dB), and low relative image errors (RIEs) (0.0107–0.0805). Experimental validations, using a hydrogel-based EIT e-skin under various deformation scenarios, further demonstrated the effectiveness of the proposed approach in real-world settings. The findings could underpin enhanced tactile interaction in soft and highly deformable robotic applications.

We have presented the newly developed anthropomorphic robot hand named the KH Hand type S and its master slave system using the bilateral controller. The use of an elastic body has improved the robot hand in terms of weight, the backlash of the transmission, and friction between the gears. We have demonstrated the expression of the Japanese finger alphabet. We have also shown an experiment of a peg-in-hole task controlled by the bilateral controller. These results indicate that the KH Hand type S has a higher potential than previous robot hands in performing not only hand shape display tasks but also in grasping and manipulating objects in a manner like that of the human hand. In our future work, we are planning to study dexterous grasping and manipulation by the robot.

Enormous benefits of using 4D printing technology referring to added dimension time combining 3D printing technology with high level smart material shape changing, engineering software has currently enhanced its applicability in industry. Triggered by external stimuli 4D printed parts are pleated & unpleated by adding geometric coding & its pressure and resistance is measured by mechanically compliant soft tactile sensor by the physical interaction with environmental dynamics & ambiguity using piezo resistive technique for its electrical resistance distinct electric potential.This paper designs Mechanical walker 4D printed which will be integral part of robotic system helpful for path tracking in automated guided vehicle for automate route system with the use of tactile sensors utilizing haptic touch module.This walker will incorporate the safety intelligence using sensing stimulus by NiTinol providing close proximity interaction procedure.

Summary. Autonomous navigation in walking robots requires that three main tasks be solved: self-localization, obstacle avoidance, and object handling. This report presents a development and application of an optical three-axis tactile sensor mounted on a robotic finger to perform object handling in a humanoid robot navigation system. Previously in this research, we proposed a basic humanoid robot navigation system called the groping locomotion method for a 21-dof humanoid robot, which is capable of defining self-localisation and obstacle avoidance. Recently, with the aim to determining physical properties and events through contact during object handling, we have been developing a novel optical three-axis tactile sensor capable of acquiring normal and shearing force. The tactile sensor system is combined with 3-dof robot finger system where the tactile sensor in mounted on the fingertip. Experiments were conducted using soft, hard, and spherical objects to evaluate the sensors performance. Experimental results reveal that the proposed optical three-axis tactile sensor system is capable of recognizing contact events and has the potential for application to humanoid robot hands for object handling purposes.

Robot grasping and manipulation relies mainly on two types of sensory data: vision and tactile sensing. Localisation and recognition of the object is typically done through vision alone, while tactile sensors are commonly used for grasp control. Vision performs reliably in uncluttered environments, but its performance may deteriorate when the object is occluded, which is often the case during a manipulation task, when the object is in-hand and the robot fingers stand between the camera and the object. This paper presents a method to use the robot's sense of touch to refine the knowledge of a manipulated object's pose from an initial estimate provided by vision. The objective is to find a transformation on the object's location that is coherent with the current proprioceptive and tactile sensory data. The method was tested with different object geometries and proposes applications where this method can be used to improve the overall performance of a robotic system. Experimental results show an improvement of around 70% on the estimate of the object's location when compared to using only vision.

Estimation of the longitudinal axis of line symmetrical soft bodies by stereophotogrammetry

A preliminary study of a new method of determining interface pressures in contact zones between hard and soft elastic bodies from measurements of strain away from the contact zone is carried out using a single beam with a known load on its topside and strain gauges spaced out along its underside. The beam is such that it can be modelled by simple beam theory. To reconstruct the load from the strain data a regularization scheme of projection type based on the non-negativeness of the pressure distribution is developed. The distribution is represented as a series of basis functions, here either Bernstein polynomials or quadratic B-splines, the coefficients in the series being determined by solving a constrained optimization problem. This method of reconstruction is applied to known theoretically simulated loads with random noise added and to experimental loads, and ways of determining good values of the regularization parameter, the number of basis functions, given. The reconstructions obtained are very satisfactory and give confidence in the application of the method to unknown loads.

In this paper, we present a concept of using tactile sensors as sufficient tools in localizing, recognizing an object in robotic in-hand manipulation tasks. Our approach operates on a moderately intensive array data that are obtained from a tactile sensor when a robotic gripper grasps an object that is small relatively to fingers. In stead of using tactile data as an array of discrete numbers, we treat it as a grayscale image. By working with successive images from tactile sensor exploiting image processing tools, we are able to extract rich information about the contact condition between an object and the gripper. Experimental results show that from the processed data, we can realize the grasped object's position/orientation, contact shape, especially the stick-slip condition on the contact surface that is derived for the first time for this sensor. Based on localized displacement phenomenon of a sliding soft object, we proposed an approach to detect the slippage quantitatively, and conducted a wide range of experiment in term of characteristics of object's movement. We also conducted a model for an object-grasping gripper with tactile feedback in various postures of the object, and a corresponding experiment setup to validate computed results. Presented result is expected to be used widely in the community due to its simplicity and reliability.

The tactile sensor design and data measurements have been playing an important role in surface material recognition. In inevitable unstructured environments, the performance of the material classification is bottlenecked by multimodal deficiency or data collection intermittency. This article investigates the influence of surface tactile data quantity on material classification in unstructured environments. We extracted tactile data features according to the material-surface-texture model and proposed an approach to determine the scope of the time window of tactile data. We utilized the machine learning classifiers to verify the effectiveness of the hand-designed features and the determined time window. Based on the combination of the Mel frequency cepstrum coefficient and statistics, the majority of the algorithms can classify the tactile data of the upper limit (60 ms in this article) with an accuracy of at least 85%. Exploiting tactile data of the lower limit (50 ms in this article), linear discriminant analysis, quadratic discriminant analysis, and support vector machine can achieve about 93% classification accuracy, and area under curve is about 0.99. The accuracy is not significantly improved with the time window beyond the upper limit, while the performance is degraded and unstable with it beneath the lower limit.

In this paper, an improved wedge damper model is presented, based on which the effects of wedge dampers on the forced response of frictionally constrained blades are investigated. In the analysis, while the blade is modeled as a constrained structure, the damper is considered as an unconstrained structure. The model of the damper includes six rigid body modes and several elastic modes, the number of which depends on the excitation frequency. In other words, the motion of the damper is not artificially constrained. When modeling the contact surfaces of the wedge damper, discrete contact points along with contact stiffness are evenly distributed on the two contact surfaces. At each contact point, contact stiffness is determined and employed in order to take into account the effects of higher frequency modes that are omitted in the dynamic analysis. Depending on the engine rpm, quasi-static contact analysis is initially employed to determine the contact area as well as the initial preload or gap at each contact point due to the centrifugal force. A friction model is employed to determine the three-dimensional nonlinear contact forces and the relationship between the contact forces and the relative motion is utilized by the Harmonic Balance method. As the relative motion is expressed as a modal superposition, the unknown variables, and thus the resulting nonlinear algebraic equations, in the Harmonic Balance method is in proportion to the number of modes employed, and therefore the number of contact points used is irrelevant. The developed method is applied to tuned bladed disk system and the effects of normal load on the rigid body motion of the damper are investigated. It is shown that, the effect of rotational motion is significant, particularly for the in-phase vibration modes.