Grasping Unstructured Objects with Full Convolutional Network in Clutter

The inherent compliance of continuum robots holds great promise in the fields of soft manipulation and safe human–robot interaction. This compliance reduces the risk of damage to the manipulated object and its surroundings. However, continuum robots possess theoretically infinite degrees of freedom, and this high flexibility usually leads to complex deformations when subjected to external forces and positional constraints. Describing these complex deformations is the main challenge in modeling continuum robots. In this study, we investigated a novel variable curvature modeling method for continuum robots, considering external forces and positional constraints. The robot configuration curve is described using the developed mechanical model, and then the robot is fitted to the curve. A ten-section continuum robot prototype with a length of 1 m was developed in order to validate the model. The feasibility and accuracy of the model were verified by the ability of the robot to reach target points and track complex trajectories with a load. This work was able to serve as a new perspective for the design analysis and motion control of continuum robots.

Continuum robots play the role of end effectors in various surgical robots and endoscopic devices. While soft continuum robots (SCRs) have proven advantages such as safety and compliance, more research and development are required to enhance their capability for specific medical scenarios. This research aims at designing a soft robot, considering the concepts of geometric and kinematic similarities. The chosen application is a semi-invasive medical application known as transesophageal echocardiography (TEE). The feasibility of fabrication of a soft endoscopic device derived from the Chorda dorsalis paragon was shown empirically by producing a three-segment pneumatic SCR. The main novelties include bioinspired design, modeling, and a navigation control strategy presented as a novel algorithm to maintain a kinematic similarity between the soft robot and the rigid counterpart. The kinematic model was derived based on the method of transformation matrices, and an algorithm based on a self-organizing map (SOM) network was developed and applied to realize kinematic similarity. The simulation results indicate that the control method forces the soft robot tip to follow the path of the rigid probe within the prescribed distance error (5 mm). The solution provides a soft robot that can surrogate and succeed the traditional rigid counterpart owing to size, workspace, and kinematics.

Network traffic flow can be represented as time series. Outliers of these time series is important for the computer network security. The existing methods for the time series anomaly detection mainly operate on the entire time series to detect the abnormal series, seldom find out the position of the abnormal point. To solve this defect, we propose a method to discover the location of the outliers in time series based on a full convolutional network. We first map the time series into two-dimensional grayscale images by sliding window approach, and then feed them into the full convolutional network. In the classified stage, the feature maps of the last convolution layer are restored to the same size as the input image by up-sampling. Finally, we predict the class for each pixel and calculate the loss on a pixel-by-pixel basis. We tested the model on Yahoo Webscope S5 anomaly benchmark dataset, and the results show that our proposed method can achieve a mean pixel accuracy of 99.4% and mean IOU of 97% on the test dataset.

In the field of robots, soft continuum robots have strong adaptability, and rigid continuum robots have large carrying capacity, but they all have some limitations. The purpose of this paper is to solve the problems of small carrying capacity of soft continuum robots and low safety of rigid continuum robots. Based on the combined action of particle jamming and fiber jamming, we use the good fluidity of spherical particles and the rigidity of spherical particles to realize the flexibility and rigidity of the robot, and put forward the requirement of flexible contact and rigid bearing. According to the large range variable stiffness actual demand, a self-locking soft continuum robot with large range variable stiffness based on the jamming phenomenon is proposed, and the principle of large range variable stiffness is explained. Then the model experiment prototype is designed and manufactured. Based on the model, the combination elastomer is defined. The basic unit is constructed and the theoretical analysis is carried out, and the equivalent of the combination elastomer stiffness model is obtained. Finally, an experimental test system is built. Through the experiment and simulation of variable stiffness, the stiffness change of combination elastomer is obtained, and the specific situation of bearing capacity change is recorded. This study provides a new method to achieve a wide range of stiffness changes, and builds a bridge from soft continuum robots to rigid continuum robots.

Optical pressure sensors based plantar image segmenting using an improved fully convolutional network

Current challenges in soft robotics include sensing and state awareness. Modern soft robotic systems require many more sensors than traditional robots to estimate pose and contact forces. Existing soft sensors include resistive, conductive, optical, and capacitive sensing, with each sensor requiring electronic circuitry and connection to a dedicated line to a data acquisition system, creating a rapidly increasing burden as the number of sensors increases. We demonstrate a network of fiber-based displacement sensors to measure robot state (bend, twist, elongation) and two microfluidic pressure sensors to measure overall and local pressures. These passive sensors transmit information from a soft robot to a nearby display assembly, where a digital camera records displacement and pressure data. We present a configuration in which one camera tracks 11 sensors consisting of nine fiber-based displacement sensors and two microfluidic pressure sensors, eliminating the need for an array of electronic sensors throughout the robot. Finally, we present a Cephalopod-chromatophore-inspired color cell pressure sensor. While these techniques can be used in a variety of soft robot devices, we present fiber and fluid sensing on an elastomeric finger. These techniques are widely suitable for state estimation in the soft robotics field and will allow future progress toward robust, low-cost, real-time control of soft robots. This increased state awareness is necessary for robots to interact with humans, potentially the greatest benefit of the emerging soft robotics field.

Inspired by the physiological structure of the hand capable of realizing the continuous change in finger stiffness when grasping objects of different masses, a self-locking soft continuum robot with a large variable-stiffness range based on particle jamming and fibre jamming is proposed in this paper to meet the requirements of it in practical application. In this paper, a variable stiffness range is derived due to the good fluidity and rigidity of the spherical particles and the low elasticity and high toughness of the fibres. Then, an analysis model is established to deduce its self-locking condition, and the deflection angle of self-locking under the influence of external force is about 0.17 rad. The maximum stiffness of the experimental prototype can reach 1223.58 N m−1 due to the limitation of the experimental materials, despite the fact that the theoretical stiffness can be increased infinitely after self-locking. To explain the adaptability of the robot, the adaptive conditions of the soft continuum robot with variable stiffness are deduced. A new evaluation index, the adaptive intensity of the soft continuum robot, is introduced and the adaptability experiments are carried out. In adaptability experiments, the maximum bending angle of the continuum robot reaches 108°. Finally, the adaptability of the soft continuum robot to different geometries is discussed.

As a function of the spatial position of the optical coherence tomography (OCT) image, retinal layer thickness is an important diagnostic indicator for many retinal diseases. Reliable segmentation of the retinal layer is necessary for extracting useful clinical information. However, manual segmentation of these layers is time-consuming and prone to bias. Furthermore, due to speckle noise, low image contrast, retinal detachment, and also irregular morphological features make the automatic segmentation task challenging. To alleviate these challenges, in this paper, we propose a new coarse-fine framework combining the full convolutional network (FCN) with a multiphase level set (named FCN-MLS) for automatic segmentation of nine boundaries in retinal spectral OCT images. In the coarse stage, FCN is used to learn the characteristics of specific retinal layer boundaries and achieve classification of four retinal layers. The boundaries are then extracted and the remaining boundaries are initialized based on a priori information about the thickness of the retinal layer. In order to prevent the overlapping of the segmentation interfaces, a regional restriction technique is used in the multi-phase level to evolve the boundaries to achieve fine nine retinal layers segmentation. Experimental results on 1280 B-scans show that the proposed method can segment nine retinal boundaries accurately. Compared with the manual delineation, the overall mean absolute boundary location difference and the overall mean absolute thickness difference were 5.88 ± 2.38μm and 5.81 ± 2.19μm, which showed a good consistency with manual segmentation by the physicians. Our experimental results also outperform state-of-the-art methods.

Soft robots are increasingly being explored and developed in various settings that demand safe and adaptable interactions between robots and their environments. In addition, soft robots exhibit passive compliant behavior and generate continuous deformations when engaging with the environment. This imposes challenges on achieving active, on-demand interaction force control, especially when feedback force-sensing devices are not available. Consequently, there is a need to explore new model-based force control paradigms for soft robots. In this article, we propose a (quasi-)static force control approach for soft robots based on compliance modeling, avoiding the necessity for feedback control loops or extensive training data collection. The proposed approach can deliver contact force control along three Cartesian axes when the robot is actuated into various configurations. The compliance matrix is derived from the robot configuration, which allows the calculation of desired deflection displacements needed to generate on-demand forces. The resulting force control is achieved by solving inverse kinematics problems based on these deflection displacements. The efficacy of our proposed controller is validated through experiments with both one- and two-segment pneumatic-driven soft continuum robots. The results demonstrate effective static force control performance, with mean control errors below 5% of the desired peak forces.

Extraterrestrial exploration presents unique challenges for robotic systems, where traditional rigid rovers face limitations in stowage volume, traction on unpredictable terrain, and susceptibility to damage. Soft robotics offers promising solutions through bio-inspired designs that can mimic natural locomotion mechanisms. We present an optimised soft spider-inspired jumping robot for extraterrestrial exploration that addresses key challenges in soft robotics: actuation efficiency, controllability, and deployment. Drawing inspiration from spider physiology, particularly their hydraulic extension mechanism, we develop a lightweight limb capable of multi-modal behaviour with significantly reduced energy requirements. Our 3D-printed soft actuator leverages pressure-driven collapse for efficient retraction and pressure-enhanced rapid extension, achieving a power-to-weight ratio of 249 W/kg. The integration of a non-backdriveable clutch mechanism enables the system to hold positions with zero energy expenditure—a critical feature for space applications. Experimental characterisation and a subsequent optimisation methodology across various materials, dimensions, and pressures reveals that the robot can achieve jumping heights of up to 0.92 body lengths. The collapsible nature of the soft limb enables efficient stowage during spacecraft transit, while the integrated pumping system facilitates self-deployment upon arrival. This work demonstrates how biologically-inspired design principles can be effectively applied to develop versatile robotic systems optimised for the unique constraints of extraterrestrial exploration.

Cardiovascular disease is becoming the leading cause of human mortality. In order to address this, flexible continuum robots have emerged as a promising solution for miniaturizing and automating vascular interventional equipment for diagnosing and treating cardiovascular diseases. However, existing continuum robots used for vascular intervention face challenges such as large cross-sectional sizes, inadequate driving force, and lack of navigation control, preventing them from accessing cerebral blood vessels or capillaries for medical procedures. Additionally, the complex manufacturing process and high cost of soft continuum robots hinder their widespread clinical application. In this study, we propose a thermally drawn-based microtubule soft continuum robot that overcomes these limitations. The proposed robot has cross-sectional dimensions several orders of magnitude smaller than the smallest commercially available conduits, and it can be manufactured without any length restrictions. By utilizing a driving strategy based on liquid kinetic energy advancement and external magnetic field for steering, the robot can easily navigate within blood vessels and accurately reach the site of the lesion. This innovation holds the potential to achieve controlled navigation of the robot throughout the entire blood vessel, enabling in situ diagnosis and treatment of cardiovascular diseases.

Continuum robots and soft robots have shown great potential in industrial and medical applications. Sensing the shapes of continuum robots is a challenging but significant problem for enhancing their performance during various tasks. In this letter, we present a novel method to estimate the shapes of continuum robots by capturing passive tendon displacements with image sensors. This method is reliable but low-cost. To reconstruct the shape of the continuum robot, an intuitive way based on screw theory and the material frame is also presented. The displacements of passive tendons are related to the integral of the geometric parameters of the continuum robot. By using passive tendon displacements, not only bending but also twisting of the robot can be calculated. The experiments show that our method can estimate the shapes of the robot deformed in different conditions. The mean distal position error is 3.01% of the length and the mean shape error is 1.86% of the length. The mean distal direction error is 4.1 degrees. Besides, sensing the shapes of continuum robots in real-time is also achieved.

Earthworms have entirely soft bodies mainly composed of circular and longitudinal muscle bundles but can handle the complexity of unstructured environments with exceptional multifunctionality. Soft robots are naturally appropriate for mimicking soft animal structures thanks to their inherent compliance. Here, we explore the new possibility of using this compliance to coordinate the actuation movements of single-type soft actuators for not only high adaptability but the simultaneous multifunctionality of soft robots. A cross-linked actuator coordination mechanism is proposed and explained with a novel conceptual design of a cross-linked network, characterization of modular coordinated kinematics, and a modular control strategy for multiple functions. We model and analyze the motion patterns for these functions, including grabbing, manipulation, and locomotion. This further enables the combination of simultaneous multi-functions with this very simple actuator network structure. In this way, a soft modular robot is developed with demonstrations of a novel continuous-transportation mode, for which multiple objects could be simultaneously transported in unstructured environments with either mobile manipulation or pick-and-place operation. A comprehensive workflow is presented to elaborate the cross-linked actuator coordination concept, analytical modeling, modular control strategy, experimental validation, and multi-functional applications. Our understanding of actuator coordination inspires new soft robotic designs for wider robotic applications.

Hyperspectral change detection (CD) can be effectively performed using deep-learning networks. Although these approaches require qualified training samples, it is difficult to obtain ground-truth data in the real world. Preserving spatial information during training is difficult due to structural limitations. To solve such problems, our study proposed a novel CD method for hyperspectral images (HSIs), including sample generation and a deep-learning network, called the recurrent three-dimensional (3D) fully convolutional network (Re3FCN), which merged the advantages of a 3D fully convolutional network (FCN) and a convolutional long short-term memory (ConvLSTM). Principal component analysis (PCA) and the spectral correlation angle (SCA) were used to generate training samples with high probabilities of being changed or unchanged. The strategy assisted in training fewer samples of representative feature expression. The Re3FCN was mainly comprised of spectral–spatial and temporal modules. Particularly, a spectral–spatial module with a 3D convolutional layer extracts the spectral–spatial features from the HSIs simultaneously, whilst a temporal module with ConvLSTM records and analyzes the multi-temporal HSI change information. The study first proposed a simple and effective method to generate samples for network training. This method can be applied effectively to cases with no training samples. Re3FCN can perform end-to-end detection for binary and multiple changes. Moreover, Re3FCN can receive multi-temporal HSIs directly as input without learning the characteristics of multiple changes. Finally, the network could extract joint spectral–spatial–temporal features and it preserved the spatial structure during the learning process through the fully convolutional structure. This study was the first to use a 3D FCN and a ConvLSTM for the remote-sensing CD. To demonstrate the effectiveness of the proposed CD method, we performed binary and multi-class CD experiments. Results revealed that the Re3FCN outperformed the other conventional methods, such as change vector analysis, iteratively reweighted multivariate alteration detection, PCA-SCA, FCN, and the combination of 2D convolutional layers-fully connected LSTM.

Soft continuum robots are more and more used in various applications due to the multiple advantages offered by the properties of hyper-elastic soft materials, such as resilience, flexibility, etc. Despite all these essential properties that make the soft robots good candidates for some real-life needs (form enclosure grasping, obstacle-free navigation, etc.), controlling their shape remains challenging because of their hyper-redundancies. This letter investigates an inverse dynamics model-based shape control of soft continuum robots in the presence and absence of external efforts. To that end, modeling based on Pythagorean Hodograph (PH) curves with prescribed lengths is combined with Euler-Bernoulli modeling to reconstruct the robot shapes and then calculate the actuator inputs in the case of external efforts. Thus, a relationship between the actuator inputs and the PH control points is obtained. With the above strategy, the actuator inputs can be computed accordingly, and therefore, real-time shape control of the soft robots for various tasks becomes possible. The results of the proposed approach are validated both numerically and experimentally using two classes of soft continuum robots - a Fluidic Elastomeric Actuator (FEA), describing a 2D soft finger robot, with a single and multiple phalanges, and a 3D continuum manipulator, namely the Compact Bionic Handling Assistant (CBHA).