Underwater Unsupervised Stereo Matching Method Based on Semantic Attention

Outlier detection and disparity refinement in stereo matching

Warping-based spectral translation network for unsupervised cross-spectral stereo matching

To achieve real-time obstacle avoidance for underwater robots, the artificial potential field method can be selected as the obstacle avoidance path planning algorithm. However, this method applies a large attractive force to the underwater robot when it is far from the target point, causing it to collide with obstacles. Additionally, when the underwater robot just enters the repulsion influence zone, the repulsion force that it experiences undergoes an abrupt change, resulting in a non-smooth obstacle avoidance path. Furthermore, developing the obstacle avoidance function based on software platforms such as MATLAB has a long algorithm verification cycle, and it is difficult to transfer the algorithm model to practical scenarios. Therefore, this paper proposes an underwater robot obstacle avoidance method based on ROS. Firstly, a dynamic adjustment factor is introduced into the potential field function of the artificial potential field method to redefine the obstacle influence zone, and an improved artificial potential field method is designed as the obstacle avoidance path planning algorithm for underwater robots. The experiment is conducted on the ROS simulation platform, and the results show that the improved algorithm can solve the two problems existing in the artificial potential field method. Secondly, nodes required for underwater robot obstacle avoidance are established based on the ROS operating system, and the improved artificial potential field method is integrated into the obstacle avoidance path planning node. Finally, the obstacle avoidance experiment of the Bluerov2 underwater robot is conducted in a pool, and the experimental results demonstrate that the proposed method can achieve real-time obstacle avoidance for underwater robots.

In this paper, we propose a hybrid approach for accurate depth acquisition by using a structured light-based method with a stereo matching. By projecting additional light patterns onto a scene, a structured light-based method works well on a textureless region where a stereo matching shows poor performance. In contrast, the patterns projected onto a rich textured region obstruct in estimating reliable depth information in the structured light-based method, while a stereo matching excels. We exploit these complementary characteristics by combining the results from both methods that outperform the one from either alone. In our fusion framework, a hybrid stereo matching is introduced, in which the disparity search range for each pixel is limited based on the initial depth map obtained by the structured light-based method. In addition, we introduce a confidence-based fusion method which combines the depth maps, while incorporating the advantages of each method. The experimental results show that the proposed method achieves to estimate accurate depth information, while other methods fail.

Time-of-Flight (ToF) sensors and stereo vision systems are both widely used for capturing depth data. They have some complementary strengths and limitations, which have been exploited in prior research to produce more accurate depth maps by fusing data from the two sources. However, among these diverse data fusion approaches, none of them provides an end-to-end neural network solution. In this work, we propose the first end-to-end ToF and stereo data fusion network using the coarse-to-fine matching framework, where the prior of ToF depth is integrated into the stereo matching process by constraining the search range of stereo matching within an interval around the ToF camera depth measurement. We adopt a dynamic search range for each pixel according to an estimated ToF error map, which is more efficient and effective than a constant one when handling various errors. The ToF error map is estimated by the ToF error estimator branching out from the stereo matching network. Both ToF error estimation and stereo matching are performed in a joint framework, with the two tasks assisting each other mutually. We also propose an upsampling module to replace the naive bilinear upsampling in the coarse-to-fine stereo matching network, which reduces the error caused by the upsampling. The proposed deep network is trained end-to-end on synthetic datasets and generalizable to real-world datasets without further fine-tuning. Experimental results show that our fusion method achieves higher accuracy than either ToF or stereo alone, and outperforms state-of-the-art fusion methods on both synthetic and real data.

Underwater robots conducting inspections require autonomous obstacle avoidance capabilities to ensure safe operations. Training methods based on reinforcement learning (RL) can effectively develop autonomous obstacle avoidance strategies for underwater robots; however, training in real environments carries significant risks and can easily result in robot damage. This paper proposes a Sim-to-Real pipeline for RL-based training of autonomous obstacle avoidance in underwater robots, addressing the challenges associated with training and deploying RL methods for obstacle avoidance in this context. We establish a simulation model and environment for underwater robot training based on the mathematical model of the robot, comprehensively reducing the gap between simulation and reality in terms of system inputs, modeling, and outputs. Experimental results demonstrate that our high-fidelity simulation system effectively facilitates the training of autonomous obstacle avoidance algorithms, achieving a 94% success rate in obstacle avoidance and collision-free operation exceeding 5000 steps in virtual environments. Directly transferring the trained strategy to a real robot successfully performed obstacle avoidance experiments in a pool, validating the effectiveness of our method for autonomous strategy training and sim-to-real transfer in underwater robots.

Learning and self-adaptation ability is highly required to be integrated in path planning algorithm for underwater robot during navigation through an unspecified underwater environment. High frequency oscillations during underwater motion are responsible for nonlinearities in dynamic behavior of underwater robot as well as uncertainties in hydrodynamic coefficients. Reactive behaviors of underwater robot are designed considering the position and orientation of both target and nearest obstacle from robot's current position. Human like reasoning power and approximation based learning skill of neural based adaptive fuzzy inference system (ANFIS) has been found to be effective for underwater multivariable motion control. More than one ANFIS models are used here for achieving goal and obstacle avoidance while avoiding local minima situation in both horizontal and vertical plane of three dimensional workspace. An error gradient approach based on input-output training patterns for learning purpose has been promoted to spawn trajectory of underwater robot optimizing path length as well as time taken. The simulation and experimental results endorse sturdiness and viability of the proposed method in comparison with other navigational methodologies to negotiate with hectic conditions during motion of underwater mobile robot.

A novel method for simultaneously acquiring and registering range data of a real object from different viewpoints is presented. Currently, most 3D model reconstruction techniques do not cooperate with the existing range data for shape recovery of future viewpoints. In this work, a stereo vision system is developed for 3D model acquisition. To reduce the computation and increase the accuracy of stereo matching algorithms, the recovered range data from previous viewpoints are registered and then used to provide additional constraints for 3D acquisition of the next viewpoint. Experiments have shown that our approach gives better performance on both execution time and stereo matching results.

Binocular stereo vision is an important branch of the research area in computer vision. Stereo matching is the most important process in binocular vision. In this paper, a new stereo matching scheme using shape-based matching (SBM) is presented to improve the depth reconstruction method of binocular stereo vision systems. The method works in two steps. First, an operator registers the pattern including the key features of an object to be measured. Then during the operation stage, the stereo camera snaps stereo images and finds the patterns in right and left images separately by means of the SBM. The 3D positions of the object are calculated by using the corresponding points of the stereo images and the projection matrices of the stereo camera. Since we apply robust image processing algorithms, such as the SBM, the proposed method becomes more reliable than the conventional stereo vision systems.

Underwater robots are an important tool for exploiting marine resources, while underwater target detection is a key technology for marine fishing by underwater robots. For the problem that the traditional algorithm is difficult to detect small targets and multiple scales in the underwater environment, this paper proposes an underwater biological detection algorithm based on the multi-scale feature fusion and attention mechanism. First, a YOLOv5 network is constructed, and a bi-directionally weighted BiFPN feature pyramid structure is fed into the neck network so that multiscale target features can be efficiently merged. Second, a channel and spatial convolutional attention mechanism is introduced between the neck network and the prediction network in order to increase the saliency of the underwater target that needs to be detected, which effectively reduces the missed and erroneous detection of underwater targets. The experimental results show that the proposed algorithm can accurately detect multi-scale underwater targets in complex underwater environments with an average accuracy of 80.18%, which is 2.87% better than YOLOv5, and an average recall of 85.06%, which is 4.81% better than YOLOv5.

Aiming at multi robot cooperation application requirements of our small-scaled underwater spherical robots, a cooperative formation control system with static and dynamic target rounding up was proposed and studied in this paper. Considering the complex underwater environment and kinematic modeling of the robot, a simplified kinematic model of underwater spherical robot with horizontal and vertical motion was established. Given the environmental disturbances in practical underwater scenarios, an adaptive control algorithm was designed to control the underwater spherical robot, which has good robustness and adaptability. To handle the application requirement of static and dynamic target rounding up with multi robots, the path planning strategy of multi robot was proposed, the modeling of static and dynamic target rounding up was analyzed and optimized, and a controller based on linear quadratic regulator method was also designed to realize static/dynamic target rounding up with multi robots. In addition, underwater target rounding up experiments with two or three spherical robots were conducted to test the feasibility and performance of the proposed formation control system, and the experimental results confirmed the validity of the proposed system. This study may inspire future underwater multi robot cooperation that have properties of high efficiency, wide range and multi tasks and has low environmental interference of aquatic environments.

Stereo matching is the key problem in many stereo vison based 3D applications. One of the factors make local stereo matching time-consuming is that every pixel has the same disparity range as the pre-set one, which should be larger than all possible disparities. An improper pre-set disparity range may lead to redundant computation for some pixels and inadequate computation for some others. In this paper, we propose an accurate and fast local stereo matching method, which employs fine disparity estimation and adaptive cost aggregation. The main contributions of our work include two parts. Firstly, we use phase-based correlation to estimate an initial disparity range for block center pixels. Secondly, we estimate a more limited disparity range for every in-block pixel according to its support weight to the block center pixel. Our disparity estimation and block matching techniques can not only reduce the disparity searching range for every pixel, but also can eliminate some pseudo match pairs. Four standard Middlebury stereo image pairs are tested to evaluate the performance of the proposed algorithm. Experimental results show that the proposed algorithm can reduce the matching time by 37.4% on average with relatively higher accuracy compared to traditional method.

Miniature underwater robots play a crucial role in the exploration and development of marine resources, particularly in confined spaces and high-pressure deep-sea environments. This study presents the design, optimization, and performance of a miniature robotic fish, powered by the oscillation of bio-inspired fins. These fins feature a rigid-flexible hybrid structure and use an eccentric rotating mass (ERM) vibration motor as the excitation source to generate high-frequency unidirectional oscillations that induce acoustic streaming for propulsion. The drive mechanism, powered by miniature ERM vibration motors, eliminates the need for complex mechanical drive systems, enabling complete isolation of the entire drive system from the external environment and facilitating the miniaturization of the robotic fish. A compact, untethered robotic fish, measuring 85 × 60 × 45 mm3;, is equipped with three bio-inspired fins located at the pectoral and caudal positions. Experimental results demonstrate that the robotic fish achieves a maximum forward swimming speed of 1.36 body lengths (BL) per second powered by all fins and minimum turning radius of 0.6 BL when powered by a single fin. In addition, the robotic fish is able to swim upstream in turbulent flow, and its autonomous version can navigate complex, obstacle-filled environments. These results underscore the significance of employing the ERM vibration motor in advancing the development of highly maneuverable, miniature untethered underwater robots for various marine exploration tasks.

This paper investigates a cross-scale space semantic feature coherent image inpainting approach since it is challenging for the existing image inpainting methods to fuse the semantic feature information effectively. Firstly, we learn the feature semantic relevance step-by-step from the high-level semantic feature map’s attention mechanism and then we apply what we have learned to the preceding low-level feature map. In order to preserve the visual and semantic coherence of image repair, the missing content can be filled by changing attention from deep to shallow in a multiscale manner. A broader receptive field is generated by partial convolution, and semantic feature relevance is achieved using a multiscale cross feature space feature attention mechanism based on semantic attention. This technique improves the extensibility and continuity of the restored images by reconstructing the semantic information of different feature spaces, not only taking into account the reuse of existing semantic space features but also including across feature spaces. The experimental results demonstrated an improvement in PSNR, SSIM, and L1 performance by 10.50%, 0.13%, and 47.09%, respectively, with clear benefits.

Research on binocular stereo vision phenotyping measurement for leafy vegetable based on 3DGS supervision