Hybrid approach for accurate depth acquisition with structured light and stereo camera

Real-time technology of stereo matching to generate depth map from stereo images is one of the important computer vision challenges in recent years. The depth map are used in various fields such as the robot binocular vision, 3D recognition system, 3D model reconstruction and so on. Because the initial depth estimation method cannot generate good results, most of the methods will accompany a refinement process afterwards. The aim of this paper is to improve the quality of initial depth map estimation without any depth refinement in real time via GPU. This paper proposes a real-time effective stereo matching using adaptive shape window on compute unified device architecture (CUDA). The proposed method estimate depth map by cost aggregation over shape-adaptive regions and generate the accurate initial depth map among the real-time methods by evaluating in Middlebury stereo benchmark.

Detection of fruit tree diseases in natural environments: A novel approach based on stereo camera and deep learning

Generation of high-quality depth maps using hybrid camera system for 3-D video

Traditional two-dimensional vision only has basic pixel information and no depth information, while three-dimensional vision has depth information, which has high application value in the era of intelligence. The mainstream methods for obtaining depth information of objects include binocular stereo matching and TOF (Time of flight) cameras. The depth obtained by binocular stereo matching is with high precision in high-texture objects but with low precision in low-texture and repeated-texture objects, while the precision of depth obtained by TOF is the exact opposite. By combining binocular stereo matching and TOF, high-precision depth maps can be obtained in various scenarios. This paper proposes a new method of combining binocular stereo matching and TOF technology. The credible depth information is obtained through TOF to guide the stereo matching of the binocular camera, and then the information obtained after the guided matching is fused with the TOF information. The final fusion result is a high-precision depth map. The experimental results show that the fused depth map has higher accuracy and the execution speed within a reasonable range.

In this paper, we propose a depth estimation framework for light field camera arrays. The goal of the proposed framework is to compute consistent depth information over the multiple cameras which is hardly achieved by conventional approaches based on the pairwise stereo matching. We first perform stereo matchings on adjacent image pairs using a convolutional neural network-based correspondence scoring model. Once the local disparity maps are estimated, we consolidate the disparity values to make them globally sharable over the multiple views. We finally refine the depth values in the image domain by introducing a novel image segmentation method considering edges in the image to obtain a semantic-aware global depth map. The proposed framework is evaluated on three different real world scenarios, and the experimental results validate that our proposed method produces accurate and consistent depth maps for images captured by the light field camera arrays.

A hierarchical stereo matching algorithm based on adaptive support region aggregation method

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

Outlier detection and disparity refinement in stereo matching

Stereo matching, utilized in diverse fields, poses a challenge to systems in resource-constrained environments due to the significant growth of computational load with image resolution. The challenge is crucial for the systems because fields utilizing stereo matching require short operational time for real-time applications and low power architecture. Stochastic computing (SC) is able to be a valuable approach to address the challenge by reducing the computational load by representing binary numbers with stochastic sequences, which are encoded as a probability value, and by leveraging the concept of mathematical probability. Also, it is possible for a system to be error-tolerant by utilizing the characteristics of stochastic computing. Therefore, in this paper, we propose an approach for lightweight and error-tolerant stereo matching with a hardware-implemented stochastic computing processor. To verify the feasibility and error tolerance of the proposed system, we implemented the proposed system and conducted experiments comparing depth maps with or without stochastic computing by calculating similarities. According to the experimental results, the proposed system indicated no significant differences in output depth maps and achieved an improvement in the depth maps from error-injected input images by an average of 58.95%. Therefore, we demonstrated that stereo matching with stochastic computing is feasible and error-tolerant.

A stereo vision system provides important support for underwater robots to achieve autonomous navigation, obstacle avoidance, and precise operation in complex underwater environments. This article proposes an unsupervised underwater stereo matching method based on semantic attention. By combining deep learning and semantic information, it fills the challenge of insufficient training data, enhances the intelligence level of underwater robots, and promotes the progress of underwater scientific research and marine resource development. This article proposes an underwater unsupervised stereo matching method based on semantic attention, targeting the missing training supervised dataset for underwater stereo matching. An adaptive double quadtree semantic attention model for the initial estimation of semantic disparity is designed, and an unsupervised AWLED semantic loss function is proposed, which is more robust to noise and textureless regions. Through quantitative and qualitative evaluations in the underwater stereo matching dataset, it was found that D1 all decreased by 0.222, EPE decreased by 2.57, 3px error decreased by 1.53, and the runtime decreased by 7 ms. This article obtained advanced results.

Computational complexity of the local stereo matching method is affected by disparity ranges. In case of depth estimation in sequential stereo images, high computational complexity is a problem in terms of real-time processing. In this paper, we propose a temporal correlation based stereo matching method in sequential images. Using temporal information in a sequential stereo matching method provides inaccurate disparity ranges, since the estimated depth map accuracy is gradually degraded. To preserve the depth map quality in temporal stereo matching procedure, we adopt the guided image filtering for matching cost aggregation. Since the guided image filtering has a similar structure with bilateral filter, it preserves an object boundary region even in restricted disparity search ranges. Inaccurately estimated disparity values from the temporal correlation are compensated by filtering based cost aggregation method. From the experiment results, we check that the proposed depth map acquisition method preserves the depth map quality in temporal domain stereo matching.

The stereo matching <span>process is one of the key areas that impact the stereo vision technologies which are commonly used in the application of three-dimensional reconstructions. The accuracy of the depth information used in three-dimensional reconstruction is directly proportional to the accuracy of the disparity obtained from stereo matching. The challenging issue in the stereo matching process is to determine the accurate corresponding point between the left image and right image, especially for image pairs that have different exposure such as different illumination and image pair with less texture region. In order to increase the accuracy of disparity value, a new stereo matching algorithm is proposed based on the combination of Sum of absolute different and census transform at matching cost computation. guided filter was used in the matching cost aggregation in order to remove noise and preserve the edge of the image. In the optimization step, the winner take all strategy is used to select the minimum matching cost. Finally, a median filter is applied to the initial disparity map for refinement purposes. The experimental results show that the algorithm is effective in reducing the error and improving the accuracy of the disparity map in different illumination regions, less textured regions and different environmental exposure.</span>

Large-scale outdoor 3D reconstruction on a mobile device

The area of computer vision is one of the most discussed topics amongst many scholars, and stereo matching is its most important sub fields. After the parallax map is transformed into a depth map, it can be applied to many intelligent fields. In this paper, a stereo matching algorithm based on visual sensitive information is proposed by using standard images from Middlebury dataset. Aiming at the limitation of traditional stereo matching algorithms regarding the cost window, a cost aggregation algorithm based on the dynamic window is proposed, and the disparity image is optimized by using left and right consistency detection to further reduce the error matching rate. The experimental results show that the proposed algorithm can effectively enhance the stereo matching effect of the image providing significant improvement in accuracy as compared with the classical census algorithm. The proposed model code, dataset, and experimental results are available at https://github.com/WangHewei16/Stereo-Matching.

In computer graphics, Stereo Vision has been a research hotspot for many years, and it is been widely used in many areas. Stereo Vision is a technique which utilizes computer to simulate human eye system. In order to achieve this simulation, two main problems need to be solved: camera calibration and stereo matching. We focus on stereo matching in this paper. After years of development some achievements have been made in stereo matching, but some problems remain unsolved. The two most important things concerned in stereo vision trend to be contradictory: accuracy and efficiency. This paper presents a method called Advanced-Census which is a good combination of SAD and Census. When finding corresponding pixel in left and right image, SAD can get high accuracy with low speed and Census have the opposite result. Advanced-Census has advantages of both SAD and Census. It retains the speed of Census while having the accuracy of SAD. Although it has the speed of Census but not enough for practical applications, so we speed up Advanced-Census using multi-thread technique and edge detection. After speeding up, Advanced-Census gets nearly real-time performance.