Initializing and accelerating Stereo-DIC computation using semi-global matching with geometric constraints

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

The primary objective of this study is to provide a methodology to generate a dense point cloud of digital surface model (DSM) and digital terrain model (DTM) from 0.6 m GSD stereo images acquired by CARTOSAT-2E satellite of the Indian Space Research Organization. These products are required for many high-resolution applications such as mapping of watersheds and watercourses; river flood modeling; analysis of flood depth, landslide, forest structure, and individual trees; design of highway and canal alignment. The proposed method consists of several processes such as orienting the stereo images, DEM point cloud extraction using the semi-global matching, and DSM to DTM filtering. The stereo model is built by performing aero triangulation and block adjustment using the ground control points. The semi-global matching algorithm is used on the epipolar images to generate the DSM in the form of dense point cloud corresponding to one height point for each pixel. The planimetric and height accuracies are evaluated using orthoimages and DSM and found to be within 3-pixel (~ 2 m) range. A method for extracting DTM by ground point filtering, to discriminate the probable ground points and the non-ground points, is provided by using discrete cosine transformation interpolation. This robust method uses a weight function to differentiate the noise points from the ground points. The overall classification efficiency kappa (κ) value averages at 0.92 for ground point classification/DTM extraction. The results of benchmarking of the GPS-aided GEO augmented navigation GPS receiver by operating it over IGS station, in static mode for collecting the checkpoints, also are presented.

An asymmetric post-processing for correspondence problem

Mobile stereo matching systems are becoming more important in many applications such as auto-driving and autonomous robots. However, to maintain its low power consumption, mobile platforms have only limited hardware resources. Accurate stereo matching methods require a high computational complexity, and it is difficult to maintain both acceptable accuracy and processing speed on the mobile platforms. To solve this trade-off, in this paper, we propose a novel acceleration approach for a well-known matching algorithm Zero-means Normalized Cross Correlation (ZNCC), and show its effectiveness on a Jetson TX2 embedded GPU. By combining our new approach, Z2- ZNCC, with the Semi-Global Matching (SGM) algorithm, our system achieves a low error rate of 7.76% while keeping 28 fps for 1242×375 pixels images with the maximum disparity of 128 on the KITTI 2015 dataset. This performance is higher than previous state-of-the-art system on the same hardware platform.

Image segments are often used as a constraint in stereo matching. However, both over-segmentation and under-segmentation can lead to disparity degradation in some regions. To obtain an accurate disparity map, a modified semi-global matching (SGM) algorithm is proposed which is based on adaptive window models. Introducing the object notion and build a new global energy function to optimise segments and estimate a disparity map jointly. The effective and efficient block coordinate descent approach is used to optimise the global energy function by merging small segments. The authors’ demonstrate the performance of the proposed algorithm on the KITTI and Middlebury benchmarks. The results show that the authors’ algorithm outperforms many state-of-the-art methods and confirm the effectiveness of approach.

The pantograph is an essential component in modern electrified trains. Due to the characters of contactless and high precision, the 3-D stereo vision, which can effectively monitor the abrasion, is of great significance for ensuring train safety. However, illumination conditions, such as overbrightness and insufficient light, have a great impact on semiglobal matching (SGM) stereo matching, inevitably affecting the integrity and accuracy of the abrasion measurement of the pantograph slipper. In this regard, a robust SGM stereo matching is proposed in this article for the 3-D abrasion measurement. First, the combination of the neighborhood information-aware census cost and multiscale fusion for cost aggregation is put forward. While insensitive to illumination condition changes, the improved census cost can quantify the texture information of the neighborhood. Moreover, multiscale cost aggregation further fuses the neighborhood information of different scales. As a result, the performance of SGM in weakly textured areas is greatly improved, especially in complex illumination conditions. Eventually, with the help of the subpixel disparity refinement and local points fitting, an accurate 3-D abrasion calculation is achieved. Practical experiments show the robustness of stereo matching. The precision verification and computation time analysis confirm the measurement accuracy and applicability.

In many cases, the Digital Surface Models (DSMs) and Digital Elevation Models (DEMs) are obtained with Light Detection and Ranging (LiDAR) or stereo matching. As an active method, LiDAR is very accurate but expensive, thus often limiting its use in small-scale acquisition. Stereo matching is suitable for large-scale acquisition of terrain information as the increase of satellite stereo sensors. However, underperformance of stereo matching easily occurs in textureless areas. Accordingly, this study proposed a Shading Aware DSM GEneration Method (SADGE) with high resolution multi-view satellite images. Considering the complementarity of stereo matching and Shape from Shading (SfS), SADGE combines the advantage of stereo matching and SfS technique. First, an improved Semi-Global Matching (SGM) technique is used to generate an initial surface expressed by a DSM; then, it is refined by optimizing the objective function which modeled the imaging process with the illumination, surface albedo, and normal object surface. Different from the existing shading-based DEM refinement or generation method, no information about the illumination or the viewing angle is needed while concave/convex ambiguity can be avoided as multi-view images are utilized. Experiments with ZiYuan-3 and GaoFen-7 images show that the proposed method can generate higher accuracy DSM (12.5–56.3% improvement) with sound overall shape and temporarily detailed surface compared with a software solution (SURE) for multi-view stereo.

The Census transform is very useful in stereo matching algorithms. But the traditional Census transform cost too much computation, and its accuracy and robustness are not satisfying. In this paper, a modified semi-global matching (SGM) algorithm based on the fast Census transform is proposed to improve the quality of the range map and decrease the computation. In addition, we modify the disparity refinement to enhance the quality of the estimated range map. Our experiments show that the proposed method performs better than the original SGM algorithm in terms of the accuracy and the anti-noise capability.

SGM-based seamline determination for urban orthophoto mosaicking

Dense stereo image matching in remotely sensed images is a challenging problem, though it has been studied for more than two decades, due to occlusions, discontinuities, geometric, and radiometric distortions. A novel multiobjective fitness function-based dense stereo matching approach using genetic algorithms (GAs) is proposed in this paper. The proposed method is useful for estimating dense disparity map with an improved number of inliers for a stereo image pair, despite the constraint of finding correct disparity at depth discontinuities. In this paper, the steps of GA, such as initialization of the population, fitness function, and crossover and mutation operation, are designed and implemented to effectively deal with the problem of dense stereo image matching. To initialize the population, a Scale Invariant Feature Transform (SIFT) descriptor is computed for each pixel and multiple-size window-based matching is performed, using the similarity measures: 1) Euclidean distance and 2) spectral angle mapper. The generated disparity maps are pruned to choose a suitable subset using the designed fitness functions, considering the constraints related to stereo image pair, such as epipolar constraint, which encodes the epipolar geometry and the similarity measure that is useful to decide accuracy of the correspondences. The two objective functions are the number of inliers computed using the fundamental matrix and an energy minimization function, considering discontinuities and occlusions. The usefulness of this approach for remotely sensed stereo image pairs is demonstrated by improving the number of inliers and favorably comparing with state-of-the-art dense stereo image matching methods.

This paper presents a stereo image coding method. We first use a new stereo matching method with difference-based adaptive searching windows to get the fine disparity. Then, we encode the disparity instead of one channel of the stereo image while the other channel is encoded normally. The stereo matching employs the continuity of difference of stereo images to compute the size of adaptive searching windows, which is superior to the approaches based on fixed searching windows. Experimental results show that the proposed scheme can provide better performance for stereo image coding.

The authors present parallel techniques to implement two vision tasks; stereo matching and image matching using linear features as primitives. The implementations are performed on a fixed size linear array and achieve processor-time optimal performance. For stereo matching, they propose an O(Nn/sup 3//P) time algorithm on a P-processor linear array, where N is the number of line segments in one image, n is the number of line segments in a window determined by the object size, and P >

Stereo vision finds a wide range of applications in automotive, object detection, robot navigation, agriculture mapping, and others. Stereo matching is a stereo algorithm targeted to identify the corresponding pixels from two or more images. This study shows the implementation of stereo matching using the Sum of Absolute Difference (SAD) algorithm to extract the object's depth or disparity from stereo images. Our key objective revolves around the implementation of the stereo matching algorithm for a small field-programmable gate array (FPGA)—requiring relatively few resources while maintaining the processing speed as well as disparity map. For meeting this requirement, we used small window buffers to compute the stereo matching. The occluded pixels were reduced by introducing secondary consistency checking implementation. From the results of experiments performed using the Zynq UltraScale+ ZCU102 FPGA board with SDSoC compiler, the processing speed for computing the stereo matching algorithm with a 4×4 window buffer was 0.038 s for an image size of 486×720 pixels and 0.051 s for 375×1242 pixels resolution. The proposed design needed 1% each of BRAM and FF and 7% of LUT. An 18% reduction in the pixel errors has been observed when employing the secondary consistency matching on the post-processing.

Efficiency and accuracy of semi-global matching (SGM) make it outperform many stereo matching algorithms and is widely used under challenging occasions. However, SGM only incorporates information along a scanline in each pass and lacks interaction between scanlines, resulting in streak artifacts in the disparity image. We introduce a local edge-aware filtering method to SGM to enhance the interaction of neighboring scanlines, since streak artifacts can be avoided. We use bilateral weights based on intensity similarity and spatial affinity between pixels to build connections among scanlines. In each pass, we recursively estimate the aggregated cost of SGM and compute the weighted average of aggregated costs for pixels in the orthogonal direction to obtain the output of our method along each scanline. As one-dimensional bilateral filtering is used in our method, the extra computation is linear to image resolution and label space, which is a small fraction of that needed by SGM. We present ablation studies using stereo pairs under both constrained and natural conditions to verify the effectiveness of our method. Extensive experiments on Middlebury and Karlsruhe Institute of Technology and Toyota Technology Institute datasets demonstrate that our method removes all streak artifacts, improves the quality of the disparity image, and outperforms many other non-local cost aggregation approaches.

LiDAR-guided stereo matching with a spatial consistency constraint