GPU-based Parallelization of a Sub-pixel High-resolution Stereo Matching Algorithm for High-throughput Biomass Sorghum Phenotyping

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

A hierarchical stereo matching algorithm based on adaptive support region aggregation method

Virtually all land plants are coated in a cuticle, a waxy polyester that prevents nonstomatal water loss and is important for heat and drought tolerance. Here, we describe a likely genetic basis for a divergence in cuticular wax chemistry between Sorghum bicolor, a drought tolerant crop widely cultivated in hot climates, and its close relative Zea mays (maize). Combining chemical analyses, heterologous expression, and comparative genomics, we reveal that: 1) sorghum and maize leaf waxes are similar at the juvenile stage but, after the juvenile-to-adult transition, sorghum leaf waxes are rich in triterpenoids that are absent from maize; 2) biosynthesis of the majority of sorghum leaf triterpenoids is mediated by a gene that maize and sorghum both inherited from a common ancestor but that is only functionally maintained in sorghum; and 3) sorghum leaf triterpenoids accumulate in a spatial pattern that was previously shown to strengthen the cuticle and decrease water loss at high temperatures. These findings uncover the possibility for resurrection of a cuticular triterpenoid-synthesizing gene in maize that could create a more heat-tolerant water barrier on the plant's leaf surfaces. They also provide a fundamental understanding of sorghum leaf waxes that will inform efforts to divert surface carbon to intracellular storage for bioenergy and bioproduct innovations.

Assuming that neighboring pixels with similar colors in stereo images, share almost the same disparities, this paper presents a new global stereo matching algorithm based on Belief Propagation and cross-based aggregation method. In this approach the hierarchical Belief Propagation strategy is followed by a left-right consistency check as an initial match. Then a refinement algorithm is applied to generated disparity map, based on cross-based region method. The initial matching performed by hierarchical Belief Propagation strategy uses less memory and improves the running speed of energy minimization function. The experimental results, evaluated by the Middlebury data sets, show that the proposed method improves the accuracy of disparity map compare to other real-time stereo matching algorithms.

Stereo matching using weighted dynamic programming on a single-direction four-connected tree

A fast stereo matching algorithm suitable for embedded real-time systems

The area of computer vision is one of the most discussed topics amongst many scholars, and stereo matching is its most important sub fields. After the parallax map is transformed into a depth map, it can be applied to many intelligent fields. In this paper, a stereo matching algorithm based on visual sensitive information is proposed by using standard images from Middlebury dataset. Aiming at the limitation of traditional stereo matching algorithms regarding the cost window, a cost aggregation algorithm based on the dynamic window is proposed, and the disparity image is optimized by using left and right consistency detection to further reduce the error matching rate. The experimental results show that the proposed algorithm can effectively enhance the stereo matching effect of the image providing significant improvement in accuracy as compared with the classical census algorithm. The proposed model code, dataset, and experimental results are available at https://github.com/WangHewei16/Stereo-Matching.

The objective of intercropping is to improve resource efficiencies for land, water and radiation. Few studies have focused on the interactive effects of water stress on radiation use efficiency (RUE) in an intercrop system. The study determines the effect of intercropping sorghum with either cowpea or bottle gourd on RUE and biomass accumulation in response to water stress. This was assessed using a split-plot design with crop sequences sole sorghum, sole cowpea, and sole bottle gourd and intercrop systems sorghum–cowpea and sorghum–bottle gourd assigned as sub-plots arranged in randomised complete blocks within main plots, water regimes: full irrigation, deficit irrigation and rainfed. We quantified specific leaf area, leaf area index as well as biomass accumulation and partitioning. Extinction coefficient, intercepted photosynthetic active radiation (IPAR) and RUE for biomass (RUEb) and grain (RUEg) were also determined. Under rainfed conditions, intercropping with either cowpea (26%) or bottle gourd (62%) improved sorghum leaf area. A reduction in specific leaf area was observed for sorghum when it was intercropped with either cowpea (−20%) or bottle gourd (−19%). Under full irrigation, sorghum stems constituted a larger proportion (62%) of final biomass in comparison to sorghum under deficit irrigation (52%) and rainfed conditions (50%). Intercropping sorghum with either cowpea or bottle gourd improved IPAR (38%), RUEb (93%) and RUEg (11%) under rainfed conditions. This study showed that under water-limited conditions intercropping reduced sorghum stem mass and increased canopy size, radiation interception, and grain yield and improved RUE. In agreement with our proposition, water availability was closely associated with radiation use efficiency. Intercropping can be recommended for semi-arid environments since it improves RUE and sorghum biomass and grain yield.

Typically, algorithms for generating stereo disparity maps have been developed to minimise the energy equation of a single image. This paper proposes a method for implementing cross validation in a belief propagation optimisation. When tested using the Middlebury online stereo evaluation, the cross validation improves upon the results of standard belief propagation. Furthermore, it has been shown that regions of homogeneous colour within the images can be used for enforcing the so-called "segment constraint". Developing from this, segment support is introduced to boost belief between pixels of the same image region and improve propagation into textureless regions

Depth map super-resolution using stereo-vision-assisted model

Most image denoising methods process each noise-corrupted pixel from the top-left to the bottom-right of the images using a sliding window. By firstly processing a noise-corrupted pixel with plenty of noise neighbor pixels in a local window may deteriorate the quality of subsequent pixels, enabling the quality of the denoised image to be reduced. In this paper, we present a method to change the process order on noise-corrupted pixels to improve the performance of bio-image denoising according to the confidence measure. If the center pixel of a local window with a non-extreme gray value (the pixel value is neither 0 nor 255 for an 8-bit gray bio-image) represents a noise-free pixel, no processing is performed. Conversely, the gray level of the center pixel is modified by a restored value. Two confidence measures are used to determine the order of producing the restored value, including direction confidence and noise-free confidence. An analysis window both with a greater quantity of noise-free pixels and with a consistent pixel change direction is defined as a high confidence region which will be processed firstly. If the variation direction of a pixel is consistent with the neighbor pixels, directional mean filtering is performed. Conversely, median filtering is performed for the pixels with low confidence where the quantity of noise-free pixels is low in a local window or the directions of pixel changes are inconsistent. The experimental results show that the proposed method can further improve the performance of an image denoising method which utilizes the sliding window from the top-left to the bottom-right. The major reason is because of the prior restoration of the noise-corrupted pixels in high confidence regions. These restored pixels are subsequently employed to restore the noise-corrupted pixels with low confidence, resulting in the quality of restored bio-image being improved.

The novel framework for estimating dense scene flow using depth camera data is demonstrated in the article. Using these estimated flow vectors to identify obstacles improves the path planning module of the autonomous vehicle's (AV) intelligence. The primary difficulty in the development of AVs has been thought to be path planning in cluttered environments. These vehicles must possess the intelligence to recognize their surroundings and successfully navigate around obstacles. The AV needs a thorough understanding of the surroundings to detect and avoid obstacles in a cluttered environment. Therefore, when determining the course, it is preferable to be aware of the kinematic behavior (position and the direction) of the obstacles. As a result, by comparing the depth images between different time frames, the position and direction of the obstacles are calculated using a 3D vision sensor. The current study focuses on the extraction of the flow vectors in 3D coordinates from the differential scene flow method. Generally, the evaluation of scene flow algorithms is crucial in determining their accuracy and effectiveness in different applications. The gradient of the vector field snake model, which extracts changes in pixel values in three directions, is combined with the scene flow technique to identify both static and dynamic obstacles. Our goal is to create a single-vision sensor-based real-time obstacle avoidance method based on scene flow estimation. In addition, the common evaluation metrics such as endpoint error (EPE), average angular error (AAE), and standard deviation angular error (STDAE) are used to measure the accuracy of different algorithms in terms of computational errors with the benchmark Middlebury datasets. The proposed technique is validated with different experiments using a Pixel-Mixed-Device (PMD) camera and a Kinect sensor as 3D sensors. Finally, the numerical and experimental results are displayed and reported.

<b><sc>Abstract.</sc></b> <b>In recent years, three-dimensional (3D) sensing has gained a great interest in plant phenotyping because it can represent the 3D nature of plant architecture. Among all available 3D imaging technologies, stereo vision offers a viable solution due to its high spatial resolution and wide selection of camera modules. However, the performance of in-field stereo imaging for plant phenotyping has been adversely affected by textureless regions and occlusions of plants, and variable outdoor lighting and wind conditions. In this research, a portable stereo imaging module namely PhenoStereo was developed for high-throughput field-based plant phenotyping. PhenoStereo featured a self-contained embedded design, which made it capable of capturing images at 14 stereoscopic frames per second. In addition, a set of customized strobe lights was integrated to overcome lighting variations and enable the use of high shutter speed to overcome motion blurs. The stem diameter of sorghum plants is an important trait for stalk strength and biomass potential evaluation but has been identified as a challenging sensing task to automated in the field due to the complexity of the imaging object and the environment. To that connection, PhenoStereo was used to acquire a set of sorghum plant images and an automated point cloud data processing pipeline was also developed to automatically extract the stems and then quantify their diameters via an optimized 3D modeling process. The pipeline employed a Mask R-CNN deep learning network for detecting stalk contours and a Semi-Global Block Matching stereo matching algorithm for generating disparity maps. The correlation coefficient (r) between the image-derived stem diameters and the ground truth was 0.97 with a mean absolute error (MAE) of 1.44 mm, which outperformed any previously reported sensing approaches. These results demonstrated that with proper customization stereo vision can be a highly desirable sensing method for field-based plant phenotyping using high-fidelity 3D models reconstructed from stereoscopic images. With the proving results from sorghum plant stem diameter sensing, this proposed stereo sensing approach can likely be extended to characterize a broad spectrum of plant phenotypes such as leaf angle and tassel shape of maize plants and seed pods and stem nodes of soybean plants.</b>

Outlier detection and disparity refinement in stereo matching

Real-time 3D sensing has many important applications in areas such as robotic navigation, virtual reality and human-computer interaction. A variety of techniques have been developed for the determination of 3D geometry information such as binocular vision, structured light and their combination. However, existing non-contact optical 3D sensing approaches have their own limitations in the process of 3D information calculation. The reliability of binocular vision is limited to textures of the object surface, and the measuring accuracy of the method based on structured-light projection is limited to the stability of the light generator and the number of projected images. In this paper, we will combine random speckle projection with stereo matching algorithms to study the related problems on 3D measurement in the scene. An unique random speckle pattern is projected to encode the object surface to reduce the influence of projector flickering, and improve the accuracy of stereo matching. Besides, we take advantage of temporal consistency to reduce the range of disparity updating to improve the speed of 3D sensing further. The tests performed on real captured images confirm the validity of our approach.