Optimization Design of Camera Pose for Network Base on Multi-view Stereo Vision

Binocular stereo vision (BSV) system has been widely used in various fields, such as intelligent manufacture, smart robot, and so on. However, the location accuracy of the current BSV still cannot fully satisfy industry requirements due to lack of a parameters optimization BSV system. In this paper, a high accuracy BSV system is proposed. This is achieved through analyzing the seven parameters of the BSV system, which are classified into two groups: system structure parameters (SSPs) and camera calibration parameters (CCPs). For the SSPs, an improved analysis model is designed to expose the possible errors caused by three parameters. Furthermore, a new correlation model among them is also proposed to analyze the errors caused by their correlation. On the other hand, for the CCPs, the orthogonal experiment model is employed for selecting the optimal combination of the four calibration parameters. Meanwhile, the weight among the four parameters is also analyzed for reducing errors. Finally, the effectiveness of our proposed method is demonstrated by a large number of experiments. It gives a useful reference to the BSV system used in applied optics research and application fields.

For volume measurement problems in the processes of additive manufacturing, a volume measurement method is proposed through using the SGBM algorithm in this paper. In this method, the internal and external parameters of the camera are calibrated by the binocular vision system. And the image processes by SGBM are performed for the image acquired by binocular stereo vision system. The processes are including filtering, correction and stereo matching to obtain the disparity information of the object to be tested in the left and right cameras. The volume of the work-piece to be tested can be obtained by the calculation that measured the three-dimensional coordinate information through the disparity information. The experimental results illustrate that the method is of certain reliability and accuracy for volume measurement.

Computer stereo vision has been paid high attention in areas such as industry measurement and automatic control. Calibration of stereo vision system is a key step to implement vision measurement. Nonlinear calibration method involves too many parameters and too complex function, so the processing of optimization is difficult and slow. In this paper, two digital cameras are used to set up stereo vision system which provides convenient experimental operation. The binocular stereo vision system is calibrated using linear calibration model. The results indicate that the method is stable and fast with high precision and practicability.

Binocular stereo vision technology is widely used in the 3D measurements of aerospace, the automotive industry, and in other fields. However, its measurement accuracy is limited by many error factors and is difficult to improve. This paper presents a modeling method for the measurement accuracy of binocular stereo vision systems, providing a theoretical basis for system optimization design and accuracy improvement. Firstly, the main factors affecting the measurement accuracy of binocular stereo vision system are quantitatively analyzed, including system structural parameters, camera distortion, marker point center positioning errors, focal length, etc. Then, based on the principles of uncertainty calculation and spatial point 3D reconstruction, the measurement error transfer function is derived. By obtaining the errors of various parameters in the calibration process, this transfer function can be used to accurately predict the error at any positions within the measurement volume. Finally, the effectiveness of the error transfer function in predicting measurement error is verified through standard point distance measurement experiments.

Binocular stereo vision is an important branch of the research area in computer vision. Stereo matching is the most important process in binocular vision. In this paper, a new stereo matching scheme using shape-based matching (SBM) is presented to improve the depth reconstruction method of binocular stereo vision systems. The method works in two steps. First, an operator registers the pattern including the key features of an object to be measured. Then during the operation stage, the stereo camera snaps stereo images and finds the patterns in right and left images separately by means of the SBM. The 3D positions of the object are calculated by using the corresponding points of the stereo images and the projection matrices of the stereo camera. Since we apply robust image processing algorithms, such as the SBM, the proposed method becomes more reliable than the conventional stereo vision systems.

Vision system is very important for a humanoid robot to work in the human working and living environments. This paper discusses a humanoid robot binocular stereo visual system, with two computers the humanoid robot will perfect realise target object location and manipulation in real-time, the two computers are linked with each other through memolink communication module. One computer is responsible for processing the object images information, the other is used to control humanoid robot to walk and manipulate. An active humanoid robot vision model is built; the 3D location principle and the calibration method of this model are analysed. This active robot vision system with two degrees of freedom enlarges its visual field and the binocular stereo vision system is the most simple camera configuration for 3D position information. Experiments show the effectiveness and robustness of the stereo visual system for a humanoid robot.

With the increasing need for higher accuracy measurement in computer vision, the precision of camera calibration is a more important factor. The objective of stereo camera calibration is to estimate the intrinsic and extrinsic parameters of each camera. We presented a high-accurate technique to calibrate binocular stereo vision system having been mounted the locations and attitudes, which was realized by combining nonlinear optimization method with accurate calibration points. The calibration points with accurate coordinates, were formed by an infrared LED moved with three-dimensional coordinate measurement machine, which can ensure indeterminacy of measurement is 1/30000. By using bilinear interpolation square-gray weighted centroid location algorithm, the imaging centers of the calibration points can be accurately determined. The accuracy of the calibration is measured in terms of the accuracy in the reconstructing calibration points through triangulation, the mean distance between reconstructing point and given calibration point is 0.039mm. The technique can satisfy the goals of measurement and camera accurate calibration.

This paper presents a novel extrinsic calibration algorithm between a binocular stereo vision system and a 2D LIDAR (laser range finder). Extrinsic calibration of these heterogeneous sensors is required to fuse information obtained separately by vision sensor and LIDAR in the context of intelligent vehicle. By placing a planar chessboard at different positions and orientations in front of the sensors, the proposed method solves the problem based on 3D reconstruction of the chessboard and geometric constraints between views from the stereovision system and the LIDAR. The three principle steps of the approach are: 3D corner points triangulation, 3D plane least-squares estimation, solving extrinsic parameters by applying a non-linear optimization algorithm based on the geometric constraints. To evaluate the performance of the algorithm, experiments based on computer simulation and real data are performed. The proposed approach is also compared with a popular calibration method to show its advantages.

In vision system of laser remanufacturing robot, binocular stereovision system combining structured light technique is used, Which applies to measure Laser Remanufacturing workpieces. The vision system which includes two cameras, a laser balopticon and a computer is fitted on the bottom of the robot, and its position is controlled by robot teach pedant. The image collected by cameras is transferred to a computer. Calibration and matching is the key to the work of the visual system. Calibration, including camera calibration and robot hand-eye calibration, is completed by MATLAB programming. The method of combining structured light with binocular stereo vision system is studied: let line structured light scanning the surface of the work piece, collect image, extract sub-pixel center of structured light by gravity algorithm and complete the image matching by epipolar geometry constraint relations. The process can be completed by MATLAB. The results of the 3D measurements of the work piece show that the system can make the robot recognize the defects and the shape of the work piece automatically, so that the vision system can satisfy the requirements auto-detection remanufacturing.

In order to obtain the phenotypic parameters of apple quickly and accurately, which were commonly used as the basis of fruit sorting, a fast estimation method of apple phenotypic parameters based on three-dimensional (3D) reconstruction was proposed in this study. In this study, a three-dimensional model was constructed to estimate the phenotypic parameters of apple, such as volume, height, diameter, and fruit shape index. Firstly, an image acquisition system was built to capture sequence images of fruit with a binocular stereo vision system, and the images were extracted and matched using the Accelerated-KAZE algorithm to create the point cloud data. Secondly, the point cloud data were matched with the algorithm of Iterative Closest Point to establish a whole model of apple, and the surface reconstruction model of fruit was obtained by constructing irregular triangulation network. Finally, the apple phenotypic parameters were calculated by means of segmentation, surface complement and integral of the fruit model. Total of 200 apples were used as samples in the experiment. By this method, the phenotypic parameters of the apples were estimated based on their 3D reconstruction model, and the linear regression analysis was carried out between the estimated values and the real values. The results showed that R2 of the linear regression fitting of each parameter was higher than 0.90. Among them, the fitting of volume was the best with R2 of 0.97. In addition, the average errors of apple volume, height, fruit shape index, maximum diameter D and minimum diameter d were 8.73 cm3, 1.43 mm, 1.28%, 0.90 mm, and 1.23 mm, respectively. According to the Chinese national standard of “fresh Apple”, the average error of the estimated result is within the range of allowable error. It indicates that the method of apple phenotypic parameter estimation based on 3D reconstruction has a high accuracy and practicability, and it can be used as the support for fruit sorting. Keywords: apple, 3D reconstruction, phenotypic parameter, stereo vision system, sequence image, sorting DOI: 10.25165/j.ijabe.20211405.6258 Citation: Ma H, Zhu X, Ji J T, Wang H, Jin X, Zhao K X. Rapid estimation of apple phenotypic parameters based on 3D reconstruction. Int J Agric & Biol Eng, 2021; 14(5): 180–188.

For filling the application requirement of searching toxic gas leak source, a multi-sensory robot, which is used for searching toxic gas leak source, is developed. This robot consists of humanoid multiple degree of freedom head system and tracked walking mechanism, with the sense of smell, hearing and stereo vision and other functions. The robot’s bionic olfaction system can detect 5 kinds of toxic gas in the same time, and the hearing system consisted of four microphones can get all kinds of sound in the general environment. Meanwhile, its binocular stereo vision system can get the 3D information of the work environmental. Distributed controll system is constructed for the robot, and comprehensive utilization of the multi-sensory function can increase the efficiency and accuracy of searching leakage. This robot can give full play the advantage of multiple degrees of freedom to expand the searching scope of the robot, and enhance the flexibility of the searching behavior. This robot system has wide application prospects in the environment of toxic gas.

Road visual navigation relies on accurate road models. This study was aimed at proposing an improved scale-invariant feature transform (SIFT) algorithm for recovering depth information from farmland road images, which would provide a reliable path for visual navigation. The mean image of pixel value in five channels (R, G, B, S and V) were treated as the inspected image and the feature points of the inspected image were extracted by the Canny algorithm, for achieving precise location of the feature points and ensuring the uniformity and density of the feature points. The mean value of the pixels in 5×5 neighborhood around the feature point at an interval of 45o in eight directions was then treated as the feature vector, and the differences of the feature vectors were calculated for preliminary matching of the left and right image feature points. In order to achieve the depth information of farmland road images, the energy method of feature points was used for eliminating the mismatched points. Experiments with a binocular stereo vision system were conducted and the results showed that the matching accuracy and time consuming for depth recovery when using the improved SIFT algorithm were 96.48% and 5.6 s, respectively, with the accuracy for depth recovery of –7.17%-2.97% in a certain sight distance. The mean uniformity, time consuming and matching accuracy for all the 60 images under various climates and road conditions were 50%-70%, 5.0-6.5 s, and higher than 88%, respectively, indicating that performance for achieving the feature points (e.g., uniformity, matching accuracy, and algorithm real-time) of the improved SIFT algorithm were superior to that of conventional SIFT algorithm. This study provides an important reference for navigation technology of agricultural equipment based on machine vision. Keywords: scale-invariant feature transform (sift), feature matching, canny operator, energy method of feature point, farmland road, depth recovery, visual navigation DOI: 10.25165/j.ijabe.20191204.4821 Citation: Yao L J, Hu D, Yang Z D, Li H B, Qian M B. Depth recovery for unstructured farmland road image using an improved SIFT algorithm. Int J Agric & Biol Eng, 2019; 12(4): 141–147.

The timber beam end effect in the torsion test is<br> evaluated using binocular stereo vision system. It is recommended by<br> BS EN 408:2010+A1:2012 to exclude a distance of two to three times<br> of cross-sectional thickness (b) from ends to avoid the end effect;<br> whereas, this study indicates that this distance is not sufficiently far<br> enough to remove this effect in slender cross-sections. The shear<br> modulus of six timber beams with different aspect ratios is determined<br> at the various angles and cross-sections. The result of this experiment<br> shows that the end affected span of each specimen varies depending<br> on their aspect ratios. It is concluded that by increasing the aspect<br> ratio this span will increase. However, by increasing the distance<br> from the ends to the values greater than 6b, the shear modulus trend<br> becomes constant and end effect will be negligible. Moreover, it is<br> concluded that end affected span is preferred to be depth-dependent<br> rather than thickness-dependant.

本文基于双目立体视觉测距原理，用普通的USB摄像头搭建双目立体视觉系统，提出了图像面积的非接触在线测量方法。首先对摄像头进行标定，得到摄像头的内参数，然后根据OpenCV的模板匹配算法，求出两幅图像中对应匹配点的水平视差，得到距离参数，最后根据平行双目视觉的成像原理，求得实际图像与其在成像平面上的图像之间的空间几何关系，进而求出图像的实际面积。该方法实现简单、成本低、实时性好。经过实验验证，该算法在一定测量范围内达到精度要求。 A binocular stereo vision system with a common USB camera is developed based on the distance measurement principle of binocular stereo vision, and the non-contact and on-line inspection method for the area of image is studied. First, we calibrate the camera and obtain the internal parameters of the camera. Second, horizontal parallax of the cor- responding points in two images is calculated by template matching based on OpenCV, and then the distance parameter is obtained. Finally, we find the spatial geometrical relationship between the image of the imaging plane and the actual image, and then the actual area of image is gotten. This method is easy to realize with low cost and real-time perform- ance. The experiment indicates that the algorithm meets the accuracy requirements within certain range of measurement.

Summary5G virtual reality technology is a technology that has attracted widespread attention in the computer field in recent years. Establishing a harmonious human–computer interaction environment in the multidimensional information space is one of the development goals of information technology. Face information is important information for identifying the identity of others. Three‐dimensional information has important applications in many fields and directions of machine vision. Therefore, the three‐dimensional common face image has a significant meaning. In this paper, binocular stereo vision system is used to collect multiple pairs of facial information images. The image correction method was used to correct the collected image pairs. According to the basic criteria of stereo matching, the advantages and disadvantages of various traditional image matching methods are analyzed. On the basis of completing the image correction, the graph cut matching method is selected as the matching method, and multiple pairs of collected image pairs are matched by this method to obtain a multi‐angle face disparity map. In this paper, after studying the basic dense matching algorithm and its improved algorithm, the SAD, SD, and NC methods of fixed window are used to compare the matching cost and the performance of the adaptive weight method in face matching. This paper studies the virtual roaming technology and the basis of the related Internet of Things (IoT) 3D face graphics theory and develops a building virtual roaming system based on MFC and OpenGL. The system draws on the basic I/0 device of the computer and provides users with a friendly MFC interface and multiple viewing angles and multiple roaming mode options for virtual faces. Experimental research shows that the reconstruction accuracy of the spatial points of the reconstructed face and the actual situation are both about 1%, which shows that the IoT 3D face modeling algorithm designed in this paper has high accuracy.